Report 2017-129 Recommendations

To avoid bias or the perception of bias, the Legislature should enact legislation that prohibits state agencies from selecting as an evaluator of grant applications a representative, former member, or former staff of any organization or person that is applying to receive grant funding from the state agency.

To comply with federal and state requirements, and to ensure consistency and fairness in its grant process, Rehabilitation should issue regulations describing its grant process from RFA development through appeals. It should submit its proposed regulations to the Office of Administrative Law no later than December 2018.

To ensure that management and staff involved in the grant process are sufficiently informed about the process and their responsibilities, Rehabilitation should require these employees to attend a kickoff meeting before the development of each RFA in which participants discuss the key stages of the grant review process, each individual's roles and responsibilities, and requirements surrounding conflicts of interest and confidentiality. Further, it should record these discussions in meeting minutes to ensure that expectations of employees are clearly defined and documented.

To comply with state laws and regulations and help ensure that staff involved in making governmental decisions during the grant process are impartial, Rehabilitation should ensure that they receive ethics training, which includes conflict-of-interest training, at least every two years.

To help ensure that staff involved in the grant process adequately protect confidential information, Rehabilitation should develop confidentiality procedures for each grant. Further, it should ensure that staff involved in the grant process sign the conflict-of-interest and confidentiality forms before the development of the RFA for each grant.

To ensure that it has received sufficient input and feedback from the disability community to inform the development of RFAs, Rehabilitation should solicit and document stakeholder input and feedback before and during the development of each RFA.

To increase transparency and ensure that applicants have the information necessary to understand the grant process, Rehabilitation should include in its RFAs clear scoring criteria and descriptions of the evaluation, award, and appeals processes, including the process it will use to address applications that receive tied scores.

To ensure that Rehabilitation maintains all relevant grant documentation and responds fully to requests for public records, it should immediately adhere to its records retention policy and save all grant-related documents, including email correspondence and attachments, to a centralized location.

To ensure consistency and fairness in the evaluation process, Rehabilitation should make sure that it accepts only complete applications submitted before the deadline, unless otherwise specified in the RFA. If the RFA specifies a hard deadline and applicants submit incomplete applications, Rehabilitation should not accept any portions of the applications submitted after the deadline and should assess the penalty for incomplete applications specified in the RFA.

To help ensure that evaluators adequately protect confidential information and that the evaluation process is fair, Rehabilitation should develop standardized evaluator training for confidentiality procedures and conflicts of interest, including a discussion of bias or the appearance of bias. Rehabilitation should also ensure that the candidates receive this training and sign conflict-of-interest and confidentiality forms before it selects evaluators. Further, it should prohibit program staff who participate in the development of an RFA from acting as evaluators for the applications Rehabilitation receives in response to that RFA.

To increase the transparency of its selection process and to ensure that it receives the most qualified evaluators possible, Rehabilitation should issue a public solicitation for evaluators for each grant that includes a description of essential and desirable qualifications.

To ensure that evaluators have the information necessary to sufficiently and fairly assess and score applications, Rehabilitation should develop training by December 2018 that can be tailored to each grant and includes at minimum the following topics:

-The purpose and relevant regulatory requirements for the grant.

- Instructions on how to score applications, including an applicant's financial information, and direction that they must provide comments to support their scores.

Rehabilitation should provide this training to evaluators before allowing them to score applications.

To ensure that it provides sufficient oversight of the grant process, Rehabilitation should ensure that the technical review teams it assigns to grants provide the director and chief deputy with a memorandum summarizing the evaluation process and the evaluators' recommended grant awardees. Rehabilitation should also designate an individual responsible for reviewing and approving the memorandum and recommended awardees before it publishes its notice of intent to award.

If it finds errors in an evaluation that merit restarting the grant process, rescoring of applications, or convening a new evaluation panel, Rehabilitation should resolve any issues before it begins the rescoring process. It should also notify applicants to ensure that they are aware of any changes to the process due to the errors. Further, it should consider promulgating regulations and amending its grant manual to permit staff to request evaluators to rescore applications or convene a new evaluation panel when it finds issues with an evaluation.

To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual that staff at the appropriate level of authority are to acknowledge all appeal requests, notify intended awardees that could be affected by the appeals, and inform the appellant of the qualifications of the review committee members. Staff at the appropriate level of authority must also notify all affected parties of the review committee's final decision within the time frame Rehabilitation establishes in regulations.

To ensure that Rehabilitation has appropriate oversight of its grant process and can sufficiently demonstrate that it followed the process, it should designate staff, separate from those involved in the respective grant process, to conduct a review of each grant process for procedural errors, evaluator prejudice, and whether evaluators supported their scores with evidence from the relevant applications before it awards grants.

To comply with federal and state requirements, and to ensure consistency and fairness in its grant process, Rehabilitation should revise and formalize the policies and procedures in its grant manual to incorporate the rules adopted by regulation and to address the recommendations in this report. The grant manual should specify that any deviations from the required grant process must be for good cause and be documented.

To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual a process for the review committees to request additional information from appellants or program staff. To allow time for an adequate review of any additional information, Rehabilitation should consider extending the time for review committees to issue their decision on appeals from 30 days to 45 days.

To ensure that it consistently and thoroughly evaluates appeals, Rehabilitation should establish in state regulations and its grant manual that to be able to rescore applications when necessary, the review committee members should be subject-matter experts or, if they are not subject-matter experts, the review committee should have the authority to recommend a new evaluation panel instead of rescoring applications itself when it identifies a reason to invalidate previous evaluations.