Our audit of Rehabilitation’s grant application and review process highlighted the following:
- Although required to have procedures in place for soliciting and awarding grants, Rehabilitation failed to formalize such procedures, which resulted in inconsistencies and inadequacies in the grant process.
- Rehabilitation had some deviations from or gaps in its grant process that raised questions about its fairness.
- It inappropriately accepted certain information from some grant applicants after deadlines stipulated in its requests for applications had passed.
- It limited its pool of prospective evaluators and did not always ensure that they were free from conflicts of interest or bias before selecting them.
- Rehabilitation did not always follow its appeals process, and the review committees did not always conduct comprehensive reviews to determine whether errors or omissions occurred, evaluator biases affected the scoring process, or evaluators supported their scores.
Results in Brief
The Department of Rehabilitation (Rehabilitation) provides services and advocacy to Californians with disabilities. Rehabilitation offers the majority of its services directly to individuals with disabilities through the vocational rehabilitation program, which provides services to disabled individuals to assist them in obtaining competitive employment. It also works in cooperation with 28 independent living centers, which are nonresidential, nonprofit, community‑based agencies designed and operated within a local community by individuals with disabilities, to provide services and advocacy. Rehabilitation supports these services through a combination of federal and state funding. Recently, its process for soliciting and evaluating grant applications and awarding grant funds (grant process) has come under scrutiny, particularly as it relates to the 2017 Systems Change Network Hub grant (Systems Change grant). In our review of this grant and three others that Rehabilitation awarded from fiscal years 2014–15 through 2017–18, we found that Rehabilitation would significantly benefit from improvements in a variety of areas to ensure that it consistently and fairly conducts the grant process, and to help it defend its award decisions.
Rehabilitation took varying approaches to the grant process, some of them questionable, for each of the four grants we reviewed, largely because it did not have formalized written procedures for the grant process. Although state and federal regulations require that Rehabilitation have procedures and a format in place, and despite developing a draft grant solicitation manual (grant manual) to document its process in 2015, Rehabilitation never finalized the draft or instructed staff to follow the details set forth in the grant manual. Rehabilitation officials stated that Rehabilitation did not finalize the grant manual because it does not award many grants and has had higher priorities. We found that the grant manual largely contained procedures that were reasonable and useful and, in the absence of formalized procedures or other sufficient direction provided to its program management and staff (program staff), we used it as the basis for our assessment of Rehabilitation’s grant process.
In our review of four grants that Rehabilitation awarded from fiscal years 2014–15 through 2017–18, we found that it did not adequately plan the grant process and that it failed to clearly define and assign staff roles and responsibilities at its outset. Rehabilitation also could not demonstrate that key staff were free from conflicts of interest; received the required ethics training; and understood confidentiality procedures before developing requests for applications (RFAs), the initial step in the grant process. By not undertaking these critical steps, Rehabilitation did not adequately ensure a consistent, complete, and fair grant process.
Additionally, RFAs should include clear scoring criteria; application deadlines; and descriptions of the evaluation, award, and appeals processes. Rehabilitation did not adequately disclose in its RFAs all necessary information to ensure a transparent competitive process and, in some cases, disclosed inaccurate information. For instance, the scoring process described in the RFAs did not always align with the actual process that the individuals evaluating the grant applications (evaluators) followed when scoring grant applications, and Rehabilitation generally did not disclose in advance its methods for awarding grants in the event of a tie. Further, when Rehabilitation initiated its grant process, we found that it did not designate a centralized location to maintain grant‑related documents and, in some cases, destroyed key documents. This contributed to Rehabilitation failing to respond fully to some requests for public records related to the grant process.
Although Rehabilitation included deadlines in its RFAs for submitting documentation, we found that it inappropriately accepted certain information from some grant applicants after these deadlines. For example, Rehabilitation accepted an entire revised section of an application from one grant applicant after the submission deadline. Instead of disqualifying the applicant as specified in the RFA, it continued to process the application and ultimately awarded one of the grant awards to this applicant—possibly preventing another qualified applicant from receiving grant funding.
Further, rather than publishing a solicitation for evaluators of grant applications and seeking the disability community’s participation, which is suggested in the grant manual, Rehabilitation selected evaluators without issuing a solicitation for three of the four grants we reviewed. As a result, it limited its pool of prospective evaluators and missed the opportunity to ensure that it obtained the most qualified evaluators possible from the larger disability community. Rehabilitation also did not always ensure that prospective evaluators were free from conflicts of interest or bias before selecting them, and in fact it selected some evaluators for one grant who had held leadership positions in an organization that had a known relationship with one grant applicant, which created at least the potential for perceived bias.
Additionally, the grant manual requires that Rehabilitation appoint a technical review team whose responsibilities include training evaluators on the grant process and program requirements before the evaluation process begins, answering evaluators’ questions related to the program or evaluation process, and ensuring after evaluations are complete that evaluators followed instructions during the evaluation process. Rehabilitation did not always appoint technical review teams to oversee the evaluation of each grant; sometimes staff fulfilled the responsibilities of the teams. However, the technical review teams and staff failed to ensure evaluators followed the evaluation process, which contributed to procedural errors in the evaluation of applications for each of the grants we reviewed. In addition, the technical review teams and staff did not adequately review evaluators’ scores and comments for each grant to ensure that evaluators followed instructions, likely resulting in some applicants appealing certain grant awards.
When Rehabilitation identifies errors in the evaluation process, the grant manual indicates that staff should begin a new RFA process to rectify errors and ensure that the grant process is fair. Although it may be necessary for Rehabilitation to restart the grant process to remedy the errors, we believe that under certain circumstances Rehabilitation can correct the issues and have evaluators rescore applications without restarting the grant process. Instead, it sometimes asked evaluators to rescore applications without correcting the issues.
The four grants we reviewed for this audit resulted in nine appeals, and Rehabilitation did not consistently adhere to the appeals process contemplated in its grant manual. State regulations require Rehabilitation's chief deputy director (chief deputy) to appoint a grant review committee (review committee) to review the appeal. In addition, the grant manual gives the chief deputy the option to acknowledge receipt of the appeal in writing and to notify the appellant of the qualifications of the review committee members. Although we found that Rehabilitation consistently acknowledged its receipt of appeals, it did not always notify appellants of the review committee members’ qualifications; thus, appellants were not fully informed about those who would conduct a review of their appeal. State regulations provide limited direction regarding how Rehabilitation should review and process appeals, requiring only that applicants submit appeals within 30 days of the notice of the intent to award, and that a review committee review the request and notify the appellant in writing of its decision within 30 days of the request. The chief deputy said that she followed state regulations in addressing appeals, but that she was unfamiliar with the additional steps provided in the grant manual.
Further, we found that the review committees did not always conduct comprehensive reviews to determine whether procedural errors or omissions had occurred, evaluator prejudice affected the scoring process, and evaluators supported their scores with evidence from the relevant applications, as the grant manual suggests. For the 2017 Systems Change grant, when the review committee found procedural errors in the grant process, Rehabilitation did not resolve these errors before awarding the grant. Further, although the grant manual suggests that the review committees should determine whether issues occurred in the grant process, such as procedural errors, given the numerous issues we found, we believe Rehabilitation should designate staff, separate from those who are responsible for developing RFAs, creating scoring criteria, and selecting evaluators, to conduct such reviews of each grant before Rehabilitation makes a final decision as to the grant recipients. This oversight of the grant process will provide Rehabilitation with additional assurance that program staff and evaluators adhered to its grant process, and help it demonstrate that the process was followed.
Summary of Recommendations
To comply with federal and state requirements, and to ensure consistency and fairness in its grant process, Rehabilitation should do the following:
- Issue regulations describing its grant process from RFA development through appeals. It should submit its proposed regulations to the Office of Administrative Law no later than December 2018.
- Revise and formalize the policies and procedures in its grant manual to incorporate the rules adopted by regulation and to address the recommendations in this report. The grant manual should specify that any deviations from the required grant process must be for good cause and be documented.
Rehabilitation should clarify the roles and responsibilities of program staff involved in the grant process and ensure that those staff are free from conflicts of interest, receive the required ethics training, and understand confidentiality procedures. To provide grant applicants a full understanding of the grant process, Rehabilitation should disclose in its RFAs clear scoring criteria and descriptions of the evaluation, award, and appeals processes. Rehabilitation should also ensure that it maintains all relevant grant documentation in a centralized location and responds fully to all requests for public records related to grants.
Rehabilitation should issue a public solicitation for evaluators for each grant and should train evaluators on conflicts of interest, including a discussion of bias, or the appearance of bias, before selecting them as evaluators. Further, to ensure that it equips evaluators with the information necessary to conduct sufficient evaluations of applications, Rehabilitation should develop evaluator training that can be tailored to each grant and includes instruction on how to evaluate applications.
Rehabilitation should also resolve issues before it rescores applications when it identifies procedural errors. Further, to ensure that Rehabilitation has appropriate oversight of its grant process and can demonstrate that it was followed, it should designate staff, separate from those involved in the respective grant process, to conduct a review for procedural errors, evaluator prejudice, and whether evaluators supported their scores with evidence from the relevant applications before it awards grants.
Rehabilitation agreed with our recommendations and indicated that it plans to implement them. We look forward to Rehabilitation’s 60‑day response to our recommendations to learn more about its progress.