2018-041 - Status of Recommendations - Table 2

5. To ensure that BRN resolves complaints regarding nurses in a timely manner, by March 1, 2017, it should develop and implement formal policies that specify required time frames for each key stage of the complaint resolution process, including time frames for how quickly complaints should be assigned to the proper investigative unit or expert witness, and how long the investigation process should take. BRN should also work with DOI to establish a reasonable goal for the length of time DOI's investigators take to conduct investigations of complaints referred to it by BRN.

8. To increase its pool of expert witnesses, by June 2017, BRN should develop and implement a process to track the effectiveness of the methods it uses to recruit expert witnesses, and then focus its efforts on those methods that prove to be the most successful.

10. To increase its pool of expert witnesses, by June 2017, BRN should take the steps necessary to increase the hourly wage it pays expert witnesses.

11. To ensure it does not risk compromising private and confidential information related to ongoing investigations of complaints, BRN should immediately ensure that any email correspondence it has with expert witnesses is transmitted securely.

12. To ensure that it is able to accurately monitor the performance of its complaint resolution process and that it has accurate data to address its staffing needs, BRN should immediately begin working with Consumer Affairs to implement cost-effective input controls for BreEZe that will require BRN staff members to enter information into a complaint record in a way that is consistent with BRN's business processes, as well as to implement changes that would cause BreEZe to accurately identify the order in which activities occur.

13. To ensure that it is able to accurately monitor the performance of its complaint resolution process and that it has accurate data to address its staffing needs, once it has implemented cost-effective input controls for BreEZe and accumulated six months of data, BRN should analyze these data to determine whether its staffing is sufficient to meet its workload.

16. To ensure that BRN and DOI consistently conduct adequate investigations and obtain sufficient and appropriate evidence to discipline nurses accused of violating the Nursing Practice Act if warranted, BRN in collaboration with Consumer Affairs should implement a mechanism by March 2017 to track and monitor supplemental investigation requests that result from investigators' failure to obtain required documentation or sufficient evidence and use this information to mitigate the causes of these failures.

20. To ensure that its enforcement unit employees appropriately address and process complaints in a consistent and efficient manner, BRN should implement a formal training program no later than December 2017. In developing this program, BRN should consult with DOI and the Attorney General to identify training that could benefit its enforcement staff, and also solicit input of its enforcement staff on areas of their job duties where they believe they need additional training.

26. To ensure that it promptly and appropriately sends notifications to complainants as state law requires, by March 2017, BRN should establish formal procedures, such as managers performing routine audits of complaint files, to monitor incoming complaints and final dispositions.

7. To ensure that it adequately tracks critical information related to its basic functions and mission, the commission should work with Consumer Affairs to ensure that the new online program will meet its needs and requirements. Once the program is in place, the commission should use it as its central means for tracking its operations.

9. To ensure that it accurately collects revenue, the commission should calculate the pension assessment by counting all the complimentary tickets issued, except for working complimentary tickets, not merely the complimentary tickets that are redeemed. If the commission does not agree that it should calculate the pension assessment by counting all the complimentary tickets issued, it should seek a change in its regulations to calculate the fee based only on the number of complimentary tickets redeemed.

22. The commission needs to establish regulations that describe its process for determining its ticket assessment for the Neurological Examination Account so that it avoids the use of underground regulations.

17. To the extent that Consumer Affairs chooses to implement BreEZe at the phase 3 regulatory entities, it should first complete a formal cost-benefit analysis to ensure that BreEZe is a cost-effective solution to meet these regulatory entities' business needs. To make certain this analysis is complete, it should include an assessment of the potential changes these regulatory entities may require to be made of the BreEZe system and the associated costs. Consumer Affairs should complete the cost-benefit analysis before investing any more resources into the implementation of BreEZe at the phase 3 regulatory entities, and it should update this analysis periodically as significant assumptions change.

35. To ensure that future training for BreEZe system rollouts is timely and effective, Consumer Affairs should provide training on the BreEZe system as close to the rollout date as possible to ensure that staff retain the information for using the system as it is implemented.

36. To ensure that future training for BreEZe system rollouts is timely and effective, Consumer Affairs should work with the regulatory entities to develop training that is specific to each entity's business processes.

35. Following the completion of the analysis (described in Recommendation 34), the Medical Board should take the appropriate follow-up actions that it deems necessary, including the investigation of physicians identified in its analysis.

5. To ensure that it can better monitor how its medical staff and contractors adhere to the informed consent requirements of California Code of Regualtions, sections 70707.1 through 70707.7 (Title 22), the Receiver's Office should develop a plan by August 2014 to implement a process by December 2014 that would include working with Corrections to establish a process whereby inmates can have witnesses of their choice when consenting to sterilization, as required by Title 22, or working to revise such requirements so that there is an appropriate balance between the need for secure custody and the inmate's ability to have a witness of her choice.

11. To ensure that inmates receive only medical services that are authorized through its utilization management process, the Receiver's Office should ensure that the computer system it procures includes functionality to electronically link medical scheduling with authorization through the utilization management process to prevent all unauthorized procedures, regardless of whether they may result in sterilization, from being scheduled.

4. Adjust current employees' leave balances in the leave accounting system to correct any improper charging of leave identified by the audit.

6. In instances where the audit has determined that an employee's leave balance was mischarged but the employee subsequently departed state service, take appropriate measures to remedy any resulting incorrect compensation of the employee for unused leave upon his or her departure, including by seeking repayment of any amount overpaid to the employee.

2. Corrections should immediately require mental health staff to score 100 percent on risk evaluation audits in order to pass. If a staff member does not pass, Corrections should require the prison to follow its current policies by reviewing additional risk evaluations to determine whether the staff member needs to undergo additional mentoring.

8. To monitor prisons' compliance with its requirement that inmates in crisis beds receive daily progress notes, Corrections should implement monitoring of these notes electronically into its audit process by the time the electronic health record system is in use systemwide in October 2017. Corrections should require prisons that are out of compliance to develop and implement quality improvement plans, and it should follow up on the prisons' implementation of those plans.

11. To address the unique circumstances that may increase its female inmates' rates of suicide and suicide attempts, Corrections should continue to explore additional programs that could address the suicide risk factors for female inmates.

12. To ensure that all prison staff receive required training related to suicide prevention and response, Corrections should immediately implement a process for identifying prisons where staff are not attending required trainings and for working with the prisons to solve the issues preventing attendance.

13. To ensure that trainers and risk evaluation mentors at all prisons are able to train staff effectively, Corrections should immediately begin requiring prisons to report the percentage of their trainers and mentors who have received training on how to conduct training and mentoring. It should work with prisons to ensure that all trainers and mentors receive adequate training.

14. To maximize the value of its trainings related to suicide prevention and response, Corrections should ensure that starting in January 2018, its trainings include all content that the special master and its own policies require.

15. To ensure that it has enough staff to provide mental health services to all inmates who require care, Corrections should review and revise its mental health staffing model by August 2018.

16. To ensure that prisons comply with its policies related to suicide prevention and response, Corrections should continue to develop its audit process and implement it at all prisons by February 2018. The process should include, but not be limited to, audits of the quality of prisons' risk evaluations and treatment plans.

17. To ensure that prisons can easily access Corrections' current policies related to mental health, Corrections should ensure that its program guide is current and complete as it works to incorporate the program guide into regulations. Corrections should immediately begin working with federal court monitors to draft regulations.

18. To ensure that suicide prevention teams meet quorum requirements, Corrections should, starting January 2018, work with prisons that consistently fail to achieve a quorum to resolve issues that may be preventing the teams from having all required members present at meetings.

21. To provide the public and relevant stakeholders with accurate information on suicides and suicide attempts in its prisons, Corrections should immediately require prison staff to work with mental health staff to reconcile any discrepancies on suicides and suicide attempts before submitting numbers to the COMPSTAT unit.

19. Corrections and Correctional Health Care should run a query of exempt positions related to the chief psychologist classification, such as clinical psychologists and senior psychologists, to determine whether any other exempt employees were improperly credited or paid for on-call or call-back assignments prior to December 2014, and seek recovery through reducing those employees' accumulated leave balances.

15. CDCR should revise the Institutional Worker Supervision Pay (IWSP) procedure to require that personnel staff review and ensure that an employee's direct supervisor signs the qualifying employee's timesheets and IWSP documents each month.

17. CDCR should enforce its current procedure to retain IWSP documentation.

18. CDCR should enforce its current procedure for personnel staff to conduct annual audits of the IWSP program.

19. CDCR should train all employees, supervisors, and personnel staff who receive, approve, or issue the extra pay to ensure that they are familiar with the requirements of the IWSP procedure and CalHR's California State Civil Service Pay Scales section 14,Pay Differential 67.

2. To ensure it can demonstrate that its fraud prevention efforts are maximizing financial recoveries for the beverage program, CalRecycle should both modify and annually update its fraud management plan to include the following:
- By December 31, 2014, formally establish a systematic process for analyzing, monitoring, and responding to the risk of fraudulent recycling of out-of-state beverage containers.
- Develop fraud estimates—by type of fraudulent activity—that quantify the potential financial losses to the beverage program and the methodology CalRecycle used to develop these estimates.
- Identify the amount of actual fraud in the prior year by type of fraudulent activity, such as the financial losses resulting from the redemption of out-of-state beverage containers or the falsification of reports used to substantiate program payments.
- Identify the amount actually recovered for the beverage program in the form of cash for restitution and penalties resulting from fraud.

3. To allow for public input and to prevent any legal challenges claiming that its policies and procedures regarding prepayment holds constitute unenforceable underground regulations, CalRecycle should adopt these policies and procedures as regulations in accordance with the Administrative Procedure Act.

1. Take appropriate corrective action against the district engineer and the supervisors for their participation in or failure to address the conflict of interest.

5. To better utilize unused space at the veterans homes, and to serve more veterans within legal restrictions, CalVet should use the information in the Yountville veterans home master plan to develop a plan for using unused space at that home to generate revenue and/or serve more veterans.

11. To ensure it is maximizing its ability to serve veterans in the State's veterans homes, CalVet should follow through with its plan to assess the bed capacity of the homes for veterans' needs after the homes in Fresno and Redding are licensed, to determine the most appropriate number of beds for the different levels of care offered at each home.

15. To allow for public input and to prevent any legal challenges that its policy of capping members' fees, its $165 monthly income-retention policy for members, and its age and admission policy are unenforceable underground regulations, CalVet should adopt these policies as regulations in accordance with the Administrative Procedure Act.

17. To more effectively and efficiently meet state purchasing and procurement requirements, CalVet should continue implementing quality assurance policies to strengthen its oversight of its purchasing practices, including conducting on-site reviews of the purchasing practices of the veterans homes.

2. The commission should determine the cause of its lack of compliance with state law requiring it to issue award decisions within 75 days of the date an intervenor submits a compensation claim, and it should determine what actions to take to rectify the problem. The commission should ensure that it has sufficient information, such as detailed tracking information regarding claims, to identify where in the process delays are occurring. If the commission determines that the current 75-day statutory period is unreasonable, it should seek a change in state law.

5. To comply fully with state law, the commission should conduct a comprehensive market rate study and update it periodically.

14. The commission should follow the requirement in state law to inspect and audit the accounting records of utilities it regulates within required time frames. If the commission chooses to continue to meet this requirement through the general rate case process, it should ensure that all utilities file a general rate case on a regular schedule so as to comply with the state law's audit requirement. However, the commission should follow alternate methods to comply with the audit requirement when a utility will not be filing for its general rate case in time to be audited within three or five years, depending on the timing of the required audit for that utility.

2. To ensure that it resolves complaints against passenger carriers in a timely manner, the commission should establish a method for prioritizing complaints and it should implement a policy specifying the maximum amount of time between the receipt of a complaint and the completion of the subsequent investigation. Further, the commission should require branch management to monitor and report regularly on its performance in meeting that policy.

6. To ensure that the branch conducts thorough investigations of passenger carriers, the commission should implement a formal training program to ensure that all investigators have adequate knowledge and skills related to regulating passenger carriers.

9. To ensure that passenger carriers submit accurate fee payments, the commission should require its fiscal staff to implement a process to verify passenger carrier fee payments and associated revenue.

10. To ensure that it complies with state law and uses passenger carrier fees appropriately, the commission should implement a process to ensure that passenger carrier fee revenues more closely match related enforcement costs.

12. To detect and deter carriers from operating illegally at airports, the branch should use as intended the five positions added for passenger carrier enforcement at airports. If the branch chooses not to designate five positions solely for this purpose, then it must be prepared to demonstrate regularly that an equivalent number of full-time positions are working on this activity.

13. To strengthen its leadership and ensure passenger carrier and public safety, the branch should produce a draft strategic plan by December 31, 2014, with a final strategic plan completed as the commission specifies. The strategic plan should include goals for the program; strategies for achieving those goals, including strategies for staff development and training; and performance measures to assess goal achievement.

3. To ensure that policy makers, enforcement officials, and the general public have access to accurate consumer complaint data in its Computer Information Management System (CIMS), the branch should continue to implement its quality management team program component focused on reviewing the categorization of complaints and correcting identified errors.

4. To ensure that policy makers, enforcement officials, and the general public have access to accurate consumer complaint data in CIMS, the branch should develop and implement tools by September 30, 2015, to measure the quality management team program's effectiveness.

11. The commission should ensure that it complies with all policy requirements in the State Administrative Manual Chapter 5300 no later than April 2016.

13. As part of developing, implementing, and maintaining an entitywide information security program, the commission should develop a risk management and privacy plan and conduct an assessment of risks facing its information assets.

16. The commission should revise its existing recovery plan to include a list of applications supporting critical business functions, their maximum acceptable outage time frames, and detailed recovery strategies for each application.

17. The commission should revise its existing recovery plan to include detailed procedures for rebuilding its technology infrastructure at an alternate processing site.

18. The commission should conduct regular tests and exercises to assess the sufficiency of the revised recovery plan and refine the plan when necessary.

2. To ensure that the choice of a vendor is sufficiently justified and that the vendor represents the best value, the CPUC should explain in its final decision how the vendor was the most qualified in all cases when the CPUC does not competitively select the vendor it directs utilities to contract with.

4. To avoid the appearance of inappropriate relationships, the CPUC should adopt a policy to prohibit commissioners from accepting gifts from regulated utilities and energy companies and free travel from organizations with significant ties to regulated utilities and other parties with financial interests in CPUC proceedings.

6. To ensure that its contracting practices align with state requirements and best practices, the CPUC should update, distribute, and follow its contracting procedures manual. The manual should identify specific responsibilities for both contracts office staff and project managers, and it should provide specific guidance about the processes the CPUC will employ to do the following:

-Fully justify civil service exemptions.

-Conduct market research for exempt contracts.

-Fully support the need for additional funding.

-Ensure that it does not change the scope of work too significantly from the original.

-Monitor contractor performance against criteria included in its contracts.

-Avoid sole-source contracts when it is able to solicit competitive bids for services.

7. To ensure that its contracting practices align with state requirements and best practices, the CPUC should provide immediate refresher training to its contract analysts and contracts office manager, and establish a regular schedule of annual training for them to attend.

8. To ensure that its contracting practices align with state requirements and best practices, the CPUC should designate a limited number of project managers for each division at the CPUC, and provide those individuals with training on the CPUC's processes related to contracting, including how to monitor progress of a contractor's work.

10. The CPUC should update its general policy on responding to California Public Records Act requests so that the policy aligns with state law.

13. The CPUC should update its regulations to require parties joining a proceeding by filing a protest or response to an application or petition, or by filing comments in response to a rulemaking proceeding to fully disclose their interests in the proceeding.

15. The CPUC should update and follow its retention policy for economic interest disclosures so that it is aligned with state law.

3. To show how air pollution emissions reductions related to the solar initiative benefit the State, the commission should include in future reports the measurable benefits of those reductions.

7. To further its mission to obtain the lowest possible rates for reliable and safe utility service for ratepayers through its reviews of balancing accounts, Ratepayer Advocates should use the commission's list of balancing accounts to guide its selection of the number, size, and type of balancing accounts to review so that its review coverage is more proportional across all utilities.

14. To ensure that Information Technology (IT) projects have the oversight needed to better position them for success, Technology should develop thresholds relating to IT project cost increases and schedule delays to inform and better justify its decision to allow an IT project to continue. If a department's IT project reaches or exceeds these thresholds, Technology should require the department to conduct a cost-benefit analysis for the project and include this analysis in a Consumer Affairs' Special Project Report (SPR). Technology should consider the results of this analysis in its decision to approve or deny the SPR and, if warranted, take action to suspend or terminate the project so that it does not allow projects with significant problems to continue without correction.

14. Although Technology indicated that its intent is not to outsource its statutory responsibility for Independent Project Oversight (IPO), in any instances where its staff conduct a portfolio review of a project's IPO, Technology should, by December 2016, establish a process for its review of documents created by the agency's IPO contractor that includes verifying whether these reports include critical analysis of project progress and vendor performance so it can intervene when necessary.

6. To help ensure the independence and objectivity of IPO analysts working in the oversight and consulting division, Technology should provide regular training regarding maintaining independence while conducting project oversight.

2. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should ensure that the project management system can centrally track and extract all data regarding project status, including time delays, cost overages, and the reasons for each.

3. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should track the reasons that projects are pending to identify its true backlog of projects. In doing so, it should develop a process to follow up on those projects that are pending to ensure that they are not on hold unnecessarily and are appropriately moving forward.

4. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should, at least annually, use the centrally tracked data to identify common themes in the causes for project delays and cost overages and develop solutions to address these issues. Further, it should report the results of its review to General Services' executive management.

5. Until the division implements its planned project management system, it should, by September 2016, develop a process to, at a minimum, identify project status and reasons for project delays as well as cost overages. Using these data, the division should modify its project management processes to ensure the efficient and effective delivery of projects.

6. The division should develop and implement a process for preparing reasonable time frames and cost estimates for its projects within the building management branch. To better inform the development of this process, the division should evaluate the branch's structure, which should include a staffing analysis, to determine whether it is effectively organized and whether it should add cost estimator positions.

7. To ensure that client agencies are paying equitable rates, by December 2016 General Services should develop and implement a strategy for allocating its administrative costs equally among all the projects it completes for client agencies, including those portions outsourced to private firms.

8. To ensure that the project management branch charges its client agencies a competitive hourly rate, by December 2016 and every two years thereafter, the division should conduct a rate analysis that fully accounts for differences between the project management branch's rate and private firms' rates. If it finds that the rates are not competitive, the division should identify and implement strategies to ensure that the project management branch's rates are as competitive as they can be with those of its private firm counterparts. Further, the division should explore and implement any other reasonable methods to ensure that it is delivering projects as cost effectively as possible.

10. To improve its communication with client agencies, the division should develop a process for providing periodic detailed bills and invoices to client agencies clearly describing the work for which it is charging.

11. To effectively evaluate the performance of its branches in delivering projects, the division should develop meaningful goals and objectives and a method of measuring its success in achieving them as part of its strategic plan that is focused on ensuring that projects are delivered on time and within budgeted cost estimates.

12. To ensure that its project management staff are adequately trained and have the information necessary to deliver projects as efficiently and effectively as possible, the division should conduct a comprehensive survey every other year of all of its client agencies to inform necessary improvements to its processes and training program and, in the interest of transparency, make the survey results public.

3. To improve its oversight of the State's noncompetitive contracts, General Services should, within 90 days, create plans for regularly performing statewide analyses to identify potential abuse or overuse of noncompetitive contracts. These analyses should include, but not be limited to, calculating the proportional value and number of the State's competitive and noncompetitive contracts and amendments, examining trends in agencies' use of noncompetitive contracts and amendments, and identifying unusual patterns among vendors receiving state contracts through noncompetitive means.

2. To increase its efforts to prevent and control diabetes, Public Health should develop a process for identifying and applying for federal funding opportunities, including routinely and proactively searching for grants. In addition, Public Health should seek funding for a grants specialist position to identify and apply for federal and other grants.

3. To protect the health, safety, and well-being of residents in long-term health care facilities, Public Health should improve its oversight of complaint processing. Specifically, by May 1, 2015, Public Health should establish a specific time frame for completing facility-related complaint investigations and entity-reported incidents (ERI) investigations and inform staff of the expectation that they will meet the time frame. Public Health should also require district offices to provide adequate, documented justification whenever they fail to meet this time frame.

4. To protect the health, safety, and well-being of residents in long-term health care facilities, Public Health should improve its oversight of complaint processing. Specifically, by May 1, 2015, Public Health should develop formal written policies and procedures for Professional Certification Branch (PCB) to process complaints about certified individuals in a timely manner. These policies and procedures should include specific time frames for prioritizing and assigning complaints to investigators, for initiating investigations, and for completing the investigations. Public Health should also inform staff of the expectation that they will meet these time frames. It should require PCB to provide adequate, documented justification whenever PCB fails to meet the time frames.

8. To protect the residents in long-term health care facilities from potential harm, Public Health should ensure that its district offices have adequate staffing levels for its licensing and certification responsibilities, including staffing levels that allow prompt investigations of complaints. Specifically, Public Health should continue working with CalHR to complete the reclassification of district offices' investigator supervisor and manager positions and then quickly fill the vacant positions at district offices.

12. To ensure that its district offices properly investigate complaints and ERIs, Public Health should make certain that all district offices follow procedures requiring supervisory review and approval of complaint and ERI investigations. If the district offices do not have a sufficient number of supervisors to review investigations they did not conduct, Public Health should arrange to assist the districts until such time that they do have a sufficient number of supervisors.

14. To ensure that it has closed complaints and ERIs appropriately, Public Health should take steps by April 2015 to verify that complaints that its field operations branch closed administratively were closed appropriately. For example, it could request the district offices to verify that the closures were appropriate.

18. To improve its enforcement, each year Public Health should evaluate the effectiveness of its enforcement system across all types of health facilities, including those in developmental centers, prepare the required annual report, and, if called for, recommend legislation to improve the enforcement system and enhance the quality of care.

8. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should address staffing issues by preparing and resubmitting to Public Health a recruitment and retention proposal, developing a succession plan, and taking necessary steps to implement its planned reorganization.

9. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should ensure that its information technology data systems have necessary safeguards, contain accurate and complete data, and support its program needs.

10. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should update and develop its regulations as necessary to ensure consistency with existing state law.

5. To ensure that it more effectively shares, receives, and uses administrative action information, Social Services should develop and maintain a centralized database containing its own administrative actions and those received from other state departments, in order to share this information among these departments as required by state law. Social Services should seek funding if it believes additional resources are necessary.

10. To ensure that Social Services evaluates the risk individuals may pose to vulnerable populations in its licensed care facilities as quickly as possible, by July 2017 Social Services should establish time frames for staff to evaluate individuals who are present in their facilities and who have received administrative actions from other departments. In addition, it should monitor and follow up with the appropriate staff regarding the status of their assessments of these individuals and their final decisions.

13. To comply with state law and better protect vulnerable populations in California's licensed care facilities, Social Services should immediately change its policy to require that its exemption analysts evaluate all infraction convictions, other than minor traffic violations, before granting exemptions to individuals. If Social Services believes it is not feasible to evaluate all of these convictions, it should report to the Legislature by June 2017 how it ensures that vulnerable populations are not at risk and should request that the Legislature change the law to eliminate infraction convictions as a crime category that Social Services must evaluate in order to grant an exemption.

16. Until the Legislature requires that Social Services receive both California and federal criminal history information before issuing a clearance or processing an exemption, to better protect vulnerable populations, Social Services should immediately revise its policy to require its regional offices to obtain all self-disclosure forms for individuals who submit fingerprints to Justice as part of an application to be present in a licensed facility. The regional offices should then forward to the Caregiver Background Check Bureau (CBCB) all self-disclosure forms that identify a conviction.

17. Until the Legislature requires that Social Services receive both California and federal criminal history information before issuing a clearance or processing an exemption, to better protect vulnerable populations, Social Services should immediately change its practice of allowing individuals who have not submitted a self-disclosure form to Social Services to have access to licensed facilities, thus reflecting the requirements of state law. In addition, the CBCB should develop a process to ensure that individuals cannot receive a clearance or an exemption without the CBCB first receiving both California and federal criminal history information if a regional office does not have a self-disclosure form for the individual.

26. To ensure that regional offices pursue legal actions in a timely manner, by July 2017 Social Services' headquarters should identify a resource—such as a unit—to monitor and follow up with the regional offices regarding the status of their legal actions related to substantiated address matches of registered sex offenders at licensed facilities.

30. To improve the oversight of psychotropic medications prescribed to foster children, Social Services should collaborate with the counties and other relevant stakeholders—including Health Care Services, as necessary—to develop and implement a reasonable oversight structure that ensures the coordination of the State's and counties' various oversight mechanisms as well as the accuracy and completeness of the information in Social Services' data system. This structure should include at least the following items:
- Identification of the specific oversight responsibilities to be performed by the various state and local government agencies.
- An agreement on how county staff such as social workers, probation officers, and public health nurses will use printed Health and Education Passports to obtain foster children's necessary mental health information—including psychotropic medications and psychosocial services—for inclusion in Social Services' data system.
- A plan to ensure that counties have sufficient staff available to enter foster children's mental health information into Social Services' data system and the resources to pay for those staff.
- An agreement on the specific information related to psychotropic medication—including but not limited to the medication name, maximum daily dosage, and court authorization date—and psychosocial services and medication follow-up appointment information that county staff must enter into Social Services' data system for inclusion in foster children's Health and Education Passports.
- Specific directions from Social Services regarding the correct medication start dates and court authorization dates counties should include in its data system and foster children's Health and Education Passports.
- An agreement on the training or guidance Social Services should provide to county staff members working with Social Services' data system to ensure that they know how to completely and accurately update foster children's Health and Education Passports.
- An agreement on how the counties will use information on the new authorization forms that the Judicial Council approved to better oversee the prescription of psychotropic medications to foster children.
- An agreement regarding how counties will implement, use, or disseminate the educational and informational materials the Quality Improvement Project has produced, including the California Guidelines for the Use of Psychotropic Medication with Children and Youth in Foster Care, Questions to Ask About Medications, and the Foster Youth Mental Health Bill of Rights.
- An agreement on the specific measures and the best available sources of data the State and counties will use to oversee foster children prescribed psychotropic medications, including psychosocial services and medication follow-up appointments.
- An agreement on how the State and counties will oversee psychotropic medications prescribed to foster children by fee-for-service providers who are not affiliated with county Medi-Cal mental health plans.
- An agreement on the extent of information related to psychotropic medications prescribed to foster children that counties will include in the self-assessments, system improvement plans, and annual progress reports they develop as part of Social Services' California Child and Family Services Reviews.
- An agreement on the extent of the information related to psychotropic medications prescribed to foster children that counties will include in their responses to Health Care Services' reviews, including its county Medi-Cal mental health plan compliance reviews and external quality reviews.

40. To promote continued improvement in the CWS system, Social Services should encourage each county CWS agency to designate personnel to update regularly their policies and procedures, to include a detailed description of the need for ongoing supervisory reviews of key aspects of their respective service processes and incorporate that description into their policies and procedures, and to designate personnel to perform regular quality assurance reviews.

41. To promote continued improvement in the CWS system, Social Services should ask each county CWS agency to report to Social Services on the status of these efforts within 60 days, six months, and one year from the publication of this audit report.

4. To ensure that counties' use of foster family agency placements is justified, Social Services should take action to implement the recommendation we previously made in our 2011 audit. Specifically, Social Services should require counties to give licensed foster homes a higher priority than foster family agencies for children that do not have identified treatment needs.

5. To ensure that counties' use of foster family agency placements is justified, Social Services should take action to implement the recommendation we previously made in our 2011 audit. Specifically, Social Services should require counties to prepare a detailed justification for any child placed with a foster family agency.

4. To ensure that all counties consistently gauge the cost-effectiveness of their early fraud detection activities and ongoing investigation efforts for the California Work Opportunities and Responsibilities to Kids (CalWORKs) and CalFresh programs, Social Services should develop a formula to regularly perform a cost-effectiveness analysis using information that the counties currently submit. Specifically, this formula should measure the savings that a county achieves for each dollar spent on antifraud efforts.

5. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should, using the results from the recommended cost-effectiveness analysis, determine why some counties' efforts to combat welfare fraud are more cost-effective than others.

6. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should seek to replicate the most cost-effective practices among all counties. Social Services should work with its legal counsel to determine whether to withhold information about these practices from public disclosure.

7. Social Services should track counties' prosecution thresholds for welfare fraud cases and determine whether they affect counties' decisions to investigate potential fraud, with a focus on determining best practices and cost-effective thresholds. If Social Services' analysis determines that varying prosecution thresholds do affect counties' decisions, it should then work with counties to implement the consistent use of these cost-effective prosecution thresholds.

9. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should address and promptly act on the four remaining recommendations that its steering committee provided in 2008.

12. To make counties' review of match lists more efficient, Social Services should revive its efforts to work with the state and federal agencies that prepare the match lists to address the counties' concerns about match list formats, content, and criteria.

2. To reduce the likelihood of inadvertent errors on replacement timesheets, Social Services should create functionality within Case Management Information and Payrolling System (CMIPS II) to allow replacement timesheets to be printed with data that had been submitted correctly on the original timesheet. Social Services should develop a plan by August 2017 that outlines actions, such as assessing the cost and seeking funding from the Legislature if necessary, that will be taken to create the functionality.

10. To enable it to track whether EDD is meeting its contractual time frame for printing and mailing timesheets, Social Services should either modify its current agreement or require in the renewal of its agreement a method for tracking the time required to print and mail timesheets. Social Services should also perform monthly reviews of the activities performed by EDD and SCO to ensure compliance with the time frames for each agreement. Additionally, Social Services should implement a process to regularly test EDD and SCO processes to ensure that they are within the required time frames.

13. Until state law is changed to facilitate providers' efforts to report their time and to reduce the potential for providers to be inadvertently suspended from the IHSS program, Social Services should inform providers of the weekly maximum number of service hours for each variation in the length of the month, rather than using a standard conversion that results in providers claiming more hours than their recipients are authorized.

14. If the Legislature amends state law as we recommend, Social Services should modify the timesheet format to incorporate the weekly authorization for services and the new two-workweek pay period. Social Services should also reconfigure its timesheet to require that all information be entered on one side of the document, including the signatures of the provider and recipient.

2. To ensure timelier fee assessments, Developmental Services should hold regional centers accountable for providing the monthly placement reports and copies of information letters required by state regulations. To encourage compliance, Developmental Services should specify in its regional center contracts that noncompliant regional centers will pay financial penalties equal to the amount of revenue lost because of their inaction.

4. To ensure that regional centers are aware of the benefits, including cost savings to the State that can be realized by using financial management service (FMS) vendors, DDS should formally communicate to regional centers regarding the model.

5. To ensure that in-home respite vendors are providing quality services and that vendors are adhering to state requirements, DDS should issue regulations requiring regional centers to conduct periodic and ongoing reviews of vendors' programs, employees, and consumer records.

6. To ensure that in-home respite vendors comply with vendor requirements on an ongoing basis, DDS should require the regional centers to develop a process to conduct biennial reviews of the vendor files the regional centers maintain and document the outcome of the review in the files. DDS should require the regional centers to take appropriate action to ensure that vendors comply, up to and including terminating the vendorization, if necessary.

7. To ensure that it is providing oversight in accordance with state law and federal requirements, DDS should ensure that it performs audits of each regional center every two years as required. In conducting these audits, DDS should consistently include a review of in-home respite services.

3. To ensure that Managed Health Care reaches accurate conclusions during its quarterly assessments of the adequacy of provider networks, Health Care Services should establish by September 2015 a process to verify the accuracy of the provider network data it receives from health plans and forwards to Managed Health Care. For example, Health Care Services could verify, for a sample of physicians claimed as part of the health plans' provider networks, that health plans have current written agreements with the providers.

4. To improve the accuracy of provider directories, by December 2015 Health Care Services should revise its processes for monitoring health plans' provider directories. Specifically, Health Care Services should review how each health plan updates and verifies the accuracy of the directory. In addition, Health Care Services should identify best practices and require the plans to adopt those practices.

6. If Health Care Services finds significant errors in a health plan's provider directory, it should work with that health plan to identify reasons for the inaccuracies and require the health plan to develop processes to eliminate the inaccuracies.

10. To ensure that Health Care Services complies with state law, it should increase its oversight of Managed Health Care to ensure that it completes the quarterly assessments required under the agreements.

1. To ensure that it provides claiming units with reasonable opportunities to address concerns with its decisions or actions, Health Care Services should, within three months, begin preparing regulations to establish and implement a formal appeals process that allows claiming units to directly appeal Health Care Services' decisions.

2. To ensure that it provides claiming units with reasonable opportunities to address concerns with its decisions or actions, Health Care Services should, within three months, inform all stakeholders, including claiming units, of the existence of this appeals process.

3. Until the Legislature implements our recommendation in Chapter 2, Health Care Services should immediately resolve weaknesses in its oversight of local educational consortia and local governmental agencies to ensure that these entities sufficiently meet their responsibilities under the administrative activities program and meet the terms of their contracts with Health Care Services. Health Care Services should update its site review and desk review procedures to include the following steps:
* A risk-based approach to selecting entities for review.
* Verification that local educational consortia and local governmental agencies are adequately meeting the oversight and administrative responsibilities described in their contracts with Health Care Services.
* Verification that contracts between local educational consortia or local governmental agencies and their claiming units do not include provisions that could result in disallowed costs, such as allowing Health Care Services' participation fee to be included in the claim calculations.
* Examination of local educational consortia and local governmental agencies' records to ensure that:
-- Costs they claim for federal reimbursement are necessary and reasonable.
-- The entities are not inappropriately earning a profit based on the fees they collect from claiming units.
-- The coding performed by local educational consortia that charge claiming units a percentage of their federal reimbursement is reasonably accurate.

4. Until the Legislature implements our recommendation in Chapter 2, Health Care Services should immediately resolve weaknesses in its oversight of local educational consortia and local governmental agencies to ensure that these entities sufficiently meet their responsibilities under the administrative activities program and meet the terms of their contracts with Health Care Services. Health Care Services should complete the oversight reviews for at least three high-risk local educational consortia or local governmental agencies by December 31, 2015, and post the results to its website.

5. Until the Legislature implements our recommendation in Chapter 2, Health Care Services should immediately resolve weaknesses in its oversight of local educational consortia and local governmental agencies to ensure that these entities sufficiently meet their responsibilities under the administrative activities program and meet the terms of their contracts with Health Care Services. Health Care Services should complete the oversight reviews for any remaining high-risk local educational consortia or local governmental agencies by June 30, 2016, and post the results to its website.

8. To minimize the risk that claiming units could include unallowable costs when calculating their reimbursement claims, Health Care Services should remind all local educational consortia and local governmental agencies that contracts with their claiming units should prohibit claiming units from seeking federal reimbursement of Health Care Services' participation fee.

10. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Health Care Services should implement a single statewide quarterly random moment time survey and develop and implement a plan to take over responsibility for conducting quarterly time surveys and performing related activities as soon as reasonably possible.

11. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Health Care Services should implement a single statewide quarterly random moment time survey and develop and issue a request for proposals to identify a responsible vendor to assist in implementing a statewide quarterly random moment time survey.

12. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Health Care Services should implement a single statewide quarterly random moment time survey and draft revisions to regulations as appropriate and to applicable documents, including the manual, oversight strategies and plans, and policy and procedure letters.

13. To the extent that local educational consortia and local governmental agencies are no longer involved in the administrative activities program, Health Care Services should develop and issue a standard contract for claiming units to sign to participate in the program.

17. To better maximize federal reimbursements for the administrative activities program, Health Care Services should, within six months, develop and implement a method to oversee and track the outreach efforts that local educational consortia and local governmental agencies use for ensuring that nonparticipating claiming units understand the benefits and consider participating in the administrative activities program.

21. To provide the public with the ability to participate fully in developing the rules governing the administrative activities program, Health Care Services should, in accordance with California's Administrative Procedure Act, immediately develop and adopt the regulations cited in the four subdivisions of Section 14132.47 of the Welfare and Institutions Code.

20. To prevent the certification of ineligible providers, Health Care Services should immediately establish a mechanism to identify the number of program sites the provider applicants' medical directors work at, and ensure that the physician ratio does not exceed 1-to-3 in accordance with state law and the certification standards.

4. To ensure that child beneficiaries throughout California can reasonably access dental services under Medi-Cal and to increase child beneficiary utilization and provider participation, Health Care Services should take the following steps for the fee-for-service delivery system by May 2015: immediately take action to resolve any declining trends identified during its monitoring efforts.

5. To help increase the number of providers participating in the program's fee-for-service delivery system, Health Care Services should improve its identification and implementation of changes that minimize or simplify administrative processes for providers. These changes should include revising its processes pertaining to dental procedures that require radiographs or photographs.

7. To ensure that the influx of beneficiaries resulting from recent changes to federal and state law is able to access Medi-Cal's dental services, Health Care Services should immediately take action to resolve any declining trends identified during its monitoring efforts.

19. To ensure that it reports in the U.S. Department of Health and Human Services' Centers for Medicare and Medicaid Services (CMS) an accurate number of child beneficiaries who received specific types of dental services from the centers and clinics, Health Care Services should continue working on a solution to capture the details necessary to identify the specific dental services rendered.

20. To make certain that it meets the requirements of the new state law and that its performance measures are accurate, Health Care Services should establish the provider-to-beneficiary ratio statewide and by county as performance measures designed to evaluate access and availability of dental services and include this measure in its October 2015 report to the Legislature.

23. To ensure that Health Care Services and its fiscal intermediaries reimburse providers only for services rendered to eligible beneficiaries, Health Care Services should obtain Social Security's Death Master File and update monthly its beneficiary eligibility system with death information.

24. To ensure that Health Care Services and its fiscal intermediaries reimburse providers only for services rendered to eligible beneficiaries, Health Care Services should coordinate with the appropriate fiscal intermediaries to recover inappropriate payments made for services purportedly rendered to deceased beneficiaries, if necessary.

33. To increase the State's assurance that foster children do not receive medically inappropriate or unnecessary psychotropic medications, Health Care Services should devise and implement within six months methods to better enforce its prior authorization requirement for the off-label use of psychotropic medications. For example, Health Care Services should revise its claims system to automatically prompt pharmacists to submit treatment authorization requests when filling prescriptions for Medi-Cal beneficiaries under age 18 when the prescribed psychotropic medications have no FDA-approved pediatric uses. Furthermore, as part of its collaboration with Social Services and the counties to develop and implement a reasonable oversight structure, Health Care Services should determine whether information from the Judicial Council's revised court authorization forms would help it better enforce its prior authorization requirements.

2. To ensure that it monitors counties to the fullest extent as the Mental Health Services Act (MHSA) specifies and that it implements best practices, Health Care Services should conduct comprehensive on-site reviews of county MHSA programs, including verifying county compliance with MHSA requirements.

3. To ensure that counties have the needed guidance to implement and evaluate their MHSA programs, Health Care Services should coordinate with the Accountability Commission and issue guidance or regulations, as appropriate, for Facilities programs and for other MHSA requirements, such as a prudent reserve.

5. To ensure that counties have the needed guidance to implement and evaluate their MHSA programs, Health Care Services should collaborate with the Accountability Commission to develop and issue guidance or regulations, as appropriate, to counties on how to effectively evaluate and report on the performance of their MHSA programs.

6. To ensure that Health Care Services and other state entities can evaluate MHSA programs and assist the Accountability Commission in its efforts, Health Care Services should collect complete and relevant MHSA data from the counties.

18. Health Care Services should develop standardized data collection guidelines or regulations, as appropriate, that will address inconsistencies in the data that counties report to the State. In developing these guidelines or regulations, Health Care Services should consult with the Accountability Commission to ensure that data collected reasonably fulfill statewide evaluation purposes.

19. To help ensure county compliance with stakeholder regulations, Health Care Services should provide technical assistance to counties on the MHSA local planning review process and ensure that its guidance to counties is clear and consistent with state regulations.

3. To promote consistency and ensure that it provides sufficient guidance to evaluators, State Hospitals should update its assessment protocol by March 2016 to include more specific instructions on how to conduct evaluations, such as what assessment instruments evaluators may use and what documents they should consider. State Hospitals should also develop a timeline for periodically reviewing and making any necessary updates to the assessment protocol.

4. To comply with state law, State Hospitals should ensure that it follows the Administrative Procedure Act for future changes to its standardized assessment protocol.

3. To improve the oversight of CSU's management personnel, the Chancellor's Office should work with campuses, bargaining unit representatives, the Public Employment Relations Board, and others as necessary to come to an agreement on the appropriate classification of coaches. The Chancellor's Office should take into account the concerns that San Diego State has raised about the labor market for these employees.

10. The Chancellor's Office should finish developing the Common Human Resources System and implement it as scheduled by December 2019.

11. Once it receives the results of its consultant's study on executive compensation, the Chancellor's Office should collaborate as soon as possible with interested parties, such as the LAO, to develop methodologies for future compensation comparisons that consider total compensation.

49. The Office of the President should clarify in the UC policies that a complainant must have and be informed about the right to end the early resolution process at any time and request that his or her complaint be handled under the university's formal process.

51. The Office of the President should clarify in the UC policies that if university officials approve an extension to an investigative timeline, the extension should be restricted to a single extension of no more than 30 days, except in limited circumstances that are beyond the university's control.

4. To determine the amount of money that it can reallocate to campuses and to ensure that it publicly presents comprehensive and accurate budget information, by April 2018 the Office of the President should develop a reserve policy that governs how large its reserves should be and the purposes for which they can be used.

5. To determine the amount of money that it can reallocate to campuses and to ensure that it publicly presents comprehensive and accurate budget information, by April 2018 the Office of the President should implement our recommended budget presentation shown in Figure 11 on page 40. Specifically, the Office of the President's budget presentation to the regents should include a comparison of its proposed budget to its actual expenditures for the previous year. It should also include all its expenditures and identify changes to the discretionary and restricted reserves. The Office of the President should combine both the disclosed and undisclosed budgets into one budget presentation.

8. To ensure that the university achieves its goals of obtaining services at the lowest cost or best value and of providing vendors with fair access to contracting opportunities, the Office of the President should revise the university's contract manual to incorporate the best practices found in the State Contracting Manual for limiting the use of amendments to repeatedly extend existing contracts.

9. To ensure that the university achieves its goals of obtaining services at the lowest cost or best value and of providing vendors with fair access to contracting opportunities, the Office of the President should revise the university's contract manual to narrow the exemption from competition to only selected professional services, similar to the State Contracting Manual.

12. To maximize benefits from the systemwide procurement initiative and to ensure that the university uses those benefits for its teaching, research, and public service missions, the Office of the President should direct all university locations to provide better documentation to substantiate actual benefits they claim related to their procurement decisions.

13. To maximize benefits from the systemwide procurement initiative and to ensure that the university uses those benefits for its teaching, research, and public service missions, the Office of the President should revise its guidance to ensure the benefits that university locations claim result from only procurement-related activities.

15. To maximize benefits from the systemwide procurement initiative and to ensure that the university uses those benefits for its teaching, research, and public service missions, the Office of the President should study ways to measure actual procurement benefits—possibly focusing this effort on benefits from larger dollar amounts—and if such measurement is not possible, it should clearly disclose to the regents and the public that the amounts it reports are based on estimates.

16. To maximize benefits from the systemwide procurement initiative and to ensure that the university uses those benefits for its teaching, research, and public service missions, the Office of the President should, if actual benefits are measurable, implement a process to monitor and report annually to the regents the estimated and actual benefits.

1. To meet its commitment to California residents, the university should replace its "compare favorably" policy with a new admission standard for nonresident applicants that reflects the intent of the Master Plan for Higher Education in California (Master Plan). The admission standard should require campuses to admit only nonresidents with admissions credentials that place them in the upper half of the residents it admits.

2. To meet its commitment to California residents, the university should amend its referral process by taking steps to increase the likelihood that referred residents ultimately enroll.

3. To ensure that campuses' interpretations of admission standards do not adversely impact residents, the university should implement a thorough process to annually evaluate the qualifications of students who apply and students who are admitted. These evaluations should highlight instances when campuses admit nonresidents who are less qualified than residents and should include corrective action steps. Moreover, this evaluation should include resident and nonresident undergraduate enrollment in majors at each campus. The university should make the results of this evaluation—including details of the academic qualifications of students who applied and who were admitted—publicly available.

8. To ensure the reasonableness of the compensation the university pays its executives, it should include—to the extent possible— all items of compensation when setting or adjusting salaries and benefits, when conducting surveys and studies, and when comparing the compensation packages of its executives to those in similar positions outside the university.

10. To improve the transparency and timeliness of its annual compensation report, the university should streamline the process it uses to prepare the report so it can be issued by April of each year.

12. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should immediately require that the campuses fully participate in all projects unless they can provide compelling evidence demonstrating a harmful effect.

13. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should, by June 30, 2016, to the extent possible, implement a process to centrally direct these funds to ensure that campuses use them to support the core academic and research missions of the university.

14. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should ensure that it substantiates that projects are actually generating savings and new revenue and that it can demonstrate how the university uses these funds.

15. To ensure that its recruiting efforts benefit residents, the university should prioritize recruiting residents over nonresidents. In particular, the university should focus its recruiting efforts broadly to ensure that it effectively recruits resident underrepresented minorities. For example, the university could establish a limit on the amount of funds it dedicates to nonresident recruiting. Further, it should develop a process to better track its nonresident and resident recruiting expenditures.

16. To determine if the campuses are using funds to further the goals of the University of California system and the Legislature, the Office of the President should begin regularly monitoring and analyzing how campuses are using both state funds and nonresident supplemental tuition. If, after the close of the fiscal year, the Office of the President determines that campuses are not using state funds and/or nonresident supplemental tuition in accordance with those goals, the Office of the President should take steps to correct the campuses' spending decisions as soon as possible.

17. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should track spending from state funds for programs that do not relate to educating students.

18. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should reevaluate these programs each year to determine whether they continue to be necessary to fulfill the university's mission.

19. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should explore whether the programs could be supported with alternate revenue sources.

22. To ensure that its rebenching efforts lead to equalized per-student funding among the campuses, the university should adopt a methodology that it can use, at least every three to five years, to update its weighting system to ensure the weight factors take into account campuses' actual costs of instruction, using the cost study that we recommend in Chapter 1 and other revenue sources if necessary.

23. To ensure that its rebenching efforts lead to equalized per-student funding among the campuses, the university should exclude from its rebenching calculation all state funding it uses for programs that do not directly relate to educating students. The university should exclude these programs only after it has evaluated them in accordance with the recommendation we made previously.

58. All universities should ensure that the differences between an informal or early resolution process and a formal investigation process are clearly explained to ensure that students know what to expect from each process. Further, they should explain that students whose cases are being handled under an informal or early resolution process have the right to move to a formal process at any time.

7. To ensure the ongoing accountability of the Office of the President, the regents should require it to implement our recommendations and report periodically on its progress.

9. To ensure that the Office of the President's financial safeguards are adequate, the regents should require the Office of the President to engage in a financial audit of only the Office of the President's operations.

14. To ensure that the Office of the President's staffing levels are justified and that costs are reasonable and align with the needs of campuses and other stakeholders, the regents should require the Office of the President to implement our recommendations and report periodically on its progress.

15. To ensure that the Office of the President is engaging in a thorough review of its systemwide and administrative costs and implementing our recommendations, the regents should develop a contract for an independent third party that can assist the regents in monitoring implementation of the three-year corrective action plan for the Office of the President. The independent third party should have expertise in higher education, public administration, and public finance. Moreover, the independent third party should have complete access to the Office of the President's documentation and its staff so that it has sufficient and appropriate information to verify the Office of the President's actions. The independent third party should report to the regents on the Office of the President's progress, challenges, and barriers to success at least quarterly.

17. To ensure that the Office of the President is engaging in a thorough review of its systemwide and administrative costs and implementing our recommendations, the regents should require the Office of the President to implement our recommendations and report periodically on its progress in doing so.

4. UC Davis should collect all late fees that its licensees owe.

9. To help ensure that university faculty and staff do not mishandle student reports of incidents, all faculty and staff should receive training annually, consistent with their role, on their obligations in responding to and reporting incidents of sexual harassment and sexual violence.

21. All universities should provide their education on sexual harassment and sexual violence to incoming students as close as possible to when they arrive on campus but no later than the first few weeks of their first semester or quarter. Further, universities should provide periodic refresher educational programs, at least annually, to all students on campus to ensure that they are aware of how to handle and report incidents of sexual harassment and sexual violence.

59. All universities should ensure that the differences between an informal or early resolution process and a formal investigation process are clearly explained to ensure that students know what to expect from each process. Further, they should explain that students whose cases are being handled under an informal or early resolution process have the right to move to a formal process at any time.

2. To demonstrate its willingness to fairly evaluate regional expenditures, Education should allow San Joaquin to reimburse its general fund for the vehicle purchase Education incorrectly disallowed.

14. To address a lack of detailed migrant program service and outcome data, Education should either expand the capabilities of its existing statewide databases or implement additional systems that would allow regions to capture more detailed data about migrant students.

1. To strengthen its administrative reviews and help ensure that school food authorities comply with the Buy American requirement, Education should update its written procedures to include a requirement that reviewers collect and retain evidence for all items they evaluate for compliance with the Buy American requirement. This update should occur no later than October 1, 2017.

22. To better understand the condition of school libraries statewide and to raise stakeholders' awareness of the State Education Board's adopted model standards, Education should work with Teacher Credentialing to assist it in identifying potential misassignments by providing staffing information reported by school districts to Teacher Credentialing by April of each academic year.

23. To better understand the condition of school libraries statewide and to raise stakeholders' awareness of the State Education Board's adopted model standards, Education should identify school districts that reported employing significantly fewer teacher librarians in fiscal year 2015-16 than in previous years and verify the accuracy of their fiscal year 2015-16 reports.

19. To provide stronger leadership with respect to school safety and nondiscrimination laws, Education, with direction from the superintendent of public instruction, should use data from the kids survey and reported suspensions and expulsions to evaluate the levels of discrimination, harassment, intimidation, and bullying students encounter and to determine the effectiveness of its own and the local education agencie's (LEA's) efforts, and report the results to the Legislature by August 1, 2014.

8. To ensure that districts, county offices, and schools receive guidance on a variety of safety issues and to comply with state law, CDE and DOJ should resume their partnership activities, as required by state law. Further, the partnership should update the 2002 handbook, Safe Schools: A Planning Guide for Action, and distribute it to all districts and county offices. If CDE or DOJ determine the need for additional funds to implement the legislative recommendations or to reestablish the partnership's activities, they should request those funds from the Legislature.

3. To ensure that all LEAs comply with federal special education requirements, Education should require them to include directly in a student's IEP document reasons for any changes to student placement or services.

8. To enable it to review additional areas of its special education program for quality assurance, Education should collect information about the frequency of the provision of each service contained in all students' individualized education programs (IEP). Education should then use this information to annually review the frequency of mental health services and follow up with SELPAs when it observes a significant reduction in the frequency of services.

9. To ensure that LEAs comply with federal and state requirements, Education should require all LEAs to use the IEP document to communicate the rationale for residential treatment and any potential harmful effects of such placement.

18. Education should analyze and report to the Legislature, by May 30, 2016, on the outcomes for students receiving mental health services statewide, including outcomes across the six performance indicators we identified, in order to demonstrate whether those services are effective. Once it has reported this statewide information, Education should provide each LEA throughout the State a report regarding the outcomes for the students the LEA served.

20. To ensure that the State knows the amount LEAs spend to provide mental health services for student IEPs, before the start of the 2017–18 fiscal year, Education should develop, and require all LEAs to follow, an accounting methodology to track and report expenditures related to special education mental health services.

29. To ensure that the State provides special education and related services to all eligible students, Education should investigate the difference between the estimated number of school aged children statewide who have a severe emotional disturbance and the number receiving mental health services through an IEP and determine the reason for such a discrepancy. Education should then take any steps necessary to assist LEAs in identifying and providing services to children who are severely emotionally disturbed.

3. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all Uniform Complaint Procedures (UCP) complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should distribute complaints and appeals to the correct divisions for investigation or review.

4. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should establish a single database to record and track all investigations of complaints and reviews of appeals. This database should capture all data necessary for Education to effectively make informed decisions related to UCP complaints or appeals. At a minimum, the database should capture the date on which Education received each complaint or appeal, the date on which it forwarded the complaint or appeal to the appropriate division for investigation or review, and the date on which it sent the decision to the complainant. The database should also include the type of complaint or appeal, the LEA involved, and the decision.

5. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should track the divisions' progress in processing complaints and appeals to ensure the divisions meet all UCP requirements, including documenting exceptional circumstances that constitute good cause for extending investigations beyond 60 days.

6. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should work with divisions to establish policies and procedures for the divisions to follow when investigating UCP complaints and reviewing appeals. The procedures should identify the individuals or units responsible for investigating complaints and reviewing appeals, the steps and time frames for conducting investigations and reviews, the requirements for issuing decisions, and the documentation that should be retained in the files.

7. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should establish and distribute a standard investigation report format that includes the required elements for the divisions to use when processing UCP complaints.

8. To ensure that it consistently processes complaints and appeals in a timely manner and that it investigates and reviews all UCP complaints and appeals in compliance with state law and regulations, by July 2017 Education should designate a central office to receive all complaints and appeals. This central office should monitor the divisions' decisions and reports on complaints and appeals to ensure that they comply with requirements.

9. To ensure that its regulations are consistent and align with state and federal requirements, by July 2017 Education should initiate revising its regulations to require its divisions to complete investigations of complaints and reviews of appeals related to all programs within 60 days of Education receiving them, including providing its decisions in writing to complainants, unless otherwise specified in statute or federal regulations.

10. To ensure that its regulations are consistent and align with state and federal requirements, by July 2017 Education should initiate revising its regulations to allow Nutrition Services to investigate all complaints as direct intervention.

16. To increase the efficiency and effectiveness of LEAs' UCP processes, Education should work with those LEAs throughout the State that receive a disproportionately high number of non-UCP complaints through the UCP process to assess the potential benefits of establishing similar mechanisms.

22. To ensure that its regulations are consistent and align with state and federal requirements, Education should revise its regulations to allow LEAs to extend investigations under exceptional circumstances that constitute good cause if the LEAs document and support with evidence the reasons for the extensions.

23. After it makes the recommended regulatory changes to allow extensions under exceptional circumstances, Education should review LEAs' extensions to investigations as part of its Federal Program Monitoring to ensure that LEAs' documentation is sufficient and that their reasons adequately justify such extensions.

28. Education should revise UCP regulations to formally establish uniform time limits for filing all types of complaints.

18. To strengthen its monitoring of staff assignments, Teacher Credentialing should work with Education to identify potential misassignments by comparing annually the staffing information reported by school districts to Education against Teacher Credentialing's credentialing records. Further, Teacher Credentialing should incorporate misassignments identified using Education's data into its existing notification, reporting, and sanctioning structure. If Teacher Credentialing believes it needs express statutory authority to do so, it should seek it.

23. To reduce the risk that it may not identify an armed prohibited person, Justice should revise its electronic matching process to use all personal identifying numbers available in its databases.

34. To ensure that it fully supports its decision to apply federal prohibition terms to individuals, Justice should review all applicable federal and state laws and continue to seek clarification from the United States Department of Juctice's Bureau of Alcohol, Tobacco, Firearms, and Explosives (ATF) and any other appropriate federal agencies to determine whether California's firearms restoration process meets federal criteria and, if not, why it does not. Justice should issue a report to the Legislature, within one year, detailing the results of its review and, if applicable, communicate why California's restoration process does not meet federal criteria and the impact that it has on prohibited persons who live in California.

18. To ensure that Social Services receives criminal history information within 14 days of receiving an individual's fingerprint information, as state law requires, by July 2017 Justice should analyze its process, including delayed transmissions; implement changes to address problems it identifies; and regularly measure itself against the requirement to determine whether it is meeting its statutory requirement.

19. To ensure that it has complete disposition information, Justice should coordinate with the Judicial Council at least once a year to share information about court-reporting gaps and to determine the need to distribute additional information to courts about reporting requirements and the manner in which to report. In addition, Justice should reconvene its advisory committee and meet on a regular basis to discuss, at a minimum, improving the frequency and timeliness with which courts report dispositions to Justice and law enforcement agencies report arrest information to Justice.

20. To ensure that it is receiving all arrest information from law enforcement agencies, at a minimum, Justice should consider trends in the number of arrest reports each law enforcement agency sends it and the number of reports that it might expect to receive from an agency given the agency's size, location, and reporting history. Whenever Justice identifies a law enforcement agency that it determines may not be reporting all required information, it should request that the agency forward all required arrest information.

9. To ensure that districts, county offices, and schools receive guidance on a variety of safety issues and to comply with state law, CDE and DOJ should resume their partnership activities, as required by state law. Further, the partnership should update the 2002 handbook, Safe Schools: A Planning Guide for Action, and distribute it to all districts and county offices. If CDE or DOJ determine the need for additional funds to implement the legislative recommendations or to reestablish the partnership's activities, they should request those funds from the Legislature.

4. To report to the Legislature about the effectiveness of its Rapid DNA Service (RADS) program and to better inform decisions about expanding the number of analyzed sexual assault evidence kits, Justice should amend its agreements with the counties participating in the RADS program to require those counties to report case outcome information, such as arrests and convictions for the sexual assault evidence kits Justice has analyzed under the program. Justice should then report annually to the Legislature about those case outcomes.

10. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, developing best practices based on the requirements stated in the federal regulations, the state guidelines and state law, and advising user agencies on the implementation of those practices. The best practices should include, but not be limited to, reviewing criminal intelligence, appropriately disseminating information, performing robust audit practices, establishing plans to recover from disasters, and meeting all of the State's juvenile notification law requirements. Justice should guide the board and the committee to develop these best practices by June 30, 2017.

11. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing user agencies that use CalGang to complete a comprehensive review of all the gangs documented in CalGang to determine if they meet the necessary requirements for inclusion and to purge from CalGang any groups that do not meet the requirements. Justice should guide the board and the committee to ensure that user agencies complete this review in phases, with the final phase to be completed by June 30, 2018.

12. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing all user agencies to complete a comprehensive review of the records in CalGang to determine if the user agencies have adequate support for the criteria associated with all the individuals they have entered as gang members. If the user agencies do not have adequate support, they should immediately purge the criteria—and, if necessary, the individuals—from CalGang. In addition, the user agencies should ensure that all the fields in each CalGang record are accurate. Justice should guide the board and the committee to ensure that user agencies complete this review in phases, with the final phase to be completed by September 30, 2019.

13. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing all user agencies to report to Justice every six months, beginning in January 2017, on their progress toward completing their gang and gang member reviews.

19. To promote transparency and encourage public participation in CalGang's meetings, Justice should post summary results from the committee's audits of CalGang records to its website unless doing so would compromise criminal intelligence information or other information that must be shielded from public release.

2. Unless the Legislature directs the board to eliminate the compliance fund's excess fund balance within a time frame of more than a year, the board should eliminate the excess fund balance by June 30, 2017, by using it to offset the licensing program's annual funding shortfall. The board should also limit the fund's future balance to no more than two months' worth of licensing program expenditures.

11. To make certain that money from the special plate funds pay only for allowable and supportable activities, Cal EMA should maintain documentation to support its charges to the antiterrorism fund. For example, it should ensure that employees submit signed time reports to support the time they spend on antiterrorism-related activities.

1. The authority should amend its regulations to bring them into accord with the Children's Hospital Bonds Act of 2004 (2004 act), thus allowing any eligible hospital to apply for the 2004 act's funds that remained as of June 30, 2014.

1. To ensure that the compensation the AOC provides is reasonable, the Judicial Council should adopt procedures that require a regular and thorough review of the AOC's compensation practices, including an analysis of the job duties of each position to ensure that the compensation aligns with the requirements of the position. This review should include comparable executive branch salaries, along with a justification when an AOC position is compensated at a higher level than a comparable executive branch position.

3. To ensure that its compensation structure is reasonable, the AOC should mirror the executive branch's practices for offering leave buyback programs in terms of frequency and amount.

4. To increase its efficiency and decrease its travel expenses, the AOC should require its directors and managers to work in the same locations as the majority of their staff unless business needs clearly require the staff to work in different locations than their managers.

11. To reduce its expenses, the AOC should cease its excessive reimbursements for meals by adopting the executive branch's meal and travel reimbursement policies.

14. To ensure that it spends funds appropriately, the AOC should develop and implement controls to govern how its staff can spend judicial branch funds. These controls should include specific definitions of local assistance and support expenditures, written fiscal policies and procedures as the rules of court require, and a review process.

19. The AOC should conduct a comprehensive survey of the courts on a regular schedule—at least every five years—to ensure that the services it provides align with their responses. The AOC should re-evaluate any services that the courts identify as being of limited value or need.

20. To justify its budget and staffing levels, the AOC should conduct the steps in CalHR's workforce planning model in the appropriate order. It should begin by establishing its mission and creating a strategic plan based on the needs of the courts. It should then determine the services it should provide to achieve the goals of that plan. The AOC should base its future staffing changes on the foundation CalHR's workforce planning model provides. Finally, the AOC should develop and use performance measures to evaluate the effectiveness of this effort.

21. To ensure that it provides services to the trial courts as efficiently as possible, the Judicial Council should explore implementing a fee-for-service model for selected services. These services could include those that are little used or of lesser value to the trial courts, as identified in our survey that we discuss in Chapter 3.

22. To justify the budget and staff level of the AOC, the Judicial Council should implement some or all of the best practices we identified to improve the transparency of AOC spending activities.

3. To improve the usefulness of the Judicial Council's semiannual reports until a statutory requirement is enacted, the AOC should work with the Judicial Council to pursue a cost-effective method to include new contracts and the complete history of contracts amended during the reporting period in the semiannual reports, including the date of the original contract; the original contract amount and duration; all subsequent contract amendments; and the date, amount, and duration of each such amendment. The AOC should present this information beginning with the semiannual report covering the July 1, 2014, through December 31, 2014, reporting period.

4. To improve the usefulness of the Judicial Council's semiannual reports until a statutory requirement is enacted, the AOC should work with the Judicial Council to pursue a cost-effective method to begin tracking additional information in its data systems for inclusion in the semiannual reports. This information should include whether a contract was competitively bid, the justification if it was not competitively bid, and whether the contract was with a Disabled Veteran Business Enterprise. For information technology contracts, the AOC should identify whether the contract was with a small business. The AOC should present this information beginning with the semiannual report covering the July 1, 2014, through December 31, 2014, reporting period.

5. The AOC should implement all of the best practices related to general and business process application controls as outlined in the U.S. Government Accountability Office's Federal Information System Controls Audit Manual no later than December 31, 2014, thereby strengthening and continuously monitoring the effectiveness of the controls over its information systems. In addition, the AOC should immediately begin implementing improvements to its controls over access to its information systems and place these improvements into effect by February 2014. Finally, the AOC should provide guidance and routinely follow up with the superior courts—requiring updates every six months until all identified issues are corrected—to ensure that they make the necessary improvements to their general and business process application controls.

6. The AOC; the Supreme Court; and the first, second, and fourth districts should implement procedures to ensure that they follow a competitive process for their procurements when required.

13. The AOC should revise the judicial contracting manual to require judicial entities to maintain documentation on their determinations of fair and reasonable pricing for purchases under $5,000.

17. The AOC should revise the judicial contracting manual to require that judicial entities maintain documentation for their evaluation and selection process used for competitive procurements. The AOC should also strengthen its procedures to ensure that bid evaluations are conducted properly and calculated correctly.

20. The AOC, Habeas Corpus Resource Center (HCRC), Supreme Court, and fourth and fifth districts should implement procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented. Additionally, the AOC should ensure that it prepares the appropriate documentation when it amends a contract that it has competitively solicited and the amendment includes a change that was not evaluated in the original competitive process.

25. The AOC should implement procedures to ensure that its internal controls over payments are followed and that procurements are approved before ordering and receiving goods and services.

4. To enhance the value of the federal Help America Vote Act of 2002 (HAVA) spending plan as a transparency and accountability tool for the Legislature, the Office should make the following modifications to its annual HAVA spending plan:

Clearly state the methodology used to report prior HAVA expenditures in the HAVA spending plan. Such a methodology should use the financial information contained in its accounting system.

Reconcile the prior HAVA expenditures with the year-end financial reports the Office provides to the California State Controller's Office.

Present prior HAVA expenditures by activity and by specific appropriation.

5. To ensure the State complies with the National Voter Registration Act of 1993 (NVRA), the Office should take all necessary steps, including seeking any necessary legislative changes, and work with the DMV to modify the driver's license application so that it may simultaneously serve as a form for voter registration.

11. The Secretary of State should adopt regulations establishing clear criteria for mistakes in election-related materials that constitute reportable errors and require counties to report these errors to it after each election.

12. Beginning in December 2018, the Secretary of State should implement annual risk-based reviews of a selection of county election officials' offices to ensure their compliance with state election laws and regulations.

13. To inform and enhance the guidance it provides to county election officials, the Secretary of State should analyze error reports and its risk-based review results to focus its guidance on topics most relevant to improving elections throughout the State.

7. To ensure that the audit and review unit's random audits of closed case files provide an effective oversight mechanism, the State Bar should follow its policy to conduct and record meetings and trainings related to the audit report's recommendations.

6. To assign purchasing cards only to appropriate staff, ensure that the State Bar's records of employees' credit limits reflect those established with the bank, and to verify that staff use purchasing cards only for allowable and necessary expenses, the State Bar should immediately develop a policy that requires justification of the business needs for employees to receive purchasing cards, and use this policy to limit the number of staff issued a purchasing card.

7. To assign purchasing cards only to appropriate staff, ensure that the State Bar's records of employees' credit limits reflect those established with the bank, and to verify that staff use purchasing cards only for allowable and necessary expenses, the State Bar should immediately restrict the use of purchasing cards to its original purpose, which was for low-dollar and frequently occurring purchases. For purchases above $5,000, the State Bar should require the vendor to bill for payment.

9. To ensure that its costs are reasonable and appropriate, the State Bar should update its meal and catering policy to align with the meal policy of the State's Executive Branch and should require individuals attending committee meetings for the State Bar to comply with standard meal per diem rates.

15. To reduce its reliance on outside legal counsel, the State Bar should continue its efforts to hire staff to fill its remaining vacant attorney positions.

17. To better measure how well its attorney discipline program is meeting the State Bar's core mission to protect the public from attorney misconduct, the State Bar should, by December 2017, identify key goals and metrics for the attorney discipline system.

2. To ensure that it maximizes its cost-recovery efforts related to the Client Security Fund, the State Bar should adopt a policy to file for money judgments against disciplined attorneys for all eligible amounts as soon as possible after courts settle the discipline cases.

9. To ensure that it accounts appropriately for information technology project costs and their related funding sources, the State Bar should develop a reasonable method for allocating information technology project costs.

10. To ensure that it accounts appropriately for information technology project costs and their related funding sources, the State Bar should apply its new cost-allocation method to the costs of its Technology Improvement Fund.

4. To improve the accuracy of information in the leave accounting system and to ensure that agencies do not improperly credit employees with leave in the future, the state controller should implement additional controls by June 2015 to prevent the leave accounting system from processing the types of inappropriate transactions we identified in our statewide electronic analysis. For example, it could develop cost-effective controls in the leave accounting system that would prevent employees from receiving annual leave and sick leave during the same pay period.

15. To make certain that money from the special plate funds pay only for allowable and supportable activities, Resources should use all appropriate funding sources to pay for any expenses that benefit multiple programs in proportion to the benefits these programs actually receive. Further, it should ensure that its allocation of such expenses to different funds is equitable and supported.

4. To fully comply with state contracting law, DWR should ensure that it competitively selects architectural and engineering consultants based on demonstrated competence and professional qualifications. In addition, DWR should document in the contract file its evaluation of the competence and professional qualifications of all contractors and any subcontractors that are added to the contract subsequent to the competitive selection process.

5. To ensure that only qualified subcontractors are added to contracts after the initial award is made, DWR should make sure that contractors select their own subcontractors and that DWR subsequently approves the selection after it verifies their qualifications.

7. To ensure that DWR manages WaterFix in an effective manner, DWR should complete both the economic analysis and financial analysis for WaterFix and make the analyses publicly available as soon as possible.

9. In order to prepare for the potential approval of WaterFix and to ensure that the project is managed properly during the design and construction phase, DWR should develop and update when necessary the associated program management plan for the design and construction phase of the project.

7. To reduce the risk of fraudulent applications, by September 2017 DMV should seek interagency agreements with the health boards responsible for licensing providers authorized to certify disabilities on placard applications. The agreements should include, but not be limited to, a review by medical experts of a sample of placard applications each quarter to ensure that the disability certifications meet state requirements. For any application that does not meet state requirements, DMV should require that the applicant and his or her provider submit the information needed so that the application meets state requirements. DMV should cancel the placards of those who do not respond within 90 days.

8. To reduce the risk of fraudulent applications, by September 2017 DMV should seek interagency agreements with the health boards responsible for licensing providers authorized to certify disabilities on placard applications. The agreements should include, but not be limited to, a process for the health boards to develop guidance for medical providers related to how to meet state requirements.

14. To properly equip its employees with the knowledge necessary to identify and report potential fraud indicators in placard applications, DMV should provide employees who process applications with training specific to the types of fraud that can occur in an application. This training should be provided by December 2017 and every other year thereafter.

18. To better equip local parking enforcement officials to promptly identify invalid placards, by December 2018 DMV should develop and implement an application, database, or other technology that will allow non-sworn parking enforcement officials to have immediate access to information on placard status.

1. To ensure that programs supported by special plates receive appropriate amounts of revenues due to them, DMV should annually collect all fees for special plates that are no longer on a vehicle but are retained by the plate owner.

3. DMV should assess the extent to which it has charged fees for special plates that are not consistent with those prescribed in statutes and take appropriate action.