2017-041 - Status of Recommendations - Table 2

7. To ensure that it adequately tracks critical information related to its basic functions and mission, the commission should work with Consumer Affairs to ensure that the new online program will meet its needs and requirements. Once the program is in place, the commission should use it as its central means for tracking its operations.

9. To ensure that it accurately collects revenue, the commission should calculate the pension assessment by counting all the complimentary tickets issued, except for working complimentary tickets, not merely the complimentary tickets that are redeemed. If the commission does not agree that it should calculate the pension assessment by counting all the complimentary tickets issued, it should seek a change in its regulations to calculate the fee based only on the number of complimentary tickets redeemed.

22. The commission needs to establish regulations that describe its process for determining its ticket assessment for the neurological account so that it avoids the use of underground regulations.

28. To comply with state law governing the pension plan, the commission needs to, after it has an accurate and complete listing of all licensed athletes and box office information by event type, conduct the analysis to determine the feasibility of expanding the pension plan to cover all athletes and report the results to the Legislature.

17. To the extent that Consumer Affairs chooses to implement BreEZe at the phase 3 regulatory entities, it should first complete a formal cost-benefit analysis to ensure that BreEZe is a cost-effective solution to meet these regulatory entities' business needs. To make certain this analysis is complete, it should include an assessment of the potential changes these regulatory entities may require to be made of the BreEZe system and the associated costs. Consumer Affairs should complete the cost-benefit analysis before investing any more resources into the implementation of BreEZe at the phase 3 regulatory entities, and it should update this analysis periodically as significant assumptions change.

35. To ensure that future training for BreEZe system rollouts is timely and effective, Consumer Affairs should provide training on the BreEZe system as close to the rollout date as possible to ensure that staff retain the information for using the system as it is implemented.

36. To ensure that future training for BreEZe system rollouts is timely and effective, Consumer Affairs should work with the regulatory entities to develop training that is specific to each entity's business processes.

35. Following the completion of the analysis (described in Recommendation 34), the Medical Board should take the appropriate follow-up actions that it deems necessary, including the investigation of physicians identified in its analysis.

1. The physical therapy board should explore the feasibility of establishing a state position to perform the duties of its current in-house consultant at a reduced cost.

3. To maximize the usefulness of the information it makes available to stakeholders and to increase accountability, the board should create policies and procedures that include clear, comprehensive guidance to counties about all aspects of performance outcome and expenditure reporting. At a minimum, such guidance should include specifying how counties should define when a juvenile has received a service and whether certain services, such as training, should qualify as serving juveniles.

5. To maximize the usefulness of the information it makes available to stakeholders and to increase accountability, the board should consider verifying the counties' data by conducting regular site visits on a rotating basis or by employing other procedures to verify data that counties submit.

12. To ensure that counties do not maintain excessive balances of unexpended block grant funds, the board should develop procedures to monitor counties' unspent funds and follow up with them if the balances become unreasonable.

5. To ensure that it can better monitor how its medical staff and contractors adhere to the informed consent requirements of Title 22, sections 70707.1 through 70707.7, the Receiver's Office should develop a plan by August 2014 to implement a process by December 2014 that would include working with Corrections to establish a process whereby inmates can have witnesses of their choice when consenting to sterilization, as required by Title 22, or working to revise such requirements so that there is an appropriate balance between the need for secure custody and the inmate's ability to have a witness of her choice.

11. To ensure that inmates receive only medical services that are authorized through its utilization management process, the Receiver's Office should ensure that the computer system it procures includes functionality to electronically link medical scheduling with authorization through the utilization management process to prevent all unauthorized procedures, regardless of whether they may result in sterilization, from being scheduled.

4. Adjust current employees' leave balances in the leave accounting system to correct any improper charging of leave identified by the audit.

6. In instances where the audit has determined that an employee's leave balance was mischarged but the employee subsequently departed state service, take appropriate measures to remedy any resulting incorrect compensation of the employee for unused leave upon his or her departure, including by seeking repayment of any amount overpaid to the employee.

19. Corrections and Correctional Health Care should run a query of exempt positions related to the chief psychologist classification, such as clinical psychologists and senior psychologists, to determine whether any other exempt employees were improperly credited or paid for on-call or call-back assignments prior to December 2014, and seek recovery through reducing those employees' accumulated leave balances.

2. To ensure it can demonstrate that its fraud prevention efforts are maximizing financial recoveries for the beverage program, CalRecycle should both modify and annually update its fraud management plan to include the following:
- By December 31, 2014, formally establish a systematic process for analyzing, monitoring, and responding to the risk of fraudulent recycling of out-of-state beverage containers.
- Develop fraud estimates—by type of fraudulent activity—that quantify the potential financial losses to the beverage program and the methodology CalRecycle used to develop these estimates.
- Identify the amount of actual fraud in the prior year by type of fraudulent activity, such as the financial losses resulting from the redemption of out-of-state beverage containers or the falsification of reports used to substantiate program payments.
- Identify the amount actually recovered for the beverage program in the form of cash for restitution and penalties resulting from fraud.

3. To allow for public input and to prevent any legal challenges claiming that its policies and procedures regarding prepayment holds constitute unenforceable underground regulations, CalRecycle should adopt these policies and procedures as regulations in accordance with the Administrative Procedure Act.

5. To ensure it effectively uses resources, CalRecycle should renegotiate its agreement with the California Department of Justice (Justice) to pay based on the cases CalRecycle refers, limiting costs to investigations over a predefined period of time. CalRecycle could agree to increase the budget for a particular investigation if Justice demonstrates that it has developed promising leads.

1. Take appropriate corrective action against the district engineer and the supervisors for their participation in or failure to address the conflict of interest.

3. Provide training to those responsible for reviewing statements of economic interests regarding how to identify conflicts of interests and when to consult with legal counsel.

7. If regional water boards continue to include staff enforcement costs in the penalty actions they issue, the state water board should revise its staff cost rate to reflect actual staff salaries and overhead cost for the certification program.

5. To better utilize unused space at the veterans homes, and to serve more veterans within legal restrictions, CalVet should use the information in the Yountville veterans home master plan to develop a plan for using unused space at that home to generate revenue and/or serve more veterans.

11. To ensure it is maximizing its ability to serve veterans in the State's veterans homes, CalVet should follow through with its plan to assess the bed capacity of the homes for veterans' needs after the homes in Fresno and Redding are licensed, to determine the most appropriate number of beds for the different levels of care offered at each home.

15. To allow for public input and to prevent any legal challenges that its policy of capping members' fees, its $165 monthly income-retention policy for members, and its age and admission policy are unenforceable underground regulations, CalVet should adopt these policies as regulations in accordance with the Administrative Procedure Act.

17. To more effectively and efficiently meet state purchasing and procurement requirements, CalVet should continue implementing quality assurance policies to strengthen its oversight of its purchasing practices, including conducting on-site reviews of the purchasing practices of the veterans homes.

3. Establish a routine of completing a monthly inventory of expendable state property after the barcode system is implemented.

2. The commission should determine the cause of its lack of compliance with state law requiring it to issue award decisions within 75 days of the date an intervenor submits a compensation claim, and it should determine what actions to take to rectify the problem. The commission should ensure that it has sufficient information, such as detailed tracking information regarding claims, to identify where in the process delays are occurring. If the commission determines that the current 75-day statutory period is unreasonable, it should seek a change in state law.

5. To comply fully with state law, the commission should conduct a comprehensive market rate study and update it periodically.

14. The commission should follow the requirement in state law to inspect and audit the accounting records of utilities it regulates within required time frames. If the commission chooses to continue to meet this requirement through the general rate case process, it should ensure that all utilities file a general rate case on a regular schedule so as to comply with the state law's audit requirement. However, the commission should follow alternate methods to comply with the audit requirement when a utility will not be filing for its general rate case in time to be audited within three or five years, depending on the timing of the required audit for that utility.

2. To ensure that it resolves complaints against passenger carriers in a timely manner, the commission should establish a method for prioritizing complaints and it should implement a policy specifying the maximum amount of time between the receipt of a complaint and the completion of the subsequent investigation. Further, the commission should require branch management to monitor and report regularly on its performance in meeting that policy.

6. To ensure that the branch conducts thorough investigations of passenger carriers, the commission should implement a formal training program to ensure that all investigators have adequate knowledge and skills related to regulating passenger carriers.

9. To ensure that passenger carriers submit accurate fee payments, the commission should require its fiscal staff to implement a process to verify passenger carrier fee payments and associated revenue.

10. To ensure that it complies with state law and uses passenger carrier fees appropriately, the commission should implement a process to ensure that passenger carrier fee revenues more closely match related enforcement costs.

12. To detect and deter carriers from operating illegally at airports, the branch should use as intended the five positions added for passenger carrier enforcement at airports. If the branch chooses not to designate five positions solely for this purpose, then it must be prepared to demonstrate regularly that an equivalent number of full-time positions are working on this activity.

13. To strengthen its leadership and ensure passenger carrier and public safety, the branch should produce a draft strategic plan by December 31, 2014, with a final strategic plan completed as the commission specifies. The strategic plan should include goals for the program; strategies for achieving those goals, including strategies for staff development and training; and performance measures to assess goal achievement.

3. To ensure that policy makers, enforcement officials, and the general public have access to accurate consumer complaint data in CIMS, the branch should continue to implement its quality management team program component focused on reviewing the categorization of complaints and correcting identified errors.

4. To ensure that policy makers, enforcement officials, and the general public have access to accurate consumer complaint data in CIMS, the branch should develop and implement tools by September 30, 2015, to measure the quality management team program's effectiveness.

11. The commission should ensure that it complies with all policy requirements in SAM Chapter 5300 no later than April 2016.

12. As part of developing, implementing, and maintaining an entitywide information security program, the commission should complete and maintain inventory of all its information assets, specifically categorizing the level of required security of the information assets based on the potential impact that a loss of confidentiality, integrity, or availability of such information would have on its operations and assets.

13. As part of developing, implementing, and maintaining an entitywide information security program, the commission should develop a risk management and privacy plan and conduct an assessment of risks facing its information assets.

14. As part of developing, implementing, and maintaining an entitywide information security program, the commission should develop, implement, and maintain an information security plan as part of its entitywide information security program.

16. The commission should revise its existing recovery plan to include a list of applications supporting critical business functions, their maximum acceptable outage time frames, and detailed recovery strategies for each application.

17. The commission should revise its existing recovery plan to include detailed procedures for rebuilding its technology infrastructure at an alternate processing site.

18. The commission should conduct regular tests and exercises to assess the sufficiency of the revised recovery plan and refine the plan when necessary.

2. To ensure that the choice of a vendor is sufficiently justified and that the vendor represents the best value, the CPUC should explain in its final decision how the vendor was the most qualified in all cases when the CPUC does not competitively select the vendor it directs utilities to contract with.

4. To avoid the appearance of inappropriate relationships, the CPUC should adopt a policy to prohibit commissioners from accepting gifts from regulated utilities and energy companies and free travel from organizations with significant ties to regulated utilities and other parties with financial interests in CPUC proceedings.

6. To ensure that its contracting practices align with state requirements and best practices, the CPUC should update, distribute, and follow its contracting procedures manual. The manual should identify specific responsibilities for both contracts office staff and project managers, and it should provide specific guidance about the processes the CPUC will employ to do the following:

-Fully justify civil service exemptions.

-Conduct market research for exempt contracts.

-Fully support the need for additional funding.

-Ensure that it does not change the scope of work too significantly from the original.

-Monitor contractor performance against criteria included in its contracts.

-Avoid sole-source contracts when it is able to solicit competitive bids for services.

7. To ensure that its contracting practices align with state requirements and best practices, the CPUC should provide immediate refresher training to its contract analysts and contracts office manager, and establish a regular schedule of annual training for them to attend.

8. To ensure that its contracting practices align with state requirements and best practices, the CPUC should designate a limited number of project managers for each division at the CPUC, and provide those individuals with training on the CPUC's processes related to contracting, including how to monitor progress of a contractor's work.

10. The CPUC should update its general policy on responding to California Public Records Act requests so that the policy aligns with state law.

11. The CPUC should develop and follow procedures to regularly track and review California Public Records Act requests it has not fully responded to and determine whether it can provide information.

13. The CPUC should update its regulations to require parties joining a proceeding by filing a protest or response to an application or petition, or by filing comments in response to a rulemaking proceeding to fully disclose their interests in the proceeding.

14. The CPUC should ensure that it has accurate information about who is required to file statements of economic interests and then verify that all such persons file those statements when required.

15. The CPUC should update and follow its retention policy for economic interest disclosures so that it is aligned with state law.

3. To show how air pollution emissions reductions related to the solar initiative benefit the State, the commission should include in future reports the measurable benefits of those reductions.

7. To further its mission to obtain the lowest possible rates for reliable and safe utility service for ratepayers through its reviews of balancing accounts, Ratepayer Advocates should use the commission's list of balancing accounts to guide its selection of the number, size, and type of balancing accounts to review so that its review coverage is more proportional across all utilities.

2. To correct the erroneous leave hours we identified in our analysis of the leave accounting system, CalHR should work with the state controller and all state agencies under its authority to review and take the appropriate action to correct the errors by January 2015.

7. To ensure that state agencies accurately account for their employees' leave benefits, CalHR should consolidate guidance by January 2015 regarding the appropriate amount of leave that employees should earn each month and provide these criteria to the state controller to use when developing the leave accounting system's monthly exception reports. For example, CalHR should identify the number of holiday credit hours that employees covered by each collective bargaining agreement should receive for working on a holiday.

10. To ensure that state agencies accurately account for their employees' leave benefits, CalHR should provide additional guidance to state agencies by January 2015 on interpreting the provisions of the collective bargaining agreements related to the amount of leave employees earn. For example, CalHR could provide scenarios to illustrate the number of hours employees should earn under common circumstances.

14. To ensure that IT projects have the oversight needed to better position them for success, CalTech should develop thresholds relating to IT project cost increases and schedule delays to inform and better justify its decision to allow an IT project to continue. If a department's IT project reaches or exceeds these thresholds, CalTech should require the department to conduct a cost-benefit analysis for the project and include this analysis in an SPR. CalTech should consider the results of this analysis in its decision to approve or deny the SPR and, if warranted, take action to suspend or terminate the project so that it does not allow projects with significant problems to continue without correction.

28. To ensure that contracting departments comply with requirements, CalTech should periodically monitor the status of contracts and enforce adherence to procurement requirements. For example, CalTech should ensure that contracting departments forward all change requests to it and that it conducts reviews of all change requests to determine whether the changes exceed the scope of the respective contract.

14. Although the Technology Department indicated that its intent is not to outsource its statutory responsibility for IPO, in any instances where its staff conduct a portfolio review of a project's IPO, the Technology Department should, by December 2016, establish a process for its review of documents created by the agency's IPO contractor that includes verifying whether these reports include critical analysis of project progress and vendor performance so it can intervene when necessary.

6. To help ensure the independence and objectivity of IPO analysts working in the oversight and consulting division, CalTech should provide regular training regarding maintaining independence while conducting project oversight.

9. To ensure that the sponsoring agencies' project status reports provide a reliable and consistent assessment of an IT project's progress, CalTech should develop and adopt specific standards that describe how to calculate and report the project's current status.

10. To attract and retain employees with appropriate experience and qualifications to perform IT project oversight, CalTech should continue its efforts to gain approval to modify and use the project manager classification for the IPO analyst role.

13. To improve its oversight training, by June 2015 CalTech should continue to implement a consistent and repeatable training plan for IPO analysts, which includes contract management, project assessment, IT systems engineering, and maintaining independence.

3. To assist reporting entities in reaching full compliance with the security standards, the technology department should ensure the consistency and accuracy of its self certification process by developing a self assessment tool by December 2015 that reporting entities can use to determine their level of compliance with the security standards. The technology department should require reporting entities to submit completed self assessments along with their self certifications.

4. To assist reporting entities in reaching full compliance with the security standards, the technology department should provide more extensive guidance and training to reporting entities regarding the self certification process, including training on how they should use the new self assessment tool.

8. The technology department should revise its certification form to require reporting entities to submit detailed information about their compliance with the security standards. It should use this information to track and identify trends in the State's overall information security.

9. The technology department should develop policies and procedures to define the process and criteria it will use to incentivize entities' compliance with the security standards.

10. To improve the clarity of the security standards, the technology department should perform regular outreach to all reporting entities to gain their perspectives, identify any unclear or inconsistent security standards, and revise them as appropriate.

2. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should ensure that the project management system can centrally track and extract all data regarding project status, including time delays, cost overages, and the reasons for each.

3. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should track the reasons that projects are pending to identify its true backlog of projects. In doing so, it should develop a process to follow up on those projects that are pending to ensure that they are not on hold unnecessarily and are appropriately moving forward.

4. To ensure long-term efficient and effective delivery of projects, the division, in its planned implementation of its new project management system in July 2017, should, at least annually, use the centrally tracked data to identify common themes in the causes for project delays and cost overages and develop solutions to address these issues. Further, it should report the results of its review to General Services' executive management.

5. Until the division implements its planned project management system, it should, by September 2016, develop a process to, at a minimum, identify project status and reasons for project delays as well as cost overages. Using these data, the division should modify its project management processes to ensure the efficient and effective delivery of projects.

6. The division should develop and implement a process for preparing reasonable time frames and cost estimates for its projects within the building management branch. To better inform the development of this process, the division should evaluate the branch's structure, which should include a staffing analysis, to determine whether it is effectively organized and whether it should add cost estimator positions.

7. To ensure that client agencies are paying equitable rates, by December 2016 General Services should develop and implement a strategy for allocating its administrative costs equally among all the projects it completes for client agencies, including those portions outsourced to private firms.

8. To ensure that the project management branch charges its client agencies a competitive hourly rate, by December 2016 and every two years thereafter, the division should conduct a rate analysis that fully accounts for differences between the project management branch's rate and private firms' rates. If it finds that the rates are not competitive, the division should identify and implement strategies to ensure that the project management branch's rates are as competitive as they can be with those of its private firm counterparts. Further, the division should explore and implement any other reasonable methods to ensure that it is delivering projects as cost effectively as possible.

9. To improve its communication with client agencies, the division should ensure that project managers are using consistent procedures by providing specific expectations related to communicating and documenting time delays, cost changes, and change orders, at a minimum.

10. To improve its communication with client agencies, the division should develop a process for providing periodic detailed bills and invoices to client agencies clearly describing the work for which it is charging.

11. To effectively evaluate the performance of its branches in delivering projects, the division should develop meaningful goals and objectives and a method of measuring its success in achieving them as part of its strategic plan that is focused on ensuring that projects are delivered on time and within budgeted cost estimates.

12. To ensure that its project management staff are adequately trained and have the information necessary to deliver projects as efficiently and effectively as possible, the division should conduct a comprehensive survey every other year of all of its client agencies to inform necessary improvements to its processes and training program and, in the interest of transparency, make the survey results public.

13. To ensure that its project management staff are adequately trained and have the information necessary to deliver projects as efficiently and effectively as possible, the division should develop and implement by December 2016 a periodic training program for staff within its project management and building management branches. This training program should include updated information that reflects any processes it revises based on its review of critical project status data and its progress toward meeting its goals.

1. To ensure public safety and provide public assurance that school districts construct projects in accordance with approved plans, the department, in conjunction with the division, should pursue legislative changes to the Field Act that would prohibit occupancy in cases in which the division has identified significant safety concerns.

2. To increase its efforts to prevent and control diabetes, Public Health should develop a process for identifying and applying for federal funding opportunities, including routinely and proactively searching for grants. In addition, Public Health should seek funding for a grants specialist position to identify and apply for federal and other grants.

3. To protect the health, safety, and well-being of residents in long-term health care facilities, Public Health should improve its oversight of complaint processing. Specifically, by May 1, 2015, Public Health should establish a specific time frame for completing facility-related complaint investigations and ERI investigations and inform staff of the expectation that they will meet the time frame. Public Health should also require district offices to provide adequate, documented justification whenever they fail to meet this time frame.

4. To protect the health, safety, and well-being of residents in long-term health care facilities, Public Health should improve its oversight of complaint processing. Specifically, by May 1, 2015, Public Health should develop formal written policies and procedures for PCB to process complaints about certified individuals in a timely manner. These policies and procedures should include specific time frames for prioritizing and assigning complaints to investigators, for initiating investigations, and for completing the investigations. Public Health should also inform staff of the expectation that they will meet these time frames. It should require PCB to provide adequate, documented justification whenever PCB fails to meet the time frames.

7. To ensure that district offices address ERIs consistently and to ensure that they investigate ERIs in the most efficient manner, Public Health should review periodically a sample of the priorities that district offices assign to ERIs to ensure compliance with best practices.

8. To protect the residents in long-term health care facilities from potential harm, Public Health should ensure that its district offices have adequate staffing levels for its licensing and certification responsibilities, including staffing levels that allow prompt investigations of complaints. Specifically, Public Health should continue working with CalHR to complete the reclassification of district offices' investigator supervisor and manager positions and then quickly fill the vacant positions at district offices.

12. To ensure that its district offices properly investigate complaints and ERIs, Public Health should make certain that all district offices follow procedures requiring supervisory review and approval of complaint and ERI investigations. If the district offices do not have a sufficient number of supervisors to review investigations they did not conduct, Public Health should arrange to assist the districts until such time that they do have a sufficient number of supervisors.

13. To make certain that its district offices comply with federal requirements regarding corrective action plans, Public Health should establish a process for its headquarters or regional management to inspect district office records periodically to confirm that they are obtaining corrective action plans according to the required time frame and verifying that facilities have performed the corrective actions described in the plans when required.

14. To ensure that it has closed complaints and ERIs appropriately, Public Health should take steps by April 2015 to verify that complaints that its field operations branch closed administratively were closed appropriately. For example, it could request the district offices to verify that the closures were appropriate.

18. To improve its enforcement, each year Public Health should evaluate the effectiveness of its enforcement system across all types of health facilities, including those in developmental centers, prepare the required annual report, and, if called for, recommend legislation to improve the enforcement system and enhance the quality of care.

8. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should address staffing issues by preparing and resubmitting to Public Health a recruitment and retention proposal, developing a succession plan, and taking necessary steps to implement its planned reorganization.

9. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should ensure that its information technology data systems have necessary safeguards, contain accurate and complete data, and support its program needs.

10. To ensure it can provide effective oversight of labs as state law requires, Laboratory Services should update and develop its regulations as necessary to ensure consistency with existing state law.

30. To improve the oversight of psychotropic medications prescribed to foster children, Social Services should collaborate with the counties and other relevant stakeholders—including Health Care Services, as necessary—to develop and implement a reasonable oversight structure that ensures the coordination of the State's and counties' various oversight mechanisms as well as the accuracy and completeness of the information in Social Services' data system. This structure should include at least the following items:
- Identification of the specific oversight responsibilities to be performed by the various state and local government agencies.
- An agreement on how county staff such as social workers, probation officers, and public health nurses will use printed Health and Education Passports to obtain foster children's necessary mental health information—including psychotropic medications and psychosocial services—for inclusion in Social Services' data system.
- A plan to ensure that counties have sufficient staff available to enter foster children's mental health information into Social Services' data system and the resources to pay for those staff.
- An agreement on the specific information related to psychotropic medication—including but not limited to the medication name, maximum daily dosage, and court authorization date—and psychosocial services and medication follow-up appointment information that county staff must enter into Social Services' data system for inclusion in foster children's Health and Education Passports.
- Specific directions from Social Services regarding the correct medication start dates and court authorization dates counties should include in its data system and foster children's Health and Education Passports.
- An agreement on the training or guidance Social Services should provide to county staff members working with Social Services' data system to ensure that they know how to completely and accurately update foster children's Health and Education Passports.
- An agreement on how the counties will use information on the new authorization forms that the Judicial Council approved to better oversee the prescription of psychotropic medications to foster children.
- An agreement regarding how counties will implement, use, or disseminate the educational and informational materials the Quality Improvement Project has produced, including the "California Guidelines for the Use of Psychotropic Medication with Children and Youth in Foster Care", "Questions to Ask About Medications", and the "Foster Youth Mental Health Bill of Rights".
- An agreement on the specific measures and the best available sources of data the State and counties will use to oversee foster children prescribed psychotropic medications, including psychosocial services and medication follow-up appointments.
- An agreement on how the State and counties will oversee psychotropic medications prescribed to foster children by fee-for-service providers who are not affiliated with county Medi-Cal mental health plans.
- An agreement on the extent of information related to psychotropic medications prescribed to foster children that counties will include in the self-assessments, system improvement plans, and annual progress reports they develop as part of Social Services' California Child and Family Services Reviews.
- An agreement on the extent of the information related to psychotropic medications prescribed to foster children that counties will include in their responses to Health Care Services' reviews, including its county Medi-Cal mental health plan compliance reviews and external quality reviews.

40. To promote continued improvement in the CWS system, Social Services should encourage each county CWS agency to designate personnel to update regularly their policies and procedures, to include a detailed description of the need for ongoing supervisory reviews of key aspects of their respective service processes and incorporate that description into their policies and procedures, and to designate personnel to perform regular quality assurance reviews.

41. To promote continued improvement in the CWS system, Social Services should ask each county CWS agency to report to Social Services on the status of these efforts within 60 days, six months, and one year from the publication of this audit report.

1. To ensure that all address matches of registered sex offenders who potentially reside or work at a licensed facility or foster home are reviewed, Social Services should improve its current mechanism to track and monitor the outcome of each address match it identifies. This tracking mechanism should allow Social Services to actively reconcile the number of address matches identified through its address comparison process with the number of completed reviews to ensure that it appropriately reviewed each match. Further, this mechanism should allow Social Services to actively monitor and report on any overdue investigations.

3. To ensure that counties' use of foster family agency placements is justified, Social Services should take action to implement the recommendation we previously made in our 2011 audit. Specifically, Social Services should continue working to revise its rates paid to foster family agencies and to ensure that it has reasonable support to justify each rate component, especially the administrative fee it currently pays these agencies.

4. To ensure that counties' use of foster family agency placements is justified, Social Services should take action to implement the recommendation we previously made in our 2011 audit. Specifically, Social Services should require counties to give licensed foster homes a higher priority than foster family agencies for children that do not have identified treatment needs.

5. To ensure that counties' use of foster family agency placements is justified, Social Services should take action to implement the recommendation we previously made in our 2011 audit. Specifically, Social Services should require counties to prepare a detailed justification for any child placed with a foster family agency.

4. To ensure that all counties consistently gauge the cost-effectiveness of their early fraud detection activities and ongoing investigation efforts for the CalWORKs and CalFresh programs, Social Services should develop a formula to regularly perform a cost-effectiveness analysis using information that the counties currently submit. Specifically, this formula should measure the savings that a county achieves for each dollar spent on antifraud efforts.

5. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should, using the results from the recommended cost-effectiveness analysis, determine why some counties' efforts to combat welfare fraud are more cost-effective than others.

6. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should seek to replicate the most cost-effective practices among all counties. Social Services should work with its legal counsel to determine whether to withhold information about these practices from public disclosure.

7. Social Services should track counties' prosecution thresholds for welfare fraud cases and determine whether they affect counties' decisions to investigate potential fraud, with a focus on determining best practices and cost-effective thresholds. If Social Services' analysis determines that varying prosecution thresholds do affect counties' decisions, it should then work with counties to implement the consistent use of these cost-effective prosecution thresholds.

8. Social Services should continue its efforts to ensure that counties follow state regulations regarding the use of the administrative disqualification hearings process until all counties have adopted the process.

9. To make certain that counties receive the greatest benefit from the resources they spend on antifraud efforts related to CalWORKs and CalFresh cases, Social Services should address and promptly act on the four remaining recommendations that its steering committee provided in 2008.

12. To make counties' review of match lists more efficient, Social Services should revive its efforts to work with the state and federal agencies that prepare the match lists to address the counties' concerns about match list formats, content, and criteria.

13. To ensure the accuracy of the overpayments that counties collect and report for the CalFresh program, Social Services should create a process to verify on a rotational basis the counties' overpayment collection reports.

16. To ensure the accuracy and consistency of the information on welfare fraud activities that counties report and that Social Services subsequently reports to the federal government, the Legislature, and internal users, Social Services should incorporate the upcoming federal changes to the revision of its instructions for completing the counties' investigation activity reports. In the interim, Social Services should issue clarifications for the most common errors Social Services observes counties make in reporting their investigation activities.

4. To ensure financial sustainability, Covered California should conduct regular reviews of enrollment, costs, and revenue and make prompt adjustments to its financial sustainability plan as necessary.

2. To ensure timelier fee assessments, Developmental Services should hold regional centers accountable for providing the monthly placement reports and copies of information letters required by state regulations. To encourage compliance, Developmental Services should specify in its regional center contracts that noncompliant regional centers will pay financial penalties equal to the amount of revenue lost because of their inaction.

4. To ensure that regional centers are aware of the benefits, including cost savings to the State that can be realized by using FMS vendors, DDS should formally communicate to regional centers regarding the model.

5. To ensure that in-home respite vendors are providing quality services and that vendors are adhering to state requirements, DDS should issue regulations requiring regional centers to conduct periodic and ongoing reviews of vendors' programs, employees, and consumer records.

6. To ensure that in-home respite vendors comply with vendor requirements on an ongoing basis, DDS should require the regional centers to develop a process to conduct biennial reviews of the vendor files the regional centers maintain and document the outcome of the review in the files. DDS should require the regional centers to take appropriate action to ensure that vendors comply, up to and including terminating the vendorization, if necessary.

7. To ensure that it is providing oversight in accordance with state law and federal requirements, DDS should ensure that it performs audits of each regional center every two years as required. In conducting these audits, DDS should consistently include a review of in-home respite services.

11. To minimize the need for overtime, the department should reassess its minimum staffing requirements, hire a sufficient number of employees to cover these requirements, and examine its employee scheduling processes.

3. To ensure that Managed Health Care reaches accurate conclusions during its quarterly assessments of the adequacy of provider networks, Health Care Services should establish by September 2015 a process to verify the accuracy of the provider network data it receives from health plans and forwards to Managed Health Care. For example, Health Care Services could verify, for a sample of physicians claimed as part of the health plans' provider networks, that health plans have current written agreements with the providers.

4. To improve the accuracy of provider directories, by December 2015 Health Care Services should revise its processes for monitoring health plans' provider directories. Specifically, Health Care Services should review how each health plan updates and verifies the accuracy of the directory. In addition, Health Care Services should identify best practices and require the plans to adopt those practices.

6. If Health Care Services finds significant errors in a health plan's provider directory, it should work with that health plan to identify reasons for the inaccuracies and require the health plan to develop processes to eliminate the inaccuracies.

7. To ensure that it can handle adequately the volume of calls from Medi-Cal beneficiaries, Health Care Services should implement an effective plan to upgrade or replace its telephone system and database to make certain that its ombudsman office can handle the volume of calls and maintain complete data to make informed management decisions.

8. To further ensure that it can handle adequately the volume of calls from Medi-Cal beneficiaries, after upgrading or replacing its systems, if Health Care Services believes that it does not have adequate staffing to address workload, it should justify its need and request additional staff.

9. To make certain that Health Care Services complies with state law requiring it to conduct annual medical audits, it should finish developing and begin adhering to its schedule for auditing all health plans in fiscal year 2015-16.

10. To ensure that Health Care Services complies with state law, it should increase its oversight of Managed Health Care to ensure that it completes the quarterly assessments required under the agreements.

1. To ensure that it provides claiming units with reasonable opportunities to address concerns with its decisions or actions, Health Care Services should, within three months, begin preparing regulations to establish and implement a formal appeals process that allows claiming units to directly appeal Health Care Services' decisions.

2. To ensure that it provides claiming units with reasonable opportunities to address concerns with its decisions or actions, Health Care Services should, within three months, inform all stakeholders, including claiming units, of the existence of this appeals process.

3. Until the Legislature implements our recommendation in Chapter 2, Health Care Services should immediately resolve weaknesses in its oversight of local educational consortia and local governmental agencies to ensure that these entities sufficiently meet their responsibilities under the administrative activities program and meet the terms of their contracts with Health Care Services. Health Care Services should update its site review and desk review procedures to include the following steps:
* A risk-based approach to selecting entities for review.
* Verification that local educational consortia and local governmental agencies are adequately meeting the oversight and administrative responsibilities described in their contracts with Health Care Services.
* Verification that contracts between local educational consortia or local governmental agencies and their claiming units do not include provisions that could result in disallowed costs, such as allowing Health Care Services' participation fee to be included in the claim calculations.
* Examination of local educational consortia and local governmental agencies' records to ensure that:
-- Costs they claim for federal reimbursement are necessary and reasonable.
-- The entities are not inappropriately earning a profit based on the fees they collect from claiming units.
-- The coding performed by local educational consortia that charge claiming units a percentage of their federal reimbursement is reasonably accurate.

4. Until the Legislature implements our recommendation in Chapter 2, Health Care Services should immediately resolve weaknesses in its oversight of local educational consortia and local governmental agencies to ensure that these entities sufficiently meet their responsibilities under the administrative activities program and meet the terms of their contracts with Health Care Services. Health Care Services should complete the oversight reviews for at least three high-risk local educational consortia or local governmental agencies by December 31, 2015, and post the results to its website.

5. Until the Legislature implements our recommendation in Chapter 2, Health Care Services should immediately resolve weaknesses in its oversight of local educational consortia and local governmental agencies to ensure that these entities sufficiently meet their responsibilities under the administrative activities program and meet the terms of their contracts with Health Care Services. Health Care Services should complete the oversight reviews for any remaining high-risk local educational consortia or local governmental agencies by June 30, 2016, and post the results to its website.

8. To minimize the risk that claiming units could include unallowable costs when calculating their reimbursement claims, Health Care Services should remind all local educational consortia and local governmental agencies that contracts with their claiming units should prohibit claiming units from seeking federal reimbursement of Health Care Services' participation fee.

10. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Heath Care Services should implement a single statewide quarterly random moment time survey and develop and implement a plan to take over responsibility for conducting quarterly time surveys and performing related activities as soon as reasonably possible.

11. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Heath Care Services should implement a single statewide quarterly random moment time survey and develop and issue a request for proposals to identify a responsible vendor to assist in implementing a statewide quarterly random moment time survey.

12. To streamline the organizational structure of its administrative activities program and to improve the program's cost-effectiveness, Heath Care Services should implement a single statewide quarterly random moment time survey and draft revisions to regulations as appropriate and to applicable documents, including the manual, oversight strategies and plans, and policy and procedure letters.

13. To the extent that local educational consortia and local governmental agencies are no longer involved in the administrative activities program, Health Care Services should develop and issue a standard contract for claiming units to sign to participate in the program.

17. To better maximize federal reimbursements for the administrative activities program, Health Care Services should, within six months, develop and implement a method to oversee and track the outreach efforts that local educational consortia and local governmental agencies use for ensuring that nonparticipating claiming units understand the benefits and consider participating in the administrative activities program.

19. To better maximize federal reimbursements for the administrative activities program, Health Care Services should, within six months, determine the extent to which claiming units can claim the unreimbursed difference between the 50 percent and 75 percent reimbursement rates for translation activities for past years and inform claiming units of the findings.

21. To provide the public with the ability to participate fully in developing the rules governing the administrative activities program, Health Care Services should, in accordance with California's Administrative Procedure Act (APA), immediately develop and adopt the regulations cited in the four subdivisions of Section 14132.47 of the California Welfare and Institutions Code.

24. To ensure that it provides stakeholders with timely access to information regarding the billing option program, Health Care Services should issue all future annual reports in a timely manner.

3. To ensure that the providers receive reimbursement for only valid services, Health Care Services should immediately direct its investigations division to determine whether it authorized any improper payments to program providers for deceased beneficiaries outside of our audit period. It should also determine whether it authorized such payments through its other Medi-Cal programs. Health Care Services should initiate efforts to recover such payments as appropriate.

18. To prevent the certification of ineligible providers, Health Care Services should immediately ensure that its enrollment division conducts LEIE and EPLS database searches of program providers at least monthly.

20. To prevent the certification of ineligible providers, Health Care Services should immediately establish a mechanism to identify the number of program sites the provider applicants' medical directors work at, and ensure that the physician ratio does not exceed 1-to-3 in accordance with state law and the certification standards.

21. To prevent the certification of ineligible providers, Health Care Services should immediately identify and perform an immediate recertification of providers that signed the Compliance Agreement to ensure that these providers are currently meeting all program requirements.

24. To prevent the certification of ineligible providers, Health Care Services should immediately develop a schedule for recertifying all program providers every five years.

25. To prevent the certification of ineligible providers, Health Care Services should immediately continue its implementation of an automated provider enrollment system.

26. To prevent the certification of ineligible providers, Health Care Services should immediately complete its program recertification on or before March 24, 2016, as federal regulations require.

27. To prevent the certification of ineligible providers, Health Care Services should immediately establish a plan for eliminating its backlog of applications for new sites and services and changes to existing certifications.

4. To ensure that child beneficiaries throughout California can reasonably access dental services under Medi-Cal and to increase child beneficiary utilization and provider participation, Health Care Services should take the following steps for the fee-for-service delivery system by May 2015: immediately take action to resolve any declining trends identified during its monitoring efforts.

5. To help increase the number of providers participating in the program's fee-for-service delivery system, Health Care Services should improve its identification and implementation of changes that minimize or simplify administrative processes for providers. These changes should include revising its processes pertaining to dental procedures that require radiographs or photographs.

7. To ensure that the influx of beneficiaries resulting from recent changes to federal and state law is able to access Medi-Cal's dental services, Health Care Services should immediately take action to resolve any declining trends identified during its monitoring efforts.

19. To ensure that it reports in the CMS-416 an accurate number of child beneficiaries who received specific types of dental services from the centers and clinics, Health Care Services should continue working on a solution to capture the details necessary to identify the specific dental services rendered.

20. To make certain that it meets the requirements of the new state law and that its performance measures are accurate, Health Care Services should establish the provider-to-beneficiary ratio statewide and by county as performance measures designed to evaluate access and availability of dental services and include this measure in its October 2015 report to the Legislature.

23. To ensure that Health Care Services and its fiscal intermediaries reimburse providers only for services rendered to eligible beneficiaries, Health Care Services should obtain Social Security's Death Master File and update monthly its beneficiary eligibility system with death information.

24. To ensure that Health Care Services and its fiscal intermediaries reimburse providers only for services rendered to eligible beneficiaries, Health Care Services should coordinate with the appropriate fiscal intermediaries to recover inappropriate payments made for services purportedly rendered to deceased beneficiaries, if necessary.

33. To increase the State's assurance that foster children do not receive medically inappropriate or unnecessary psychotropic medications, Health Care Services should devise and implement within six months methods to better enforce its prior authorization requirement for the off-label use of psychotropic medications. For example, Health Care Services should revise its claims system to automatically prompt pharmacists to submit treatment authorization requests when filling prescriptions for Medi-Cal beneficiaries under age 18 when the prescribed psychotropic medications have no FDA-approved pediatric uses. Furthermore, as part of its collaboration with Social Services and the counties to develop and implement a reasonable oversight structure, Health Care Services should determine whether information from the Judicial Council's revised court authorization forms would help it better enforce its prior authorization requirements.

4. Caltrans, the Energy Commission, Food and Agriculture, and Health Care Services should put in writing those policies and procedures related to intellectual property that they believe are necessary and appropriate to enable their staff to identify, manage, and protect their intellectual property.

2. To ensure that it monitors counties to the fullest extent as the MHSA specifies and that it implements best practices, Health Care Services should conduct comprehensive on-site reviews of county MHSA programs, including verifying county compliance with MHSA requirements.

3. To ensure that counties have the needed guidance to implement and evaluate their MHSA programs, Health Care Services should coordinate with the Accountability Commission and issue guidance or regulations, as appropriate, for Facilities programs and for other MHSA requirements, such as a prudent reserve.

5. To ensure that counties have the needed guidance to implement and evaluate their MHSA programs, Health Care Services should collaborate with the Accountability Commission to develop and issue guidance or regulations, as appropriate, to counties on how to effectively evaluate and report on the performance of their MHSA programs.

6. To ensure that Health Care Services and other state entities can evaluate MHSA programs and assist the Accountability Commission in its efforts, Health Care Services should collect complete and relevant MHSA data from the counties.

7. To ensure that Health Care Services and other state entities can evaluate MHSA programs and assist the Accountability Commission in its efforts, Health Care Services should resolve all known technical issues with the partnership and client services systems and provide adequate and expert resources to manage the systems going forward.

18. Health Care Services should develop standardized data collection guidelines or regulations, as appropriate, that will address inconsistencies in the data that counties report to the State. In developing these guidelines or regulations, Health Care Services should consult with the Accountability Commission to ensure that data collected reasonably fulfill statewide evaluation purposes.

19. To help ensure county compliance with stakeholder regulations, Health Care Services should provide technical assistance to counties on the MHSA local planning review process and ensure that its guidance to counties is clear and consistent with state regulations.

3. To promote consistency and ensure that it provides sufficient guidance to evaluators, State Hospitals should update its assessment protocol by March 2016 to include more specific instructions on how to conduct evaluations, such as what assessment instruments evaluators may use and what documents they should consider. State Hospitals should also develop a timeline for periodically reviewing and making any necessary updates to the assessment protocol.

4. To comply with state law, State Hospitals should ensure that it follows the Administrative Procedures Act for future changes to its standardized assessment protocol.

1. To ensure that colleges receive consistent and fair treatment and are able to address deficiencies, the chancellor's office should work with the community colleges and request clearer guidance from the commission regarding what actions would allow for the full two-year period in which to remediate concerns and what actions would constitute good cause for extending the time an institution has to address deficiencies beyond two years. In doing so, the chancellor's office should also encourage the commission to specify in its policies those scenarios under which it would exercise the good cause exception so that institutions would have a better understanding of when they might reasonably expect additional time to address deficiencies.

2. To ensure that community colleges and the public are fully informed regarding the accreditation process, the chancellor's office should assist community colleges in communicating their concerns to the commission regarding its transparency and in developing proposals for improving the commission's transparency policies and practices. The chancellor's office should also encourage the commission to publish policies describing the role of its staff in the commission's decision-making processes.

3. To make certain that institutions receive fair treatment in appealing decisions that terminate their accreditation, the chancellor's office should work with the community colleges to advocate that the commission change certain aspects of its appeal process. Specifically, in keeping with the spirit of accreditation, when institutions have taken steps to correct deficiencies that led to the decision to terminate accreditation, the institutions should be allowed to have information on those corrections heard as evidence in their appeal. Further, the commission president's involvement in selecting the appeal panel's counsel should be revisited.

4. To strengthen institutions' understanding of what they must do to comply with standards, and to provide them with the opportunity to address certain issues that could jeopardize their compliance, the chancellor's office, in collaboration with the community colleges, should encourage the commission to develop formal opportunities for institutions to communicate with and receive feedback from the commission on institutional self-studies and other reports before a formal evaluation takes place. In doing so, the chancellor's office should consider the practices of other regional accreditors and identify those that would best meet the needs of California's community colleges.

5. Community colleges, as members of the commission, should communicate their concerns about and ideas for improvement of training on the accreditation process to the commission. To provide assurance to colleges that they may suggest this information freely, the chancellor's office should coordinate communication between the commission and the colleges. Further, in order to build collegial relationships, engage new people in the accreditation process, and extend additional training to those already involved in accreditation, the chancellor's office should encourage the commission to develop an annual conference focused on accreditation and oversight.

7. To allow colleges flexibility in choosing an accreditor, the chancellor's office should identify other accreditors who are able to accredit California community colleges or who would be willing to change their scopes to do so.

8. To allow colleges flexibility in choosing an accreditor, the chancellor's office should assess the potential costs, risks, and feasibility of creating a new independent accreditor.

20. To ensure that its respective institutions comply with the Clery Act, the Community Colleges Chancellor's Office should develop written policies and procedures to provide guidance to its institutions on how to report accurate Clery Act crime statistics and ensure that all required disclosures are included in its respective institutions' annual security reports. The Community Colleges Chancellor's Office should then annually revisit the written policies and procedures to ensure that they are up to date.

4. Within six months of the date the Legislature clarifies its intent regarding California Education Code, Section 89708, the Chancellor's Office should develop and issue final guidance to campuses regarding supplanting, including identifying appropriate oversight mechanisms for ensuring campuses' compliance with this law.

19. To ensure that its respective institutions comply with the Clery Act, the CSU Office of the Chancellor should develop written policies and procedures to provide guidance to its institutions on how to report accurate Clery Act crime statistics and ensure that all required disclosures are included in its respective institutions' annual security reports. The CSU Office of the Chancellor should then annually revisit the written policies and procedures to ensure that they are up to date.

18. To ensure that its respective institutions comply with the Clery Act, UCOP should finalize and implement its draft policy that will provide additional guidance and oversight to its institutions.

23. To address the improper acts we identified, the university should collect $1,802 from the official for the wasteful expenses he claimed for lodging and meals during his trip to England, the expenses he incurred within the vicinity of his headquarters, and the business meal expenses.

49. The Office of the President should clarify in the UC policies that a complainant must have and be informed about the right to end the early resolution process at any time and request that his or her complaint be handled under the university's formal process.

51. The Office of the President should clarify in the UC policies that if university officials approve an extension to an investigative timeline, the extension should be restricted to a single extension of no more than 30 days, except in limited circumstances that are beyond the university's control.

1. To meet its commitment to California residents, the university should replace its "compare favorably" policy with a new admission standard for nonresident applicants that reflects the intent of the Master Plan. The admission standard should require campuses to admit only nonresidents with admissions credentials that place them in the upper half of the residents it admits.

2. To meet its commitment to California residents, the university should amend its referral process by taking steps to increase the likelihood that referred residents ultimately enroll.

3. To ensure that campuses' interpretations of admission standards do not adversely impact residents, the university should implement a thorough process to annually evaluate the qualifications of students who apply and students who are admitted. These evaluations should highlight instances when campuses admit nonresidents who are less qualified than residents and should include corrective action steps. Moreover, this evaluation should include resident and nonresident undergraduate enrollment in majors at each campus. The university should make the results of this evaluation—including details of the academic qualifications of students who applied and who were admitted—publicly available.

7. To improve its internal operations and promote cost savings related to the nearly $13 billion it spent on employee salaries in fiscal year 2014-15, the university should conduct a systemwide assessment to identify ways to streamline and reduce its employee costs.

8. To ensure the reasonableness of the compensation the university pays its executives, it should include—to the extent possible— all items of compensation when setting or adjusting salaries and benefits, when conducting surveys and studies, and when comparing the compensation packages of its executives to those in similar positions outside the university.

9. To ensure that its process for establishing and revising salaries for its top executives is documented, thorough, and consistently applied, the university should implement the five outstanding recommendations from its 2013 internal review report by August 2016.

10. To improve the transparency and timeliness of its annual compensation report, the university should streamline the process it uses to prepare the report so it can be issued by April of each year.

12. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should immediately require that the campuses fully participate in all projects unless they can provide compelling evidence demonstrating a harmful effect.

13. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should, by June 30, 2016, to the extent possible, implement a process to centrally direct these funds to ensure that campuses use them to support the core academic and research missions of the university.

14. To maximize the savings and new revenue from the Working Smarter initiative and ensure that the university uses them for its academic and research missions, the Office of the President should ensure that it substantiates that projects are actually generating savings and new revenue and that it can demonstrate how the university uses these funds.

15. To ensure that its recruiting efforts benefit residents, the university should prioritize recruiting residents over nonresidents. In particular, the university should focus its recruiting efforts broadly to ensure that it effectively recruits resident underrepresented minorities. For example, the university could establish a limit on the amount of funds it dedicates to nonresident recruiting. Further, it should develop a process to better track its nonresident and resident recruiting expenditures.

16. To determine if the campuses are using funds to further the goals of the University of California system and the Legislature, the Office of the President should begin regularly monitoring and analyzing how campuses are using both state funds and nonresident supplemental tuition. If, after the close of the fiscal year, the Office of the President determines that campuses are not using state funds and/or nonresident supplemental tuition in accordance with those goals, the Office of the President should take steps to correct the campuses' spending decisions as soon as possible.

17. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should track spending from state funds for programs that do not relate to educating students.

18. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should reevaluate these programs each year to determine whether they continue to be necessary to fulfill the university's mission.

19. To ensure that it spends state funds prudently for programs that do not directly relate to educating students, the university should explore whether the programs could be supported with alternate revenue sources.

22. To ensure that its rebenching efforts lead to equalized per-student funding among the campuses, the university should adopt a methodology that it can use, at least every three to five years, to update its weighting system to ensure the weight factors take into account campuses' actual costs of instruction, using the cost study that we recommend in Chapter 1 and other revenue sources if necessary.

23. To ensure that its rebenching efforts lead to equalized per-student funding among the campuses, the university should exclude from its rebenching calculation all state funding it uses for programs that do not directly relate to educating students. The university should exclude these programs only after it has evaluated them in accordance with the recommendation we made previously.

8. To help ensure that university faculty and staff do not mishandle student reports of incidents, all faculty and staff should receive training annually, consistent with their role, on their obligations in responding to and reporting incidents of sexual harassment and sexual violence.

20. All universities should provide their education on sexual harassment and sexual violence to incoming students as close as possible to when they arrive on campus but no later than the first few weeks of their first semester or quarter. Further, universities should provide periodic refresher educational programs, at least annually, to all students on campus to ensure that they are aware of how to handle and report incidents of sexual harassment and sexual violence.

58. All universities should ensure that the differences between an informal or early resolution process and a formal investigation process are clearly explained to ensure that students know what to expect from each process. Further, they should explain that students whose cases are being handled under an informal or early resolution process have the right to move to a formal process at any time.

4. UC Davis should collect all late fees that its licensees owe.

9. To help ensure that university faculty and staff do not mishandle student reports of incidents, all faculty and staff should receive training annually, consistent with their role, on their obligations in responding to and reporting incidents of sexual harassment and sexual violence.

21. All universities should provide their education on sexual harassment and sexual violence to incoming students as close as possible to when they arrive on campus but no later than the first few weeks of their first semester or quarter. Further, universities should provide periodic refresher educational programs, at least annually, to all students on campus to ensure that they are aware of how to handle and report incidents of sexual harassment and sexual violence.

59. All universities should ensure that the differences between an informal or early resolution process and a formal investigation process are clearly explained to ensure that students know what to expect from each process. Further, they should explain that students whose cases are being handled under an informal or early resolution process have the right to move to a formal process at any time.

2. To demonstrate its willingness to fairly evaluate regional expenditures, Education should allow San Joaquin to reimburse its general fund for the vehicle purchase Education incorrectly disallowed.

14. To address a lack of detailed migrant program service and outcome data, Education should either expand the capabilities of its existing statewide databases or implement additional systems that would allow regions to capture more detailed data about migrant students.

2. To assist the district with establishing priorities, and to ensure that the public is aware of those priorities, the state superintendent should direct his state administrator to develop annual performance objectives and an action plan to address FCMAT's findings and recommendations. Such an action plan should describe for the public why certain findings were prioritized and what steps the state administrator plans to take to improve the district's FCMAT scores.

4. To provide the public an opportunity to fully understand the requirements for and the progress made toward restoring local control to the district's governing board, the state superintendent should direct his state administrator to establish regular advisory board agenda items to answer the public's questions concerning the efforts made toward achieving the exit criteria.

19. To provide stronger leadership with respect to school safety and nondiscrimination laws, Education, with direction from the superintendent of public instruction, should use data from the kids survey and reported suspensions and expulsions to evaluate the levels of discrimination, harassment, intimidation, and bullying students encounter and to determine the effectiveness of its own and the LEAs' efforts, and report the results to the Legislature by August 1, 2014.

21. To provide stronger leadership with respect to school safety and nondiscrimination laws, Education, with direction from the superintendent of public instruction, should within the next six months and annually thereafter, update and replace the resources on its Web site to provide more relevant information on best practices, such as preventing and responding to incidents related to a protected characteristic or that occur through cyberbullying, the U.S. DOE report on state bullying legislation, and best practices in other states, such as the Massachusetts law on LEA staff training requirements.

3. To ensure that all LEAs comply with federal special education requirements, Education should require them to include directly in a student's IEP document reasons for any changes to student placement or services.

8. To enable it to review additional areas of its special education program for quality assurance, Education should collect information about the frequency of the provision of each service contained in all students' IEPs. Education should then use this information to annually review the frequency of mental health services and follow up with SELPAs when it observes a significant reduction in the frequency of services.

9. To ensure that LEAs comply with federal and state requirements, Education should require all LEAs to use the IEP document to communicate the rationale for residential treatment and any potential harmful effects of such placement.

18. Education should analyze and report to the Legislature, by May 30, 2016, on the outcomes for students receiving mental health services statewide, including outcomes across the six performance indicators we identified, in order to demonstrate whether those services are effective. Once it has reported this statewide information, Education should provide each LEA throughout the State a report regarding the outcomes for the students the LEA served.

20. To ensure that the State knows the amount LEAs spend to provide mental health services for student IEPs, before the start of the 2017-18 fiscal year, Education should develop, and require all LEAs to follow, an accounting methodology to track and report expenditures related to special education mental health services.

28. Education should require all LEAs and SELPAs that hold contracts for mental health services to annually obtain and retain copies of contractor personnel lists and the credentials or licenses for contractor personnel who provide mental health services to students in their respective LEA or SELPA.

29. To ensure that the State provides special education and related services to all eligible students, Education should investigate the difference between the estimated number of school aged children statewide who have a severe emotional disturbance and the number receiving mental health services through an IEP and determine the reason for such a discrepancy. Education should then take any steps necessary to assist LEAs in identifying and providing services to children who are severely emotionally disturbed.

23. To reduce the risk that it may not identify an armed prohibited person, Justice should revise its electronic matching process to use all personal identifying numbers available in its databases.

34. To ensure that it fully supports its decision to apply federal prohibition terms to individuals, Justice should review all applicable federal and state laws and continue to seek clarification from the ATF and any other appropriate federal agencies to determine whether California's firearms restoration process meets federal criteria and, if not, why it does not. Justice should issue a report to the Legislature, within one year, detailing the results of its review and, if applicable, communicate why California's restoration process does not meet federal criteria and the impact that it has on prohibited persons who live in California.

4. To report to the Legislature about the effectiveness of its RADS program and to better inform decisions about expanding the number of analyzed sexual assault evidence kits, Justice should amend its agreements with the counties participating in the RADS program to require those counties to report case outcome information, such as arrests and convictions for the sexual assault evidence kits Justice has analyzed under the program. Justice should then report annually to the Legislature about those case outcomes.

10. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, developing best practices based on the requirements stated in the federal regulations, the state guidelines and state law, and advising user agencies on the implementation of those practices. The best practices should include, but not be limited to reviewing criminal intelligence, appropriately disseminating information, performing robust audit practices, establishing plans to recover from disasters, and meeting all of the State's juvenile notification law requirements. Justice should guide the board and the committee to develop these best practices by June 30, 2017.

11. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing user agencies that use CalGang to complete a comprehensive review of all the gangs documented in CalGang to determine if they meet the necessary requirements for inclusion and to purge from CalGang any groups that do not meet the requirements. Justice should guide the board and the committee to ensure that user agencies complete this review in phases, with the final phase to be completed by June 30, 2018.

12. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing all user agencies to complete a comprehensive review of the records in CalGang to determine if the user agencies have adequate support for the criteria associated with all the individuals they have entered as gang members. If the user agencies do not have adequate support, they should immediately purge the criteria—and, if necessary, the individuals—from CalGang. In addition, the user agencies should ensure that all the fields in each CalGang record are accurate. Justice should guide the board and the committee to ensure that user agencies complete this review in phases, with the final phase to be completed by September 30, 2019.

13. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, instructing all user agencies to report to Justice every six months, beginning in January 2017, on their progress toward completing their gang and gang member reviews.

14. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, developing standardized periodic training content for all CalGang users and training instructors. Justice should guide the board and the committee to develop such standardized training content by June 30, 2017.

15. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, establishing a plan to recertify all CalGang users and training instructors on the new training content. Justice should guide the board and the committee to complete the draft plan by June 30, 2017, and the recertification training by June 30, 2018.

16. As the Legislature considers creating a public program for shared gang database oversight and accountability, Justice should guide the board and the committee to identify and address the shortcomings that exist in CalGang's current operations and oversight. The guidance Justice provides to the board and the committee should address, but not be limited to, developing policies and procedures requiring the disabling of user accounts for all individuals who no longer have a need to or right to access CalGang because they have separated from their employment with user agencies or for other reasons. Justice should guide the board and the committee to identify and disable all such accounts by September 30, 2016.

18. To promote transparency and hold the board, the committee, and user agencies accountable for implementing and adhering to criminal intelligence safeguards, Justice should post quarterly reports on its website, beginning June 30, 2017, that summarize how it has guided the board and the committee to implement and adhere to criminal intelligence safeguards; the progress the board, the committee, and the user agencies have made in implementing and adhering to these safeguards; the steps these entities still must take to implement these safeguards; and any barriers to the board's and the committee's success in achieving these goals.

19. To promote transparency and encourage public participation in CalGang's meetings, Justice should post summary results from the committee's audits of CalGang records to its website unless doing so would compromise criminal intelligence information or other information that must be shielded from public release.

4. To ensure that it can accurately estimate any shifts in worker productivity and state revenue, BOE should strengthen its current methodology by analyzing the productivity and revenue collection of its employees and by monitoring those metrics at least semiannually. Additionally, BOE should support its methodology with documentation.

2. Unless the Legislature directs the board to eliminate the compliance fund's excess fund balance within a time frame of more than a year, the board should eliminate the excess fund balance by June 30, 2017 by using it to offset the licensing program's annual funding shortfall. The board should also limit the fund's future balance to no more than two months' worth of licensing program expenditures.

1. To make certain that emergency response agencies receive reimbursements on time, Cal EMA should establish procedures to ensure that paying entities do not delay reimbursements.

11. To make certain that money from the special plate funds pay only for allowable and supportable activities, Cal EMA should maintain documentation to support its charges to the antiterrorism fund. For example, it should ensure that employees submit signed time reports to support the time they spend on antiterrorism-related activities.

1. The authority should amend its regulations to bring them into accord with the 2004 act, thus allowing any eligible hospital to apply for the 2004 act's funds that remained as of June 30, 2014.

1. To ensure that the compensation the AOC provides is reasonable, the Judicial Council should adopt procedures that require a regular and thorough review of the AOC's compensation practices including an analysis of the job duties of each position to ensure that the compensation aligns with the requirements of the position. This review should include comparable executive branch salaries, along with a justification when an AOC position is compensated at a higher level than a comparable executive branch position.

3. To ensure that its compensation structure is reasonable, the AOC should mirror the executive branch's practices for offering leave buyback programs in terms of frequency and amount.

4. To increase its efficiency and decrease its travel expenses, the AOC should require its directors and managers to work in the same locations as the majority of their staff unless business needs clearly require the staff to work in different locations than their managers.

11. To reduce its expenses, the AOC should cease its excessive reimbursements for meals by adopting the executive branch's meal and travel reimbursement policies.

14. To ensure that it spends funds appropriately, the AOC should develop and implement controls to govern how its staff can spend judicial branch funds. These controls should include specific definitions of local assistance and support expenditures, written fiscal policies and procedures as the rules of court require, and a review process.

19. The AOC should conduct a comprehensive survey of the courts on a regular schedule—at least every five years—to ensure that the services it provides align with their responses. The AOC should re-evaluate any services that the courts identify as being of limited value or need.

20. To justify its budget and staffing levels, the AOC should conduct the steps in CalHR's workforce planning model in the appropriate order. It should begin by establishing its mission and creating a strategic plan based on the needs of the courts. It should then determine the services it should provide to achieve the goals of that plan. The AOC should base its future staffing changes on the foundation CalHR's workforce planning model provides. Finally, the AOC should develop and use performance measures to evaluate the effectiveness of this effort.

21. To ensure that it provides services to the trial courts as efficiently as possible, the Judicial Council should explore implementing a fee-for-service model for selected services. These services could include those that are little used or of lesser value to the trial courts, as identified in our survey that we discuss in Chapter 3.

22. To justify the budget and staff level of the AOC, the Judicial Council should implement some or all of the best practices we identified to improve the transparency of AOC spending activities.

2. The Judicial Council should develop a corrective action plan by February 29, 2016 to address the recommendation from our December 2013 audit report related to the controls over its information systems. The corrective action plan should include prioritizing the tasks, resources, primary and alternative funding sources, and milestones for all of the actions required to fully implement its framework of information system controls by June 2016. Further, the Judicial Council should continue to provide guidance and routinely follow up with the superior courts to assist with their effort to make the necessary improvements to their information system controls.

3. To improve the usefulness of the Judicial Council's semiannual reports until a statutory requirement is enacted, the AOC should work with the Judicial Council to pursue a cost-effective method to do the following:

- Include new contracts and the complete history of contracts amended during the reporting period in the semiannual reports, including the date of the original contract; the original contract amount and duration; all subsequent contract amendments; and the date, amount, and duration of each such amendment. The AOC should present this information beginning with the semiannual report covering the July 1, 2014, through December 31, 2014, reporting period.

4. To improve the usefulness of the Judicial Council's semiannual reports until a statutory requirement is enacted, the AOC should work with the Judicial Council to pursue a cost-effective method to do the following:

- Begin tracking additional information in its data systems for inclusion in the semiannual reports. This information should include whether a contract was competitively bid, the justification if it was not competitively bid, and whether the contract was with a Disabled Veteran Business Enterprise. For information technology contracts, the AOC should identify whether the contract was with a small business. The AOC should present this information beginning with the semiannual report covering the July 1, 2014, through December 31, 2014, reporting period.

5. The AOC should implement all of the best practices related to general and business process application controls as outlined in the U.S. Government Accountability Office's Federal Information System Controls Audit Manual no later than December 31, 2014, thereby strengthening and continuously monitoring the effectiveness of the controls over its information systems. In addition, the AOC should immediately begin implementing improvements to its controls over access to its information systems and place these improvements into effect by February 2014. Finally, the AOC should provide guidance and routinely follow up with the superior courts—requiring updates every six months until all identified issues are corrected—to ensure that they make the necessary improvements to their general and business process application controls.

6. The AOC, the Supreme Court, and the first, second, and fourth districts should implement procedures to ensure that they follow a competitive process for their procurements when required.

13. The AOC should revise the judicial contracting manual to require judicial entities to maintain documentation on their determinations of fair and reasonable pricing for purchases under $5,000.

17. The AOC should revise the judicial contracting manual to require that judicial entities maintain documentation for their evaluation and selection process used for competitive procurements. The AOC should also strengthen its procedures to ensure that bid evaluations are conducted properly and calculated correctly.

20. The AOC, HCRC, Supreme Court, and fourth and fifth districts should implement procedures to ensure that required noncompetitive procurement processes, such as preparing justifications and obtaining approval for sole-source procurements, are properly documented. Additionally, the AOC should ensure that it prepares the appropriate documentation when it amends a contract that it has competitively solicited and the amendment includes a change that was not evaluated in the original competitive process.

25. The AOC should implement procedures to ensure that its internal controls over payments are followed and that procurements are approved before ordering and receiving goods and services.

4. To enhance the value of the HAVA spending plan as a transparency and accountability tool for the Legislature, the Office should make the following modifications to its annual HAVA spending plan:

Clearly state the methodology used to report prior HAVA expenditures in the HAVA spending plan. Such a methodology should use the financial information contained in its accounting system.

Reconcile the prior HAVA expenditures with the year-end financial reports the Office provides to the California State Controller's Office.

Present prior HAVA expenditures by activity and by specific appropriation.

5. To ensure the State complies with the NVRA, the Office should take all necessary steps, including seeking any necessary legislative changes, and work with the DMV to modify the driver's license application so that it may simultaneously serve as a form for voter registration.

7. To ensure that the audit and review unit's random audits of closed case files provide an effective oversight mechanism, the State Bar should follow its policy to conduct and record meetings and trainings related to the audit report's recommendations.

1. To reduce the length of time that victims of dishonest lawyers must wait for reimbursement from the Client Security Fund, the State Bar should continue to explore fund transfers, member fee increases, and operating efficiencies that would increase resources available for payouts.

2. To ensure that it maximizes its cost-recovery efforts related to the Client Security Fund, the State Bar should adopt a policy to file for money judgments against disciplined attorneys for all eligible amounts as soon as possible after courts settle the discipline cases.

3. To ensure that it maximizes its cost-recovery efforts related to the Client Security Fund, the State Bar should adopt a policy to evaluate annually the effectiveness of the various collection methods it uses to recover funds from disciplined attorneys.

9. To ensure that it accounts appropriately for information technology project costs and their related funding sources, the State Bar should develop a reasonable method for allocating information technology project costs.

10. To ensure that it accounts appropriately for information technology project costs and their related funding sources, the State Bar should apply its new cost-allocation method to the costs of its Technology Improvement Fund.

17. To ensure that it retains appropriate supervision and control over the State Bar's financial affairs, the board should establish a policy that includes a description of the parameters for the creation of nonprofit organizations limiting such organizations to the purposes consistent with the law and the State Bar's mission.

18. To ensure that it retains appropriate supervision and control over the State Bar's financial affairs, the board should establish a policy that includes a description of the board's oversight role in relation to the State Bar's nonprofit organizations.

19. To ensure that it retains appropriate supervision and control over the State Bar's financial affairs, the board should establish a policy that includes requirements to make sure that the board reviews and approves all documents the State Bar uses in the creation and use of a nonprofit organization, including original and amended bylaws as well as agreements between the State Bar and the organization.

20. To ensure that it retains appropriate supervision and control over the State Bar's financial affairs, the board should establish a policy that includes requirements ensuring that the board reviews, approves, and monitors regularly the budgets and other financial reports of any nonprofit organizations.

21. To ensure that it retains appropriate supervision and control over the State Bar's financial affairs, the board should establish a policy that includes requirements that the State Bar develop policies and procedures to prevent the mingling of its funds and any nonprofit organization's funds.

4. To improve the accuracy of information in the leave accounting system and to ensure that agencies do not improperly credit employees with leave in the future, the state controller should implement additional controls by June 2015 to prevent the leave accounting system from processing the types of inappropriate transactions we identified in our statewide electronic analysis. For example, it could develop cost-effective controls in the leave accounting system that would prevent employees from receiving annual leave and sick leave during the same pay period.

7. To ensure that the Legislature has the information necessary to meet the State's restoration goals and to plan for the State's future financial obligations related to mitigation, the Resources Agency should work with Fish and Wildlife and Water Resources to meet with the Legislature regularly to provide updates on the status of its restoration efforts and the feasibility study to ensure that the Legislature has the information necessary to make funding and other informed decisions.

8. To ensure that the Legislature has the information necessary to meet the State's restoration goals and to plan for the State's future financial obligations related to mitigation, the Resources Agency should work with Fish and Wildlife and Water Resources to develop an estimate of the costs, adjusted for inflation, that the State may incur for fulfilling its financial obligations related to mitigation under the QSA. The Resources Agency should include this information in the feasibility study so the Legislature is fully aware of the estimated costs and timing of the State's future financial obligations.

15. To make certain that money from the special plate funds pay only for allowable and supportable activities, Resources should use all appropriate funding sources to pay for any expenses that benefit multiple programs in proportion to the benefits these programs actually receive. Further, it should ensure that its allocation of such expenses to different funds is equitable and supported.

8. To prevent unauthorized leave buyback transactions, the department should provide training by December 2013 to all department managers and personnel staff who might be involved in leave buyback transactions to ensure that they understand the State's requirements regarding leave buybacks.

3. To avert possible legal challenges, the Authority should ensure that the independent peer review panel adheres to the Bagley-Keene Open Meeting Act or seek a formal opinion from the Office of the Attorney General (attorney general) regarding whether the panel is subject to this act.

1. To ensure that programs supported by special plates receive appropriate amounts of revenues due to them, Motor Vehicles should annually collect all fees for special plates that are no longer on a vehicle but are retained by the plate owner.

3. Motor Vehicles should assess the extent to which it has charged fees for special plates that are not consistent with those prescribed in statutes and take appropriate action.