Report 2012-122 Recommendation 5 Responses

Report 2012-122: Mental Health Services Act: The State's Oversight Has Provided Little Assurance of the Act's Effectiveness, and Some Counties Can Improve Measurement of Their Program Performance (Release Date: August 2013)

Recommendation #5 To: Health Care Services, Department of

To ensure that counties have the needed guidance to implement and evaluate their MHSA programs, Health Care Services should collaborate with the Accountability Commission to develop and issue guidance or regulations, as appropriate, to counties on how to effectively evaluate and report on the performance of their MHSA programs.

Annual Follow-Up Agency Response From August 2017

DHCS has contracted with a vendor to perform data clean up on the data included in the DHCS Client Services Information System and MHSA Data Collection and Reporting System. The purpose of this project is to provide DHCS, the MHSOAC, and the counties with accurate and complete data that may be used to track performance outcomes for Community Services and Supports, Full Service Partnership clients. The project will be completed over the next 12-18 months, with DHCS maintaining data quality thereafter. Counties have already received their comprehensive data error reports, and technical assistance calls to support the interpretation and use of these reports are currently underway. After the data clean up project is complete, DHCS will work with the MHSOAC to determine the kind of guidance that needs to be produced to assist counties with effectively evaluating and reporting on the performance of their MHSA-funded programs.

  • Estimated Completion Date: Next 12-18 months

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented


Annual Follow-Up Agency Response From September 2016

As a first step toward evaluating and reporting on MHSA-funded programs, DHCS determined that it is necessary to work with counties to improve the quality of the Full Service Partnership data reported to the DHCS MHSA Data Collection and Reporting System. DHCS sought, and was provided with, a re-appropriation of MHSA funds to strengthen data quality, which will be the focus of our efforts in FY 2016-17. Thereafter, DHCS will work with the MHSOAC to determine the kind of guidance that needs to be produced to assist counties with effectively evaluating and reporting on the performance of their MHSA-funded programs

  • Estimated Completion Date: Ongoing

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented


Annual Follow-Up Agency Response From October 2015

DHCS is planning to start discussions with the MHSOAC to determine the kind of guidance that needs to be produced to assist counties with effectively evaluating and reporting on the performance of their MHSA-funded programs.

  • Estimated Completion Date: December 2016

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented


1-Year Agency Response

DHCS continues to support the Mental Health Services Oversight and Accountability Commission's (MHSOAC) Evaluation Master Plan through ongoing data reporting system improvements, county assistance in data reporting, data monitoring and reporting for compliance, and ongoing data system planning and coordination meetings.

As part of the review and development of MHSA regulations, DHCS continues to consult with the Accountability Commission to determine any regulatory changes needed to ensure data are collected consistently and in support of the Evaluation Master Plan.

If needed, draft regulations will be developed in consultation with the Accountability Commission and made available for public comment in the winter of 2015, with final regulations adopted during the fall of 2016.

DHCS also continues the development of the Performance Outcomes System Plan for Medi-Cal Specialty Mental Health Services for Children and Youth (POS). In September of 2013, DHCS staff began reviewing the status of county data submission to the Client and Services Information (CSI) System. As of June 30, 2014, 79% of counties had submitted CSI data within the 60-day submission window. Data review and cleanup, as well as timely submission of data, is critical to the quality of data for the POS and other mental health services evaluation efforts. Beginning in October 2014, DHCS will provide quality improvement reports to the counties in order to prepare for the initial outcomes reports for specialty mental health, which are scheduled to be released in December 2014.

  • Estimated Completion Date: Fall 2016
  • Response Date: August 2014

California State Auditor's Assessment of 1-Year Status: Partially Implemented


6-Month Agency Response

DHCS continues to support the Accountability Commissions Evaluation Master Plan through ongoing data reporting system improvements, county assistance in data reporting, and data monitoring and reporting compliance.

DHCS continues its ongoing effort to identify outcomes/measures for children and youth as part of the Performance Outcomes System Plan for Medi-Cal Specialty Mental Health Services for Children and Youth (POS). The POS system plan was submitted to the Legislature on November 1, 2013 and the system implementation plan was submitted on January 1, 2014. Since some of the evaluation efforts in the POS and the Evaluation Master Plan are similar, DHCS and the Accountability Commission continue to collaborate to develop and issue guidance to counties on how to effectively evaluate and report on program performance.

DHCS continues to work with counties on their submission of mental health data. As part of the MHSA training and technical assistance contract, the contractor will work with counties to develop evaluation reports on several evidence-based childrens mental health services. The contractor will provide counties with technical assistance during the evaluation, and afterwards as part of quality improvement. During contract development, DHCS will continue to review deliverables related to county data reporting and performance outcomes evaluation with the Accountability Commission.

  • Estimated Completion Date: Spring 2014
  • Response Date: February 2014

California State Auditor's Assessment of 6-Month Status: Partially Implemented


60-Day Agency Response

DHCS continues to support and further the activities of the Accountability Commission's Evaluation Master Plan through ongoing data reporting system improvements, county assistance in data reporting, and data monitoring and reporting compliance efforts.

DHCS and the Accountability Commission continue to collaborate on various data workgroup efforts to help in creating uniform, standard data reporting and outcome evaluations.

DHCS continues to work with counties on their submission of mental health data. As part of the MHSA training and technical assistance contract, DHCS will work with the contractor and the Accountability Commission to develop training for counties on data reporting and performance outcomes evaluation in support of local and statewide performance outcome objectives and standards.

DHCS will meet with the Accountability Commission beginning in November 2013 to plan for the county guidance and training needed to effectively evaluate and report on the performance of MHSA funded programs.

  • Estimated Completion Date: Fall 2014
  • Response Date: October 2013

California State Auditor's Assessment of 60-Day Status: Pending


All Recommendations in 2012-122

Agency responses received are posted verbatim.


Report type

Report type
















© 2013, California State Auditor | Privacy Policy | Conditions of Use | Download Adobe PDF Reader