Report 2012-122 Summary - August 2013

Mental Health Services Act:

The State's Oversight Has Provided Little Assurance of the Act's Effectiveness, and Some Counties Can Improve Measurement of Their Program Performance

HIGHLIGHTS

Our performance review of the Mental Health Services Act (MHSA) highlighted the following:

RESULTS IN BRIEF

Providing effective services and treatment for those who suffer from mental illness or who are at risk of mental illness is an issue of great statewide and national importance. Recent statistics by the U.S. Department of Health indicate that approximately 11 million U.S. adults, or 4.8 percent of the population, had serious mental illnesses in 2009. Critical incidents, such as the school shooting in Sandy Hook, point to the seriousness of these issues. Over time California has attempted to serve its mentally ill population through a variety of services and programs, and in 2004 the voters approved Proposition 63, the Mental Health Services Act (MHSA), to expand on these services and to use innovative methods more likely to identify, mitigate, and treat mental illness. The MHSA stresses that mental illnesses are extremely common, affecting almost every family in California, and that the failure to provide timely treatment can destroy individuals and families. It states, "No individual or family should have to suffer inadequate or insufficient treatment due to language or cultural barriers to care. Untreated mental illness is the leading cause of disability and suicide and imposes high costs on state and local government . . . . State and county governments are forced to pay billions of dollars each year in emergency medical care, long-term nursing home care, unemployment, housing, and law enforcement, including juvenile justice, jail and prison costs."

The MHSA imposes a 1 percent income tax on individuals earning over $1 million for counties1 to use to provide mental health services to individuals severely affected by or at risk of serious mental illness. From fiscal years 2006-07 through 2011-12—the period of our review—almost $7.4 billion was directed to counties for their MHSA programs. The MHSA addresses a broad continuum of service needs, and its five components target different aspects of mental health services, including intensive services in the Community Services and Supports and Prevention and Early Intervention components, and exploring creative approaches to mental health services in the Innovation component. The remaining two MHSA components generally focus on expanding, educating, and training the local public mental health workforce and improving infrastructure; they are not designed to provide direct mental health services.

A focus of the MHSA is accountability, and a significant stated purpose of the MHSA is "to ensure that all funds are expended in the most cost effective manner and services are provided in accordance with recommended best practices subject to local and state oversight to ensure accountability to taxpayers and to the public." Initially, the MHSA assigned the responsibility of overseeing MHSA programs primarily to two state entities—the California Department of Mental Health (Mental Health) and the Mental Health Services Oversight and Accountability Commission (Accountability Commission). However, these state entities have provided little oversight of county implementation of MHSA programs and their effectiveness. We expected that Mental Health and the Accountability Commission would have used a process to monitor, guide, and evaluate county implementation that built on their broad and specific MHSA oversight responsibilities and also incorporated best practices in doing so, but that is not what we found.

The opportunity exists for the state entities currently responsible for oversight to better demonstrate the effectiveness of the MHSA. Effective late June 2012, legislation transferred most of Mental Health's oversight role to the California Department of Health Care Services (Health Care Services). Health Care Services is moving forward with these oversight responsibilities, which include collaborating with the Accountability Commission on its evaluation efforts, but this is still in the early planning stages and it is too soon to tell whether its efforts will address all of our concerns. Nevertheless, because of the minimal oversight Mental Health and the Accountability Commission provided in the past, the State has little current assurance that the funds directed to counties for MHSA programs have been used effectively and appropriately.

We expected that Mental Health would base its monitoring of county MHSA programs on the provisions of the performance contract that the MHSA required Mental Health to enter into with each county. However, in fiscal year 2008-09, Mental Health stopped using the performance contract and began using an agreement that offered little specificity as to the steps a county should take to assure compliance with the MHSA. Functionally, it appears Mental Health treated the agreement as simply a means of providing MHSA funding to counties. Although the assurances within the agreement may have satisfied the minimal requirements set forth in state law, had Mental Health made better use of the agreement as a tool for holding counties accountable for their use of MHSA funds, it would have significantly bolstered the State's oversight role. We also identified shortcomings in certain counties' evaluation and reporting on the effectiveness of their MHSA programs. These shortcomings might have been mitigated had Mental Health chosen to use the performance contracts to improve the quality of county processes for measuring program performance. Going forward, Health Care Services can use its performance contracts with counties to ensure that they specify program goals, identify data that are measurable and meaningfully associated with their goals, and use these data to evaluate the efficacy of their programs. The director indicated that Health Care Services intends to initiate efforts to monitor the adequacy of the counties' administration of MHSA programs. If consistently undertaken, these efforts could address some of the issues we noted about Mental Health's past monitoring.

We also found no evidence that Mental Health conducted systematic and comprehensive monitoring to ensure that counties did, in fact, implement their state-approved MHSA plans. The limited reviews we found failed to provide assurance that all counties consistently followed MHSA requirements and spent taxpayer funds appropriately. Further, Mental Health appears to have relied on county assertions or certifications as its main assurance that a county was complying with certain MHSA requirements. As a starting point, requiring assertions or certifications is useful in informing the county of what is expected and provided Mental Health with some assurance that the county intended to comply with MHSA requirements. However, without performing on-site reviews to ensure that the county had performed as asserted, Mental Health risked that the county may have misused state funds.

In addition, given that one focus of the MHSA is to ensure accountability to taxpayers and the public, we expected that the State would also evaluate the effectiveness of MHSA programs. However, the state entities given that responsibility—Mental Health, the Accountability Commission, and a third entity—have thus far not provided assurance that the MHSA is effective. Mental Health did not conduct a systematic evaluation of the effectiveness of MHSA programs during its tenure. Although it required counties to submit data concerning mental health services and the clients receiving those services, in most cases, Mental Health either failed to consistently obtain certain data or did not ensure that all counties reported the required data. Further, the Accountability Commission did not adopt a framework for evaluation until late March 2013—more than eight years after the passage of the MHSA. The Accountability Commission indicated that its efforts were initially focused on reviewing county plans for proposed MHSA programs because evaluation efforts needed to wait for the programs to mature. Although it seems reasonable that programs need time to mature before they are evaluated, the Accountability Commission began entering into ad hoc contracts related to evaluation in 2009; therefore, it seems to have judged those MHSA programs as mature enough for evaluation at that time.

Further, we expected that Mental Health would have taken steps to ensure that counties received the guidance necessary to effectively evaluate and report on the performance of their MHSA programs. However, Mental Health did not provide explicit direction to the counties on how to evaluate their programs effectively, including directions for setting reasonable goals, establishing specific objectives, and gathering the data necessary to meaningfully measure program performance. When the responsible state entities do not provide guidance to counties for effective program evaluation, the public cannot be sure that MHSA programs are achieving their intended purposes.

Thus, it is not surprising that our review of four county departments—Los Angeles County Department of Mental Health (Los Angeles), County of Sacramento Department of Health and Human Services (Sacramento), County of San Bernardino Department of Behavioral Health Administration (San Bernardino), and Santa Clara County Mental Health Department (Santa Clara)—found that these counties used differing and inconsistent approaches to assess and report on their MHSA programs. For example, some counties could not effectively demonstrate through their processes that their MHSA programs are achieving the stated intent. Although the four reviewed counties generally included program goals in their MHSA plans, not all had communicated those goals to program providers, thereby not articulating expectations that providers demonstrate efforts to achieve those goals. Counties were also inconsistent in collecting data related to program goals and how completely they analyzed and reported on those data to determine if counties were achieving stated program goals.

Moreover, we found that the four counties rarely developed specific objectives to assess the effectiveness of program services. Setting specific goals and objectives and demonstrating that programs are achieving them seems particularly relevant for the Innovation component. Media reports have reflected skepticism about counties' Innovation programs, some of which include acupuncture and yoga. The media's perception of Innovation programs is likely because they may include novel or creative approaches to a mental health practice that may actually be very beneficial, but because the link between the program and the mental health benefit is not clear, these programs are sometimes questioned. Assessing and reporting on program effectiveness is therefore critical to ensure that only effective programs are continued and that the taxpayers and the public are assured that MHSA funds are put to the best use.

Finally, the MHSA requires counties to articulate plans for addressing the mental health needs of their communities, to include stakeholders in the community planning process, and to update the plans annually. The four counties reviewed complied with state regulations that specific groups of stakeholders and community representatives be included throughout the planning process and with community planning regulations that require staffing and training practices related to developing those plans. However, counties did not always document in their MHSA plans and annual updates how they had circulated their draft plans to the community as required. In addition, Mental Health's guidance to counties on plan content has been inconsistent and this may have contributed to the issues we found with county documentation. Nevertheless, failure to properly document these important steps means counties cannot point to their plans to assure their stakeholders and the broader public that they have considered feedback on their plans and developed programs that address the communities' needs.

RECOMMENDATIONS

HEALTH CARE SERVICES

To ensure that it monitors counties to the fullest extent, including conducting the monitoring MHSA specifies as well as implementing best practices, Health Care Services should do the following:

To improve the quality of county processes for measuring program performance, Health Care Services should use its performance contracts with counties to ensure that the counties do the following:

To ensure that counties have the needed guidance to implement MHSA programs, Health Care Services should collaborate with the Accountability Commission and develop and issue guidance or regulations, as appropriate, to counties on how to effectively evaluate and report on MHSA program performance.

To ensure that Health Care Services and other responsible state entities can evaluate MHSA programs and assist the Accountability Commission in its evaluation efforts, Health Care Services should collect complete and relevant MHSA data from the counties.

To help ensure county compliance with stakeholder regulations, Health Care Services should provide technical assistance to counties on the MHSA local planning process and ensure that its guidance to counties is clear and consistent with state regulations.

ACCOUNTABILITY COMMISSION

In order to fulfill its responsibilities to evaluate MHSA programs, the Accountability Commission should undertake the evaluations specified in its recently adopted framework for evaluation.

SACRAMENTO, SAN BERNARDINO, AND SANTA CLARA

Each county should review its existing MHSA contracts and by December 31, 2013, or as soon as is feasible, amend them as necessary to include plan goals.

AGENCY COMMENTS

The three state entities and three counties to which we made recommendations—Health Care Services, the Accountability Commission, the California Mental Health Planning Council, and the counties of Sacramento, San Bernardino, and Santa Clara—agreed with our recommendations and generally agreed with the report's conclusions. We did not make any recommendations to Los Angeles.

1 County indicates a county mental health department, two or more county mental health departments acting jointly, and/or city-operated programs receiving funds per California Welfare and Institutions Code, Section 5701.5.