Report 2010-112 All Recommendation Responses

Report 2010-112: Employment Development Department: Its Unemployment Program Has Struggled to Effectively Serve California's Unemployed in the Face of Significant Workload and Fiscal Challenges (Release Date: March 2011)

Recommendation #1 To: Employment Development Department

To further enhance its corrective action planning process as a means of improving the unemployment program, the department should identify corrective actions that specifically address the timeliness measures it is trying to meet.

1-Year Agency Response

The department's 2012 State Quality Service Plan (quality plan), which serves as the principal vehicle for planning, recording, and managing its unemployment program's efforts to strive for excellence in service, includes corrective actions to address federal timeliness measures for promptly issuing initial unemployment payments and making nonmonetary determinations of claimant's eligibility for benefits. (2013-406, p. 127)

Response Date: March 2012

California State Auditor's Assessment of 1-Year Status: Fully Implemented

Recommendation #2 To: Employment Development Department

To further enhance its corrective action planning process as a means of improving the unemployment program, the department should develop milestones that are specific and are tied to corrective actions to allow for monitoring the incremental progress of its corrective actions, similar to the milestones it established for some of the activities in its federal fiscal year 2011 corrective action plans.

1-Year Agency Response

The department's quality plan for 2012 now has corrective actions with related milestones. (2013-406, p. 128)

Response Date: March 2012

California State Auditor's Assessment of 1-Year Status: Fully Implemented

Recommendation #3 To: Employment Development Department

To further enhance its corrective action planning process as a means of improving the unemployment program, the department should establish several key performance targets or benchmarks that are tied to each specific corrective action, to effectively gauge the impact of the actions on its goal of achieving the acceptable levels related to the timeliness measures.

Annual Follow-Up Agency Response From September 2015

In the October 2014 annual status report, EDD communicated that a Business Process Innovation Team developed measurable milestones for each required Corrective Action Plan (CAP) included in the California's 2015 State Quality Service Plan (SQSP). This plan included the Unemployment Insurance (UI) portion of the SQSP narrative and CAPs for First Payment Promptness and Non-monetary Determination Timeliness. A copy of the 2015 SQSP will be provided upon request. Subsequent to our last annual report, we received notification that the United States Department of Labor approved the 2015 SQSP on September 30, 2014 (see Attachment).

To demonstrate our commitment to develop measurable milestones, an Integrity Team, with representation from the UI Branch, Tax Branch, and Policy, Accountability and Compliance Branch, developed additional measurable milestones for each required CAP in California's 2016 SQSP. The multi-branch team allows for insight from staff most knowledgeable in the areas of operation related to different federal performance measures.

The team meets throughout the year to brainstorm additional actions to develop into measurable milestones for each specific CAP. Maintaining performance measures for each specific milestone allows for the tracking and analysis of improvements in performance before, during, and after the implemented action. Through this process, EDD is able to effectively gauge the impact of each milestone related to the timeliness performance measures.

Completion Date: September 2014

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

EDD provided us with documentation that includes performance indicators and targets it has identified to measure the outcomes of its actions as they relate to federal timeliness measures. These documents also include information on how well EDD is performing related to these targets for the solutions it indicated that it has implemented.

Annual Follow-Up Agency Response From October 2014

This recommendation is fully implemented with the submission of California's State Quality Service Plan (SQSP) for Federal Fiscal Year 2015, which includes action plans to improve federal performance measures.

A group of EDD subject matter experts, known as the Business Process Innovation Team (formerly the Business Process Assessment Team), has been working with a third-party consultant to conduct reviews of Unemployment Insurance (UI) functions and business processes to identify areas for improvement and streamlining. To date, the team has reviewed the claim filing and determinations processes to identify bottlenecks that cause delays in getting the first UI payment to eligible claimants. After a review and analysis of a sample of claims, the team made specific data-driven recommendations to improve federal performance measures for first payment promptness and determination timeliness. These recommendations were incorporated in the SQSP document recently submitted to the United States Department of Labor. The Business Process Innovation Team will track and measure the outcomes of those solutions and the impact on the performance measures.

Additionally, as part of EDD's ongoing continuous improvement efforts, EDD will utilize the new reporting data available in the new UI benefit payment system to establish performance targets and to assist in effectively gauging the impact of the Corrective Action Plans in the SQSP on our performance goals.

Completion Date: October 2014

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

The State Quality Service Plans the Employment Development Department (EDD) provided us did not include specific performance benchmarks or targets. We followed up with EDD, and it indicated that it does not plan to include this level of detail in its State Quality Service Plans. EDD provided us with a document that describes its Business Process Innovation methodology for improving its timeliness measures, which includes descriptions of performance indicators that it will use to measure the outcomes of its actions. Although we believe that this is a promising step, EDD has indicated in its document that it is still in the process of obtaining and analyzing data. Moreover, this document still does not include specific quantitative benchmarks or targets tied to the performance indicators it identified. Thus, it appears that this effort is still in progress and the recommendation is not fully implemented.

Auditee did not substantiate its claim of full implementation
Auditee did not address all aspects of the recommendation

Annual Follow-Up Agency Response From October 2013

This recommendation has been partially implemented due to chronic federal underfunding in the Unemployment Insurance (UI) program, which has led to reduced staff, combined with the need to implement the UI Modernization Continued Claims Redesign project as a high priority. The EDD made every effort to develop key performance targets that tied to specific corrective actions when drafting the State Quality Service Plan for 2014. The EDD established key performance targets for those areas that lend themselves to measurement and included methodology to measure the success of those particular strategies. However, in September 2013 the EDD launched a new internal payment system that significantly changes the way EDD manages and processes the work and the new system provides completely different reporting mechanisms. New business rules were established to enable EDD to measure the age of the work, the type of the work and whether or not the work items could affect payment timeliness. As with any new system, it will take time for staff to adjust to the new way of processing work and a new baseline needs to be set before realistic performance targets can be set. To further aid it in this endeavor, the EDD plans to conduct business process assessment activities with the aid of a third party to help it rethink and redesign UI business processes to achieve significant improvements in performance that will ultimately help EDD improve its federal performance measures.

Dependent on the extent of its federal underfunding, EDD anticipates completion by the end of 2014.

Estimated Completion Date: End of 2014

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2012

This recommendation has not been fully implemented; the EDD anticipates that this status will be confirmed by the BSA follow-up review report expected to be issued in November 2012. In its 2013 SQSP, the EDD began establishing key performance targets that tied to specific corrective actions. The EDD will continue with this practice when developing its 2014 SQSP by establishing additional key performance targets for the improvement strategies that lend themselves to measurement and include the methodology that will be used to measure the success of those particular strategies. However, while the EDD thinks that not all of the improvement strategies identified in the 2014 SQSP will be discretely measurable due to the nature of the strategies, the EDD will make every effort to tie key performance targets to strategies when possible. For example, marketing strategies to educate claimants about more efficient certification processes are activities that are sometimes difficult to measure outcomes but they are a necessary activity that should ultimately result in performance improvement. The EDD's 2014 SQSP will be submitted to the U.S. Department of Labor by October 2013.

Estimated Completion Date: October 2013

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Recommendation #4 To: Employment Development Department

As part of an overall strategy to limit the number of calls it receives while still providing timely and effective customer service, the department should use existing data and additional data from the new phone system to gain a better understanding of why people request to speak to an agent. Using this information, the department should further develop strategies and measurable goals related to achieving a reduction in call volumes. For example, to ensure that virtually all calls are able to gain access to the voice response portion of its new phone system, the department should monitor the volume of blocked call attempts and work with its phone system vendor if necessary to increase the system's capacity.

Annual Follow-Up Agency Response From August 2017

EDD provided examples of various types of analyses that it performed, such as tracking inbound calls and blocked calls and comparing the number of claimants using UI online for payment information and the number of callers requesting to speak to an agent.

Completion Date: August 2017

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Fully implemented based on documentation cited in agency response.

Annual Follow-Up Agency Response From September 2016

As part of an overall strategy to limit the number of calls it receives, while still providing timely and effective customer service, EDD implemented a number of action plans related to reducing call volumes and ensuring that virtually all claimants are able to gain access to Unemployment Insurance (UI) services either through the voice response portion of the new phone system or through online services. Some of these action plans, as outlined in previous responses, have included:

1. Incorporating the Alternate Base Period in the Interactive Voice Response system to increase access to information through self-service options.

2. Leveraging the power of UI OnlineSM to provide an alternative communications platform to the public.

3. Implementing the Scheduled Call-Back System in order to convert blocked calls into resolved calls.

In order to utilize these action plans to their fullest extent, EDD has used the data from the new phone system to gain a better understanding of why people request to speak to an agent. Most recently in this regard, EDD conducted the UI Self-Service Options Survey before and after the UI Modernization Continued Claims Redesign Project. The initial survey was completed in September 2013, and a follow-up survey was completed in April 2016. The survey provided valuable information that helped EDD strengthen its efforts to improve its service delivery systems.

As a result of these and other efforts, EDD has been able to put programs in place to offer claimants a wide variety of communication solutions, specifically targeting the majority of caller demands. This has, thereby, limited call volume without decreasing service quality; resulting in a total call volume reduction of 83.2 percent and a blocked-call reduction of 96.3 percent between Fiscal Year 2013-14 and Fiscal Year 2014-15 (see Attachment).

Completion Date: May 2016

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

EDD provided an update of the table we included in our report titled "Unemployment Rate and Call Center Data by State Fiscal Year" through fiscal year 2015-16, which showed a significant reduction in total call attempts and a modest improvement in percentage of calls answered by an agent. Further, EDD provided documentation showing how it evaluates the effectiveness of its phone system, such as conducting surveys, to obtain an understanding of why people request to speak with an agent. In addition, it discussed certain services, such as UI online, that it indicated have reduced the demand to speak to a UI staff member. Although its documentation did not include many specific examples of how it has used the more robust management information available under its new phone system to further develop strategies and measurable goals related to achieving a reduction of call volumes, the actions EDD has reported it has taken through offering self services, such as UI online, and its purported reduction of total call attempts appear to have resolved many of our concerns. However, the percentage of calls answered by an agent seems to continue to be an issue. We continue to encourage EDD to maximize the enhanced management information in its phone system to ensure it is providing timely and effective customer service, such as periodically tracking key information against established benchmarks, similar to how it implemented recommendation #3.

Auditee did not address all aspects of the recommendation

Annual Follow-Up Agency Response From September 2015

This recommendation has been partially implemented. To explore strategies to reduce call volume, EDD is conducting UI Self-Service Options Surveys before and after implementation of the UI OnlineSM self-services phase of the UI Modernization Continued Claims Redesign Project. The first survey's population included people who called the UI claims' main English and Spanish telephone lines (not the automated self-service line) between June 28 and July 2, 2013. The online survey was conducted in both English and Spanish from July 26 through September 10, 2013. Results of the first survey showed that the most common reasons claimants called were to obtain claim status information (36% of respondents), file or reopen a claim (21%), certify for benefits (20%), and get EDD Debit CardSM or benefit payment information (18%).

The roll-out of the UI OnlineSM self-service options to claimants began in April and was complete in July 2015. To allow for a full year of customer experience, EDD will conduct the second survey in early 2016 to identify claimants' reasons for calling, reasons for requesting an agent instead of using the self-service options, and suggestions for improving the self-service options.

Estimated Completion Date: Summer of 2016

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2014

This recommendation has been partially implemented. The EDD is conducting UI Self-Service Options Surveys before and after implementation of the UI Online self-services phase of the UI Modernization Continued Claims Redesign Project to explore strategies to reduce call volume. The first survey's population included people who called the UI claims main English and Spanish telephone lines (not the automated self-service line) between June 28 and July 2, 2013. The online survey was conducted in both English and Spanish from July 26 through September 10, 2013. Results of the first survey showed that the most common reasons claimants called were to obtain claim status information (36% of respondents), file or reopen a claim (21%), certify for benefits (20%), and get EDD Debit CardSM or benefit payment information (18%).

UI Online has been scheduled for release in the spring of 2015. Once UI Online is implemented and all claimants have been phased into the system, EDD will conduct the second survey to identify claimants' reasons for calling, reasons for requesting an agent instead of using the self-service options, and suggestions for improving the self-service options.

The EDD will continue to periodically survey claimants in the future to obtain information about the adequacy of self-services via the telephone and UI Online in order to continue to make customer service improvements.

The EDD anticipates completion by the end of 2015.

Estimated Completion Date: December 2015

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2013

This recommendation has been partially implemented due to chronic federal underfunding in the UI program, which has led to reduced staff, combined with the need to implement the UI Modernization Continued Claims Redesign project as a high priority. The EDD acknowledges the value in obtaining detailed information about why callers request to speak to an agent. To obtain information at the level of specificity that the audit recommended, the EDD is conducting UI Self-Service Options Surveys before and after implementation of the UI Modernization Continued Claims Redesign project to determine claimants' reasons for calling, their reasons for requesting an agent instead of exercising one of the self-service options available to them, and their suggestions for improving the self-service options. By conducting the survey before and after the project implementation, researchers may assess the extent to which the enhancements and additional self-service options impact callers' decisions to call into the system and to request an agent. The EDD will use the information from the survey to explore additional strategies to reduce call volume.

The first survey's population included people who called the UI claims main English and Spanish telephone lines (not to be confused with the automated self-service line) between June 28 and July 2, 2013. The online survey was conducted in both English and Spanish from July 26 through September 10, 2013. Individuals without computer access had the option to complete it over the phone with an EDD researcher. The EDD anticipates completing the data analysis from the first survey and reporting the results by November 2013.

Dependent on the extent of its federal underfunding, EDD anticipates completion by the end of 2014.

Estimated Completion Date: End of 2014

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2012

This recommendation has not been fully implemented; the EDD anticipates that this status will be confirmed by the BSA follow-up review report expected to be issued in November 2012. The EDD has made significant improvements in increasing the number of callers who have accessed UI phone services. Between 2009 and 2011, there was a 94.4 percent decrease in blocked calls and an increase of 77 percent in answered calls. A key factor responsible for these improvements is the implementation of the UI Modernization Call Center Network Platform and Application Upgrade project. Implementation of this project contributed significantly to improving access to EDD's call services when Primary Call Centers received the new call center infrastructure and the capacity was expanded to include Primary Adjudication Centers in February 2011 and the Anaheim Pacific Center in June 2011.

The UI Branch continues to analyze its phone system call volume data along with other non-phone workload items such as on-line inquiries (i.e., Ask EDD), eligibility determinations, appeals, adjustments, etc. to identify opportunities to improve customer service. The UI Branch regularly balances the various workload levels with available staffing levels through daily workload management activities in an attempt to meet customer service demand and federal performance measures.

The UI Branch has focused on high-priority efforts to continue to improve and expand overall access to UI services. Efforts such as the automated Telephone Claim Filing Wrap-Up and Transfer project, the Public Service Program expansion to train more staff statewide, the incorporation of the ABP information into the existing Interactive Voice Response (IVR) system implementation, and the UI Modernization Continued Claims Redesign (CCR) project all have or will have a positive effect on public access to UI program services. The EDD has prioritized and focused resources and funding in these efforts to gain the most on its investment by increasing access to more information via self-service options on the IVR or UI on-line services.

The CCR project will significantly enhance the IVR system by adding new claimant self-service functionality. When the EDD completes the CCR project in March 2013, it will:

• Enhance the EDD TeleCertSM and the EDD WebCertSM by expanding the types of claims that can be processed by these self-service certification channels. These new services will provide a more user friendly method of certifying for UI benefits that will assist in reducing common errors made on the paper Continued Claim form that can cause benefit payment delays. Also, the CCR project will gather more eligibility information upfront from the claimant during the certification process and is anticipated to help improve determination timeliness and quality.

• Reduce the need for callers to speak to an agent or submit an on-line inquiry request by providing callers with additional information through the IVR and more self-service functionality through an on-line claimant account.

Performing an extensive analysis of call volumes and developing comprehensive strategies before the CCR project is completed would not be a productive use of EDD's limited resources for two reasons. First, the IVR code is currently frozen while the CCR project is underway. The EDD's Information Technology Branch places a code freeze or moratorium on programming code changes that can affect or jeopardize the integrity of ongoing information technology projects. The IVR code freeze will prevent code changes to be released into the CCR project and protect the integrity of that project. Therefore, no actions based on the findings of our analysis could be implemented until after the CCR project is implemented in March 2013. Second, the EDD expects the CCR project will have an impact on call volumes and when a caller needs to speak with an agent. Once the CCR project is complete, the EDD will conduct a more comprehensive analysis to determine what changes would be cost beneficial for further improving service access and delivery for EDD customers. The EDD expects to conduct this analysis in 2013.

However, the EDD recognizes the value in obtaining information about why callers request to speak to an agent before the CCR project is implemented. Accordingly, prior to implementing the CCR project, the EDD will begin surveying claimants who have phoned and requested to speak to an agent to obtain more specific reasons for the calls and requests. This data, obtained outside the phone system, will be analyzed to determine what changes could be made to EDD's service delivery systems. This information will also assist the EDD in identifying potential enhancements to the IVR when changes to the IVR are once again allowed.

Estimated Completion Date: September 2013

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Recommendation #5 To: Employment Development Department

To evaluate the effectiveness of its other efforts to provide services to claimants in ways that do not require them to speak to agents, such as Web-Cert and Tele-Cert, the department should periodically summarize and assess the more robust management information available under its new phone system.

Annual Follow-Up Agency Response From August 2017

Completion Date: August 2017

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Fully implemented based on documentation cited in agency response.

Annual Follow-Up Agency Response From September 2016

The EDD regularly utilizes the management information that is available under the new phone system and UI Online(SM), as well as data provided by other methods, such as the UI Self-Service Options Survey, to implement this recommendation. The result of these efforts has allowed EDD to evaluate the effectiveness of service methods that do not require the claimant to speak to an agent. The EDD has enhanced and expanded its service delivery systems, such as making updates to the Interactive Voice Response System, and developing new methods of educating UI customers about UI Online(SM) to reduce the demand to speak to an EDD staff member.

Completion Date: May 2016

California State Auditor's Assessment of Annual Follow-Up Status: Partially Implemented

Auditee did not address all aspects of the recommendation

Annual Follow-Up Agency Response From September 2015

This recommendation has been partially implemented. The EDD acknowledges the value in obtaining detailed information to evaluate the effectiveness of its self-service efforts. Accordingly, EDD will utilize the UI Self-Service Options Surveys as described in response to Recommendation No. 4 to obtain the information needed to determine what changes could be made to EDD's service delivery systems. This information will also assist EDD in identifying potential enhancements to the Interactive Voice Response System.

Estimated Completion Date: Summer of 2016

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2014

This recommendation has been partially implemented.

The EDD acknowledges the value in obtaining detailed information to evaluate the effectiveness of its self-service efforts. Accordingly, EDD will utilize the UI Self-Service Options Surveys as described in response to Recommendation No. 4 to obtain the information needed to determine what changes could be made to EDD's service delivery systems. This information will also assist EDD in identifying potential enhancements to the Interactive Voice Response System.

The EDD anticipates completion of this recommendation by the end of 2015.

Estimated Completion Date: December 2015

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2013

This recommendation has been partially implemented due to chronic federal underfunding in the UI program, which has led to reduced staff, combined with the need to implement the UI Modernization Continued Claims Redesign project as a high priority. The EDD acknowledges the value in obtaining detailed information to evaluate the effectiveness of its self-service efforts. System reports do not provide insight as to why callers request to speak to agents and are unable to use EDD's self-service features. Accordingly, the EDD will utilize the UI Self-Service Options Surveys as described in response to Recommendation #4 to obtain the information needed to determine what changes could be made to EDD's service delivery systems. This information will also assist EDD in identifying potential enhancements to the Interactive Voice Response system.

Dependent on the extent of its federal underfunding, EDD anticipates completion by the end of 2014.

Estimated Completion Date: End of 2014

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2012

This recommendation has not been fully implemented; the EDD anticipates that this status will be confirmed by the BSA follow-up review report expected to be issued in November 2012. The EDD has made significant improvements in increasing the number of callers who have accessed UI phone services. Between 2009 and 2011, there was a 94.4 percent decrease in blocked calls and an increase of 77 percent in answered calls. A key factor responsible for these improvements is the implementation of the UI Modernization Call Center Network Platform and Application Upgrade project. Implementation of this project contributed significantly to improving access to EDD's call services when Primary Call Centers received the new call center infrastructure and the capacity was expanded to include Primary Adjudication Centers in February 2011 and the Anaheim Pacific Center in June 2011.

The CCR project will significantly enhance the IVR system by adding new claimant self-service functionality. When the EDD completes the CCR project in March 2013, it will:

Estimated Completion Date: September 2013

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Recommendation #6 To: Employment Development Department

To maximize federal funding and provide unemployment benefits to those eligible under the alternate base period, the department should closely monitor its resources and project schedule to avoid any further delays in implementing the client database and ensure that it completes the alternate base period project by the federal deadline.

1-Year Agency Response

The federal labor department certified the department's application for incentive funds in June 2011 and the department received a maximum transfer of $839 million in July 2011. In July 2012 the department reported to the Legislature that it had successfully implemented the alternate base period and that it had processed 1,767 valid claims for the alternate base period as of May 12, 2012, which we verified. (2013-406, p. 129)

Response Date: November 2012

California State Auditor's Assessment of 1-Year Status: Fully Implemented

Recommendation #7 To: Technology, California Department of

To help ensure that the department completes the alternate base period project by the federal deadline so that the State preserves its eligibility to receive $839 million in incentive funds, the California Technology Agency should closely monitor the department's progress toward implementing the client database and alternate base period projects and provide assistance to the department, as necessary.

1-Year Agency Response

According to its one-year response, the Technology Agency monitored monthly project status reports and project schedules and met with the department bi-weekly to review progress, issues, and risks specific to the Alternate Base Period project. (2013-406, p. 129)

Response Date: March 2012

California State Auditor's Assessment of 1-Year Status: Fully Implemented

Recommendation #8 To: Employment Development Department

To better track and improve the timeliness of determinations for the training benefits program and to assist claimants in understanding self arranged training requirements, the department should take measures to ensure that its staff correctly enter all data into the training benefits program's streamline database.

1-Year Agency Response

The department reported that it had made corrections to the database to ensure that data fields are validated and to prevent blank or empty fields. Our follow-up review assessed the data in the streamline database, and the department appears to have corrected the issues we initially identified. (2013-406, p. 129)

Response Date: November 2012

California State Auditor's Assessment of 1-Year Status: Fully Implemented

Recommendation #9 To: Employment Development Department

To better track and improve the timeliness of determinations for the training benefits program and to assist claimants in understanding self-arranged training requirements, the department should track and report the number of claimants it determines are both eligible and ineligible for the self-arranged training and the reasons for these determinations, to better focus some of its recommendations toward how it can assist claimants in understanding the program's criteria.

Annual Follow-Up Agency Response From October 2014

In December 2013, the EDD implemented a significant change to its eligibility policy for claimants who are attending school or training. Effective December 7, 2013, there is no eligibility issue and UI benefit payments continue uninterrupted when claimants certify that they began training and that they are available for work.

This change in policy significantly reduced the number of California Training Benefits that EDD conducts, which are only necessary when claimants certify they are unavailable for work or not seeking work due to school or training attendance. In addition, eligibility for the training benefits program becomes moot if claimants no longer need to be exempted from availability and seek work requirements.

After the statewide implementation of the streamlined eligibility process in September 2013, EDD now sends detailed information about the training benefits program and eligibility criteria along with a request for training program information when claimants certify they are attending school or training and unavailable for work, and there are no additional non-school eligibility issues present.

Since 2011, EDD has continued to produce and monitor weekly reports that track the number of claimants who were eligible or ineligible for the training benefits program and the statutory reasons for the outcome. The EDD collects extensive data through the streamline process, which is used to continuously improve determination timeliness, identify why claimants fail to qualify for the training benefits program, and better assist claimants in understanding the program's criteria.

Completion Date: January 2014

California State Auditor's Assessment of Annual Follow-Up Status: Resolved

EDD provided us with a document that reports a significant decrease in determinations related to the California Training Benefits program since EDD changed its eligibility policy for school or training attendance in December 2013. Specifically, EDD reported that overall determinations related to the California Training Benefits program from 2013 to November 2014 decreased by more than half, which significantly mitigates the concerns we raised in our report.

Annual Follow-Up Agency Response From October 2013

The data identified that claimants are frequently ineligible under California Unemployment Insurance Code 1269.1(b), which establishes the claimant's need for training. The EDD identified that staff were determining eligibility based on information that did not effectively reflect current labor market conditions. To assist staff in basing decisions on current conditions, the EDD released a UI Program Notice in May 2013 with specific criteria to assist staff in determining whether claimants need training or have job prospects with existing skills. The EDD continues to evaluate the effectiveness of these procedures by monitoring weekly reports.

Using this data, the EDD also continues to focus on developing the most effective means to communicate program criteria to claimants. As part of the effort to implement SB 1291, Chapter 278, in 2014 that expands eligibility for training benefits, EDD evaluated current web content and identified updates and additions to release with the new eligibility criteria. In addition, the UI program continues to work closely with workforce partners to educate claimants on criteria for self-arranged training eligibility and to assist claimants in seeking programs that have pre-approved eligibility for the program.

Dependent on the extent of its federal underfunding, EDD anticipates completion by the end of 2014.

Estimated Completion Date: End of 2014

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2012

The EDD believes this recommendation has been fully implemented; however, the EDD anticipates BSA will consider this recommendation to be partially implemented because the EDD has not yet developed the comprehensive recommendations regarding program improvements that will be due to the Legislature in 2016. The EDD acknowledges that it is waiting to collect more data before developing those recommendations, but it has not delayed taking action to provide additional information to claimants about the program criteria based on data collected so far.

In July 2011, the EDD implemented a process to track claimants who are eligible and ineligible for the self-arranged training in the CTB program and the reasons for the determination outcomes. The EDD has posted Fact Sheets, Tip Sheets, and YouTube videos to the EDD website to assist claimants to better understand the requirements of the CTB program. The EDD will continue to analyze this information to identify the primary reasons for disqualifications and will take appropriate steps to market additional information for claimants to assist them in understanding the program's criteria.

Estimated Completion Date: 2016

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Recommendation #10 To: Employment Development Department

To better track and improve the timeliness of determinations for the training benefits program and to assist claimants in understanding self-arranged training requirements, the department should track the number of claimants that it finds to be both ineligible for self-arranged training and ultimately ineligible for unemployment benefits and develop strategies to expedite the determination process for these claimants.

Annual Follow-Up Agency Response From October 2014

As explained in the response to Recommendation No. 9, the EDD's change to its eligibility policy for school or training attendance in December 2013 addressed the concern that claimants were ultimately ineligible for UI benefits based solely on their school or training attendance. As a result of the policy change, the number of training benefits determinations has been significantly reduced. Most claimants who report school or training attendance and indicate that they remain able and available to work continue to remain eligible for UI benefits and they no longer have their payments delayed due to a determination appointment.

The EDD continues to collect detailed data from the streamlined determination process to ensure that claimants who are unavailable for work due to school or training attendance receive expedited training determinations. The data collected through the streamline process for claimants who are ineligible for both self-arranged training and UI benefits is continuously used to identify areas for improvement for all training benefits determinations.

The policy change, combined with the statewide implementation of the streamlined training benefits eligibility process, allows EDD to complete training benefits determinations on average within 48 hours of receipt of school or training information from the majority of claimants through the streamline determination process.

Completion Date: January 2014

California State Auditor's Assessment of Annual Follow-Up Status: Resolved

Annual Follow-Up Agency Response From October 2013

This recommendation has been partially implemented due to chronic federal underfunding in the UI program, which has led to reduced staff, combined with the need to implement the UI Modernization Continued Claims Redesign project as a high priority. To implement this recommendation, the EDD began collecting a random sample of claimants that it found to be both ineligible for self-arranged training and ultimately ineligible for unemployment benefits in late 2012 and on-going. The claims included in the sample were analyzed to determine the reason the claimants are ultimately ineligible for benefits and the findings were documented. The results of the review covering the fourth quarter of 2012 show that the majority of claimants were ultimately ineligible for benefits due to a voluntary separation from employment without good cause under California Unemployment Insurance Code 1256. The results from the first quarter of 2013 are being reviewed. It is taking longer to analyze the results due to a resource contention to implement the UI Modernization Continued Claims Redesign project.

The EDD continues to collect detailed data from the streamlined determination process and has routinely utilized the streamline process to identify areas for improvement for all training benefits determinations. With the statewide release of the streamline process for self-arranged training in September 2013 for claimants in continued claims status, the EDD has ensured that more claimants will receive expedited determinations and that an increasing amount of data is available for claimants who are ineligible for both self-arranged training and unemployment benefits. In addition, the successful implementation of the UI Modernization Continued Claims Redesign project will provide enhanced data collection in the future for all training benefits determinations.

Dependent on the extent of its federal underfunding, EDD anticipates completion by the end of 2014.

Estimated Completion Date: End of 2014

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

Annual Follow-Up Agency Response From October 2012

This recommendation has not been fully implemented; the EDD anticipates that this status will be confirmed by the BSA follow-up review report expected to be issued in November 2012. This recommendation was based on CTB program data from 2009. In 2009, the average time frame for processing determinations was 23 days. Through EDD's streamline efforts and improvements in overall determinations' timeliness, the average wait time for a CTB determination appointment during 2011 was reduced to an average of 9.4 days.

Claimants are denied UI benefits for many reasons not related to CTB issues such as not being able and available to work, or the reason for their separation from employment. Because the EDD does not know until the determination is complete whether or not claimants will be eligible for UI benefits, there is no way to single out claimants attending school or training for an expedited determination. The EDD continues to focus on improving the timeliness on all determinations so the CTB determinations will also be processed more timely.

Performing an analysis as recommended by the BSA would be extremely labor intensive and costly. However, the EDD agrees with BSA's most recent recommendation to implement an alternative strategy to select a quarterly random sample of claims through the end of 2013 and identify the reasons for claimants being found ineligible for CTB benefits and ultimately ineligible for UI benefits. Using this sample data, any areas that are repetitive and specifically interconnected with the CTB program will be reviewed for potential process or educational improvements.

Estimated Completion Date: February 2014

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

All Recommendations in 2010-112

Agency responses received after June 2013 are posted verbatim.