Report 2010-122 Recommendation 4 Responses

Report 2010-122: California Department of Transportation: Its Capital Outlay Support Program Should Strengthen Budgeting Practices, Refine Its Performance Measures, and Improve Internal Controls (Release Date: April 2011)

Recommendation #4 To: Transportation, Department of

To improve performance metrics related to the support program, Caltrans should continue to explore the use of additional metrics, such as a measure based on a productivity index as described in a March 2011 draft study by the University of California, Davis.

Annual Follow-Up Agency Response From October 2013

Efforts to date on this recommendation included the results of the UC Davis Study and the Production Study modeled after the UC Davis Study. There were no useful applications for measurable metrics from either study. A copy of the Production Study was transmitted with the 2012 annual status.

After an additional year of evaluation, we concluded that these efforts do not have sufficient information to evaluate productivity due to the wide diversity of project factors (Locations, Environment, Type of Project, Design Strategy, Right of Way, etc.) that contribute to production hours needed to complete projects. At this time, Caltrans plans to continue its use of Support-to-Capital as a program-level performance metric based on project size.

California State Auditor's Assessment of Annual Follow-Up Status: Fully Implemented

Our recommendation was to continue to explore alternative approaches to measuring its performance. It appears Caltrans has done so and concluded that other measures it was reviewing were not adequate and that the support-to-capital ratio is still useful.

Annual Follow-Up Agency Response From October 2012

As Caltrans reported in the one-year status update, the additional metric was produced by the target date of July 2012. However, the Production Study identified four potential measures (a copy of the Production Study is attached.) Caltrans has reviewed the four measures and believes that measure four is a viable measure. Measure four is similar to the support-to-capital goal and uses hours to mitigate labor cost charges and indexes capital to mitigate capital escalation changes.

However, Caltrans is not prepared to adopt this as a full measure until the following is done:

• Additional years of data are needed to ensure the method reflects long-term production information.

• Additional analysis is needed to establish trends based on project groups that are based on the capital value.

• Additional analysis is needed to establish trends based project groups that are based on the capital value.

• Additional effort is needed to evaluate the pre-construction phase. The construction phase hours were evaluated first to see if there was any potential for a reasonable measure.

Once the issues discussed above are complete, Caltrans will have a better time estimate for full implementation of this recommendation. In the meantime, Caltrans is moving forward with the following timeline:

• Evaluate and determine a potential measure for project groups by capital value by January 2013.

• Evaluate and determine potential pre-construction measure by July 2013.

Caltrans believes that the effectiveness of a long-term measure will not be known for several years. The existing data sample was based on the past four years but additional years are necessary in order to have a complete set of data. Caltrans is also considering the impacts on productivity by the ongoing furloughs and the constraints of limited overtime hours.

California State Auditor's Assessment of Annual Follow-Up Status: Not Fully Implemented

All Recommendations in 2010-122

Agency responses received after June 2013 are posted verbatim.