Report 2010-102 Recommendations

To better manage costs of future IT projects, the AOC should estimate costs at the inception of projects.

To better manage costs of future IT projects, the AOC should employ appropriate budget and cost management tools to allow it to appropriately budget, track, manage, and estimate costs.

To better manage costs of future IT projects, the AOC should ensure that cost estimates are accurate and include all relevant costs, including costs that superior courts will incur.

To better manage costs of future IT projects, the AOC should disclose costs that other entities will likely incur to the extent it can reasonably do so.

To better manage costs of future IT projects, the AOC should update cost estimates on a regular basis and when significant assumptions change.

To better manage costs of future IT projects, the AOC should disclose full and accurate cost estimates to the Judicial Council, the Legislature, and stakeholders from the beginning of projects.

To better manage costs of future IT projects, the AOC should ensure that it has a long-term funding strategy in place before investing significant resources in a project.

Although the Judicial Council has the legal authority to compel the courts to adopt CCMS, to better foster superior court receptiveness to deploying CCMS, the AOC should continue to work with the superior courts that have deployed the civil system to ensure it is addressing their concerns in a timely and appropriate manner.

Although the Judicial Council has the legal authority to compel the courts to adopt CCMS, to better foster superior court receptiveness to deploying CCMS, the AOC should work with superior courts to address concerns about hosting data at the California Court Technology Center (Technology Center). Further, the AOC should take steps to ensure that superior courts do not lose productivity or efficiencies by hosting data at the Technology Center.

To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should obtain IV&V and IPO services at the beginning of the projects and ensure this independent oversight is in place throughout and follows best practices and industry standards appropriate for the size and complexity of the project.

To understand whether CCMS is a cost-beneficial solution to the superior courts' case management needs, the AOC should continue with its planned cost-benefit study and ensure it completes this study before spending additional significant resources on the project. The AOC should ensure that this study includes a thorough analysis of the cost and benefits of the statewide case management project, including a consideration of costs and benefits it believes cannot be reasonably quantified. The AOC should carefully evaluate the results of the study and present a recommendation to the Judicial Council regarding the course of action that should be taken with CCMS. Further, the AOC should fully share the results of the study as well as its recommendation to all interested parties, such as the superior courts, justice partners, the Legislature, and the Information Office. The AOC should update this cost-benefit analysis periodically and as significant assumptions change.

To ensure the statewide case management project is transparent, the AOC should make sure all key decisions for future activities on CCMS are documented and retained.

To ensure its contract with the development vendor protects the financial interests of the State and the judicial branch, the AOC should consider restructuring its current contract to ensure the warranty for CCMS is adequate and covers a time period necessary to ensure that deployment of CCMS has occurred at the three early adopter courts and they are able to operate the system in a live operational environment.

If the Judicial Council determines that CCMS is in the best interest of the judicial branch and it directs the AOC to deploy the system statewide, assuming funding is available, the AOC should ensure that any contract it enters into with a deployment vendor includes cost estimates that are based on courts’ existing IT environments and available resources to assist with deployment activities.

If the Judicial Council determines that CCMS is in the best interest of the judicial branch and it directs the AOC to deploy the system statewide, assuming funding is available, the AOC should ensure that any contract it enters into with a deployment vendor includes well-defined deliverables.

If the Judicial Council determines that CCMS is in the best interest of the judicial branch and it directs the AOC to deploy the system statewide, assuming funding is available, the AOC should ensure that any contract it enters into with a deployment vendor includes adequate responsibility is placed on the vendor for conducting key steps in the deployment of the system.

The Judicial Council should make certain that the governance model for CCMS ensures that approval of contracts and contract amendments that are significant in terms of cost, time extension, and/or change in scope occur at the highest and most appropriate levels, and that when contracts or contract amendments above these thresholds are approved, that the decision makers are fully informed regarding both the costs and benefits.

To ensure that any future IT projects are in the best interest of the judicial branch and the State, the AOC should complete a thorough analysis of the project’s cost and benefits before investing any significant resources and time into its development, and update this analysis periodically and as significant assumptions change.

To ensure that any future IT projects are in the best interest of the judicial branch and the State, the AOC should document and retain all key decisions that impact the project in general, including the goals of the project.

To ensure that any future IT projects are in the best interest of the judicial branch and the State, the AOC should better structure contracts with development and deployment vendors to protect the financial interests of the judicial branch and ensure the contracts provide for adequate warranty periods.

To ensure that the financial implications of the statewide case management project are fully understood, the AOC should report to the Judicial Council, the Legislature, and stakeholders a complete accounting of the costs for the interim systems and CCMS. This figure should be clear about the uncertainty surrounding some costs, such as those that the AOC and superior courts will incur for deployment of CCMS.

The AOC should require superior courts to identify their past and future costs related to the project, particularly the likely significant costs that superior courts will incur during CCMS deployment, and include these costs in the total cost.

The AOC should be clear about the nature of the costs that other entities, such as justice partners, will incur that are not included in its total.

The AOC should update its cost estimate for CCMS on a regular basis as well as when significant assumptions change.

To address the funding uncertainty facing CCMS, the AOC should work with the Judicial Council, the Legislature, and the governor to develop an overall strategy that is realistic given the current fiscal crisis facing the State.

Although the Judicial Council has the legal authority to compel the courts to adopt CCMS, to better foster superior court receptiveness to deploying CCMS, the AOC should use the results from its consultant’s survey of the superior courts to identify and better understand the courts’ input and concerns regarding CCMS, including the manner in which the project has been managed by the AOC. To the extent the survey results indicate courts have significant concerns regarding CCMS or that they believe their case management systems will serve them for the foreseeable future, the AOC should take steps to address these concerns and overcome any negative perceptions and modify its deployment plan for CCMS accordingly.

The AOC should continue working with local and state justice partners to assist them in their future efforts to integrate with CCMS, and in particular provide local justice partners the information needed to estimate the costs involved.

Before embarking on future statewide IT initiatives and to ensure it secures appropriate support from users of the systems being proposed, the AOC should determine the extent to which the need for the IT initiative exists, including the necessary information to clearly demonstrate the extent of the problem the IT initiative will address.

Before embarking on future statewide IT initiatives and to ensure it secures appropriate support from users of the systems being proposed, the AOC should take steps to ensure that superior courts support the solution the AOC is proposing to address the need, which could include conducting a survey of courts to determine their level of support.

Before embarking on future statewide IT initiatives and to ensure it secures appropriate support from users of the systems being proposed, the AOC should if necessary, determine whether other stakeholders, including local and state justice partners, support the IT initiative.

To provide for an appropriate level of independent oversight on CCMS, the AOC should expand and clarify the scope of oversight services and require that oversight consultants perform oversight that is consistent with best practices and industry standards.

To ensure that no gaps in oversight occur between CCMS development and deployment, the AOC should ensure that it has IV&V and IPO services in place for the deployment phase of CCMS. Further, to allow for independent oversight of the IV&V consultant, the AOC should use separate consultants to provide IV&V and IPO services.

To ensure no significant quality issues or problems exist within CCMS, the AOC should retain an independent consultant to review the system before deploying it to the three early adopter courts. This review should analyze a representative sample of the requirements, code, designs, test cases, system documentation, requirements traceability, and test results to determine the extent of any quality issues or variances from industry standard practices that would negatively affect the cost and effort required of the AOC to operate and maintain CCMS. If any quality issues and problems identified by this review can be adequately addressed, and system development can be completed without significant investment beyond the funds currently committed, the AOC should deploy it at the early adopter courts during the vendor’s warranty period.

To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should employ separate firms for IV&V and IPO services to allow for the IPO consultant to provide independent oversight on the IV&V consultant as well as the project team’s response to IV&V findings.

To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should ensure that the staff performing IV&V and IPO services have experience and expertise that is commensurate with the size, scope, and complexity of the project they are to oversee.

To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should ensure that independent oversight is not restricted in any manner and that all parties—the IV&V and IPO consultants, senior management, the project management team, and the development vendor—understand that the IV&V and IPO consultants are to have complete access to all project materials.

To ensure that future major IT projects receive appropriate independent oversight over technical aspects and project management, the AOC should address promptly and appropriately the concerns that independent oversight consultants raise.