Report 2014-120 Summary - April 2015

California Public Utilities Commission

:

It Needs to Improve the Quality of Its Consumer Complaint Data and the Controls Over Its Information Systems

HIGHLIGHTS

Our audit of the California Public Utilities Commission's (commission) consumer complaint data and the controls over its information systems highlighted the following:

RESULTS IN BRIEF

The telecommunications industry has undergone a profound transformation in recent years with the advent of new technologies such as cable-based Voice over Internet Protocol (VoIP) telephone services. While federal law specifies that the Federal Communications Commission (FCC) maintains regulatory jurisdiction over interstate and international telecommunications, it generally gives the states jurisdiction over their intrastate telecommunications. With certain restrictions, California has designated responsibility for regulating its intrastate telecommunication services to the California Public Utilities Commission (commission).

The mission of the commission is to protect consumers and ensure that California utility customers have safe, reliable utility service at reasonable rates. It is also responsible for helping consumers resolve issues with the industries it regulates. The commission's Consumer Affairs Branch (branch) supports the commission's mission by helping consumers resolve disputes or informal complaints with certain utilities. The branch also provides the commission and other entities, such as the Legislature, with information about the complaints it receives from consumers regarding utilities. Branch staff enter data they receive from consumers into the Consumer Information Management System (CIMS), a database that contains, among other data, complaint information that is organized by category of complaint.

Despite the need for reliable data on the nature of the complaints the branch receives, the quality of the commission's complaint data is questionable. In 17 of the 45 complaints we selected and reviewed for accuracy, we found that the branch did not correctly categorize the complaints in CIMS.1 Although the branch has provided training to its staff in classifying complaints, we identified gaps in that training and also noted that the branch has not systematically reviewed its staff's classification of complaints. As a result, the branch's complaint data do not accurately reflect the complaints it receives, and the branch is providing users with inaccurate data. Further, although the branch has made improvements to CIMS that enhance the quality of certain complaint data elements, these improvements are not effective if complaint data are entered incorrectly when the branch first receives the complaint.

In September 2014, the branch initiated a quality management team program to increase data quality by, among other things, reviewing the accuracy of how staff classify informal complaints. However, the branch has yet to implement tools to measure this program's effectiveness. We believe continuing this program and developing such measurement tools will help the branch improve the accuracy of the data and improve its value to users.

We also found that the branch could do more to capture the complete nature of complaints consumers have reported. In addition to categorizing complaints, CIMS also allows staff to enter additional information about complaints in the form of attributes. For example, if a consumer complained to the commission that sometimes he or she did not have phone service and was also frustrated with the representative of the phone company, branch staff could categorize one of these issues as the primary reason for the complaint and the other as an attribute. We found 16 of 30 general telecommunication complaints we reviewed had information that could have been included in CIMS as attributes. However, branch staff added attributes for only four of those complaints, thereby omitting descriptive complaint information from the other 12. When the branch does not take advantage of opportunities to record complaint details, it is providing incomplete information to other commission divisions that use the branch's complaint data, and it is also missing opportunities to provide the commission's divisions and stakeholders with richer, more useful information.

The commission's ability to identify VoIP complaints is limited because the California Public Utilities Code, Section 710 is ambiguous about whether VoIP providers must provide information to the commission that would assist it in responding informally to VoIP complaints. Not all VoIP providers are required to register with the commission and report information regarding their VoIP customers, and the commission staff do not believe they have the legal authority to compel VoIP providers to report this information. According to an October 2014 branch report, the inability to connect some complaints with VoIP providers in CIMS is a key challenge in producing reports and assisting California consumers because the branch's ability to process and report on VoIP complaints is directly tied to the quality of information in CIMS about VoIP providers.

Further, in nine of the 12 VoIP-related complaints we reviewed that were submitted to the branch after it issued guidance in May 2013 for processing such complaints, the branch either did not provide appropriate information to complainants or did not forward the complaint to the utility as required. As a result, some consumers with VoIP-related complaints did not receive information that might have helped them resolve their complaints, such as contact information for the FCC.

Members of the public and entities, such as the Legislature and the FCC, may request data related to consumer complaints from the commission by making an external data request, meaning a request that comes from outside the commission. They may submit such requests to one of the commission's divisions, such as the Office of Governmental Affairs, which then typically contacts the branch to fulfill the request. In addition, commission entities, such as the Communications Division and the Safety and Enforcement Division, use branch consumer complaint data to develop policy and to identify trends, among other functions.

Our review of selected external data requests indicated that the commission generally processes the requests quickly and effectively. We did, however, identify two related requests in which a miscommunication between the requestor and commission staff may have resulted in the requestor's expectations not being fully met. To address this issue, the branch has proposed, but not formally adopted, modifications of its procedures for processing requests that we believe, if followed, could help the commission avoid miscommunications in the future.

We also determined that the commission could make information about contacts and informal complaints from telecommunications consumers (complaint data) on its website more robust and easily accessible to consumers and other stakeholders. The branch began posting complaint data to the commission's website in 2012 in order to assist consumers, and in December 2013 it developed a plan for improving the quality of the data posted online. The branch's plan envisioned including online counts of complaints organized by utility company and category of complaint by January 2014, a measure of consumer satisfaction by April 2014, and possibly a separate report of VoIP-related complaints by July 2014. As of January 2015, the branch had posted only the utility company and category-specific data. Further, the complaint data the commission posts on its website can be difficult to find because consumers must follow three nonintuitive links to navigate from the commission's home page to the location where complaint data are posted.

Finally, as part of our assessment of the validity and reliability of the CIMS data, we used the policy requirements in Chapter 5300 of the State Administrative Manual (SAM) as a benchmark for evaluating the controls the commission has implemented over its information systems because the commission acknowledges that these requirements are good business practices. We expected that the commission would have developed adequate plans, policies, and procedures to provide for the proper protection of its information assets and to ensure its ability to sustain and recover critical information technology services should an unexpected human-made or natural disaster jeopardize its information assets. However, despite certifying to the California Department of Technology (CalTech) that it had complied with all policy requirements in Chapter 5300 of SAM, we found that key information security documents either were nonexistent or lacked critical components. Specifically, the commission has yet to inventory all of its information assets, assess the risks to those assets, and develop an information security plan that provides a strategy for mitigating those risks. Further, the commission does not have an incident response plan that provides for a timely response to, and recovery from, an information security incident, such as a malicious cyber attack. Finally, although the commission has a current technology recovery plan, we question its usefulness because the plan fails to consistently identify critical applications, establish acceptable outage time frames for these applications, and develop strategies for recovery. Until the commission improves the controls it has implemented over its information systems, the confidentiality, integrity, and availability of its information systems will continue to be at risk.

RECOMMENDATIONS

Legislative

To ensure that the commission has the information it needs to better report on VoIP-related complaints, the Legislature should give the commission the authority to collect information from providers regarding their VoIP customers and require VoIP providers to furnish this information to the commission.

Commission

To ensure that policy makers, enforcement officials, and the general public have access to accurate consumer complaint data in CIMS, the branch should do the following:

To ensure that policy makers, enforcement officials, and the general public have access to more complete and meaningful consumer complaint data, the branch should, to the fullest extent possible, include the attributes of each complaint in the data it records in CIMS.

To ensure that branch staff provide the appropriate assistance to consumers with VoIP-related complaints, the branch should, by September 30, 2015, further train its staff on providing correspondence to complainants as required by its guidelines.

To ensure that consumers have access to the complaint data that will enhance their ability to make informed choices about their telecommunication services, the branch should, by June 30, 2015, create an updated plan that specifies the types of data the branch intends to post online and a timeline for fully implementing that plan.

To ensure that the public can easily locate customer complaint data the branch publishes on its website, the commission should make navigating to its customer complaint data more intuitive and direct.

The commission should ensure that it complies with all policy requirements in SAM Chapter 5300 no later than April 2016. Specifically, the commission should do the following:

AGENCY COMMENTS

The commission generally agreed with our findings and stated its goal was to address all of our recommendations, but indicated that the implementation of the recommendations is dependent upon available resources.

1 We tested 30 complaints selected from all telecommunication complaints received in fiscal years 2011-12 through 2013-14 (general complaints). We also reviewed 15 VoIP-related complaints received by the branch between January 1, 2013, and June 30, 2014.