Report 2017-107 All Recommendation Responses

Report 2017-107: Santa Clara County Registrar of Voters: Insufficient Policies and Procedures Have Led to Errors That May Have Reduced Voters' Confidence in the Registrar's Office (Release Date: October 2017)

Recommendation #1 To: Santa Clara County Registrar of Voters

To ensure the accuracy of voting district boundaries and to allow Santa Clara to make changes to existing boundaries as necessary, Santa Clara should establish a procedure requiring Mapping staff to ask each voting district either to certify that its boundaries are accurate and unchanged or to provide an updated map of its boundaries at least 125 days before each general district election.

60-Day Agency Response

Originally responded 12/22/17. We have developed and implemented a comprehensive procedure, and accompanying policy, whereby Mapping Division staff will, at least 125 days before each general district election (i.e., Consolidated Election held in odd-numbered years, and Primary Election held in even-numbered years), mail notification letters to all voting districts requesting confirmation of district boundaries. A paper map representing the most current map on file with the Registrar of Voters (ROV) will accompany the notification letter. The paper map includes a section for the district official to either sign-off on the accuracy of the map on file, or else provide additional information so that the Mapping Division can make changes to the district boundary. If the Mapping Division makes changes, an updated map would be sent to the district for a fresh review and sign-off. Included in this procedure is the maintenance of a detailed Tracking Log. This procedure has already been applied beginning in November 2017 and has resulted in better response and information than received in prior years, with the map package receiving positive feedback from several districts. The procedure and policy have been attached separately as Supporting Documentation.

  • Completion Date: December 2017
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Fully Implemented


Recommendation #2 To: Santa Clara County Registrar of Voters

Santa Clara should immediately coordinate with Information Services to access the most current maps from other county departments, such as the Santa Clara Assessor, to verify the accuracy of the district maps.

60-Day Agency Response

Originally responded 12/22/17. Since 2013, the Mapping Division has received a report annually from the Santa Clara County Assessor's Office that has been used for proofing district boundaries by verifying which districts belong to which addresses based on the Assessor's active roll. As stated in the State Audit Report (2017-107), a meeting was held in May 2017 with Information Services Department (ISD) representatives, Assistant Registrar of Voters Virginia Bloom, Mapping Division Manager Bob Nguyen, and the State Audit Team, whereat it was determined that more frequent access to the Assessor's rolls would be granted on a quarterly basis. On August 10, 2017, ISD provided the Santa Clara County Statement of Understanding Pertaining to GIS Data to the ROV Mapping Division Manager for execution, allowing for open access (i.e., at any time). Note that the Assessor's open access data sets are designated as "interim reference" data, except when released as part of the published annual roll. On December 14, 2017, Virginia Bloom and Bob Nguyen attended the kickoff County-wide Enterprise GIS committee meeting, and as committee members plan to attend and request relevant GIS data-sharing agenda items be added to future meetings. The signed Understanding Pertaining to GIS Data and materials related to the Enterprise GIS committee meeting have been attached separately as Supporting Documentation. An update to the status of this recommendation will be made at the next report back to the State Auditors.

  • Estimated Completion Date: March 2018
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Pending


Recommendation #3 To: Santa Clara County Registrar of Voters

To reduce errors and potentially its workload, Santa Clara should research by January 2018 its opportunities to integrate mapping software with its election management software, and Santa Clara should implement this integration of mapping software technology by June 2018.

60-Day Agency Response

Originally responded 12.22.17. On several occasions in recent months we have reached out to our election management system vendor, DFM Associates, requesting a meeting to discuss options for enhancement of the Precinct/District Modules of the Election Information Management System (EIMS). A request was also sent out to all counties via the California Association of Clerks and Election Officials' (CACEO) ElecNet listserv requesting best practices and procedures, including those relating to import/export options for district boundary verification. The Mapping Division Manager has prepared a summary of the best practices received by other counties. He called Alameda County to request information relating to their import/export program, however was told that the import/export program is proprietary to the vendor (DIMS) of that county's system, therefore Alameda was not at liberty to share it with us. A follow-up to our inquiry with our vendor DFM was made on December 14, 2017, at which time DFM's representatives acknowledged receipt of our request and advised that they have not conducted any research into further options, offering that our staff work with theirs for use of existing reports for proofing purposes. The Summary of Best Practices in other counties, correspondence with other counties and DFM, and a memo regarding recent communications have been attached separately as Supporting Documentation. Further research will be conducted and an update to the status of this recommendation will be made at the next report back to the State Auditors.

  • Estimated Completion Date: June 2018
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Pending


Recommendation #4 To: Santa Clara County Registrar of Voters

Santa Clara should promptly seek compensation from its vendors for all costs associated with rectifying vendor errors that occur in the future.

60-Day Agency Response

Originally responded 12.22.17. On December 15, 2017, we implemented "ROV Policy - Decision Matrix_Implemented 121517," which incorporates the action item to track time spent by staff on rectifying vendor errors, and the requirement to seek reimbursement for error costs where vendor agreements allow. This two-part action item in the ROV Decision Matrix (see 12.22.17 Response #9) will be carried out when vendor product(s) and/or service(s) are determined to be root cause of an error in voting materials or information. The action items that will be carried out after each such error include: a) Identify Root Cause, a.i) If cause of error was due to vendor or district error, then add follow-up with vendor/district, conduct timesheet time study for hours spent on error, and seek reimbursement for error costs (where vendor agreements allow), b) Determine Affected Voters or Potential Affected Voters, c) Identify Prevention Steps, d) Update Error Tracking Log, d.i) Document any reasons from deviating from this Decision Matrix in the ROV Error Tracking Log, and e) Include Information in Post-Election Report. While this policy was implemented on December 15, 2017, the ROV has yet to have the opportunity to put the policy into practice. ROV Policy - Decision Matrix_Implemented 121517 has been attached separately as Supporting Documentation.

  • Completion Date: December 2017
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Fully Implemented

To the extent that Santa Clara follows its established policy by seeking compensation from its vendors for all costs associated with rectifying vendor errors that occur in the future, this recommendation is fully implemented.


Recommendation #5 To: Santa Clara County Registrar of Voters

To make certain that its staff learn of election-related errors and identify trends in error types, and to allow Santa Clara to identify necessary modifications to processes that will reduce or eliminate such errors, Santa Clara should immediately formalize a policy requiring the continued use of a spreadsheet similar to the one it created to track election-related errors.

60-Day Agency Response

Originally responded 12.22.17. On December 15, 2017, we implemented "ROV Policy - Tracking Errors_Implemented 121517," which requires the Registrar of Voters administration to maintain an error log in spreadsheet format. The information the spreadsheet "ROV Error Tracking Log" will track include : a) Election Date, b) Election Type, c) Error Description, d) Approximate # of Voters Affected, e) # Ballot Types or CVIGs Affected, f) Notification Methods, g) ROV Error?, h) Other Error Source(s), i) Believed Root Cause, j) Steps Taken/Procedure Changes/Notes, and k) Deviations from ROV Decision Matrix. The ROV Error Tracking Log will be shared with managers and staff annually at a Leadership Team Meeting in January, and again after any election-related error necessitates an update to the log. Any deviations to the post-error actions spelled out in the ROV Decision Matrix (see 12.22.17 Response #9) will be documented in the ROV Error Tracking Log. While this policy was implemented on December 15, 2017, the ROV has yet to have the opportunity to put the policy into practice. The policy has been attached separately as Supporting Documentation.

  • Completion Date: December 2017
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Fully Implemented


Recommendation #6 To: Santa Clara County Registrar of Voters

To ensure accuracy and consistency in the creation, review, and distribution of election-related materials, Santa Clara should review and document in detail all policies and procedures by October 2018, prioritizing its documentation for the divisions that are responsible for the most frequent and egregious election-related errors. Specifically, Santa Clara should review and formalize Mapping policies and procedures by January 2018, to allow time for implementation before the June primary election process. By October 2018, Santa Clara should review and formalize policies and procedures for the remaining divisions--including Ballot Layout, Candidate Services, and Vote by Mail--to provide adequate time for implementation before the November general election process.

6-Month Agency Response

Progress from the last report includes continued efforts to document all policies and procedures in time for implementation before the June 2018 primary election process. For the record, several key positions within the Registrar's Office have changed since the December 2017 report. The vacant Management Analyst position of the Policies & Procedures Division has been filled as of late March. And, also for that division, the recruitment for the vacant Associate Management Analyst is on-going. This division is responsible for the creation and implementation of the Department's Policies and Procedures. The Election Division Coordinators for both the Ballot Layout Division (BLD) and the Candidate Services Division (CSD) have both retired effective end of 2017. The Election Process Supervisor was promoted to Coordinator in BLD and now have several new personnel within BLD. In CSD, the recruitment to fill this vacancy is on-going at this time. The office has also hired three Management Aide positions to help write up the documentation of training, policies, and procedures while working with each of the specific division managers. We have begun to formalize policies and procedures for all of our divisions prioritizing the Ballot Layout, Candidate Services, Mapping and Vote-by-Mail divisions. Numerous draft documents have been created for each of the named divisions and tested during the nomination filing and ballot layout periods of the June 2018 Election. Those documents are currently under review at this time. It is anticipated that approved final policies and procedures will be produced at the next report back in October 2018.

  • Estimated Completion Date: October 2018
  • Response Date: April 2018

California State Auditor's Assessment of 6-Month Status: Pending


60-Day Agency Response

Originally responded 12.22.17. In accordance with the recommended implementation time frame of January 2018, we have reviewed and formalized a policy and numerous procedures for the Mapping Division. We have begun reviewing and formalizing policies and procedures for the Ballot Layout, Candidate Services, and Vote-by-Mail divisions targeting completion by October 2018, in order to meet the recommended implementation timeframe and provide adequate time for implementation before the November 2018 General Election. To the extent possible, the intention is to centralize all procedures for each division into a unified procedure manual for each division. We have prioritized the Mapping Division Procedure Manual first in order to meet the January 2018 recommended implementation time frame, and implement Recommendation #1's procedure to request each voting district certify that its boundaries are accurate through receiving and verifying the accuracy of the current map on file with the Registrar of Voters. In addition, on December 15, 2017, we implemented "ROV Policy - Requesting Proof from Printing Services or Vendor_Implemented 121517" in order to address a concern in the State Audit report #2017-107 that we had not documented our practice of verifying the accuracy of voter data in print or mailing jobs. This policy, a new policy for the Mapping Division to implement the procedure to request each voting district certify its boundaries, and the Mapping Division Procedure Manual as of December 22, 2017, have been attached separately as Supporting Documentation.

  • Estimated Completion Date: October 2018
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Pending


Recommendation #7 To: Santa Clara County Registrar of Voters

To reduce the risk of staff errors, inconsistencies in procedures, and the loss of institutional knowledge in the creation, review, and distribution of election-related materials, Santa Clara should develop and implement training for its staff that includes instructions on its comprehensive policies and procedures. The development of this training should take place concurrently with Santa Clara's detailed documentation of its policies and procedures, and Santa Clara should require relevant staff to attend this training before each major election.

6-Month Agency Response

Progress from the last report includes continued efforts to formalize the training by division to align with the policies and procedures created. The EIMS vendor, DFM, was contacted and is currently conducting trainings of its specific modules for each of the specific divisions; i.e., Candidate Services, Precinct Operations, and VoteCal for our Voter Registration Division. Evidence as supporting documentation has been uploaded separately. Additionally, CACEO is offering Signature Verification Training in different locations throughout the state and the office has made arrangements to send key staff to attend those trainings. Also attached separately as Supporting Documentation are the staff emails identifying those to attend, as well as the CACEO flyers announcing the trainings. Individual Training by Position procedures, as well as sign-in sheet templates, have been created for the Candidate Services Division, Ballot Layout Division, Vote by Mail Division, and the Mapping Division. Each of those documents have been uploaded separately as Supporting Documentation, as well as evidence of those trainings by division. The Mapping Division also collected the training materials from the County ISD GIS division which is also included. Specific training was also created for the Candidate Services and Ballot Layout divisions relating to the Changes and Issues Log as outlined in the Handoff procedures for each of those divisions.

  • Estimated Completion Date: October 2018
  • Response Date: April 2018

California State Auditor's Assessment of 6-Month Status: Pending


60-Day Agency Response

Originally responded 12.22.17. As the development of more detailed and centralized policies and procedures progresses, our division managers will introduce more formalized training for staff that includes instruction on the comprehensive policies and procedures. Staff will be required to attend this formalized training prior to each major election. The recently updated Mapping Division Procedure Manual (see 12.22.17 Response #6) will facilitate formalized training of the Mapping Division staff prior to the major elections in 2018. The Candidate Services Division has begun the process of formalizing its nomination training by requiring, before each training session, staff to sign an attendance sheet that includes the materials to be covered for that session. In addition, procedures have been created for developing and conducting nomination training of both staff and candidates for elected office. A procedure for preparing and conducting nomination training for staff, a procedure for preparing for and conducting a judicial seminar, and a procedure for preparing for and conducting a seminar for city office and district candidates, have been attached separately as Supporting Documentation. Also attached separately as Supporting Documentation are the staff nomination training attendance sheets and the PowerPoint presentation used for the December 7, 2017, Judicial Seminar.

  • Estimated Completion Date: October 2018
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Pending


Recommendation #8 To: Santa Clara County Registrar of Voters

To ensure the accuracy of election-related materials, Santa Clara should immediately implement a procedure for candidates, voting districts, or others who submit documents to have them verify the accuracy of the electronic versions of those documents once Santa Clara has formatted them.

60-Day Agency Response

Originally responded 12.22.17. We have developed a procedure for candidates, voting districts, and measure argument submitters to verify the accuracy of electronic versions of voting material documents once the Ballot Layout Division has formatted them, against the original submitted documents. This procedure, called the "Municipality and Author Review," requires the Candidate Services Division to ensure a legible email address is provided, and after materials have been transferred to Ballot Layout, the Ballot Layout Division to create a special log for tracking the review process. After a document has been formatted in English, it is a) proofread multiple times, b) placed in a "Done" tray, c) uploaded to the translation vendor's portal, d) emailed to the municipal clerk or author, e) logged as "emailed" in the special log, and f) filed in batch order awaiting response. Any responses are reviewed to ensure if they are valid (i.e., an error was introduced by the ROV) or invalid (i.e., the error was in the original). These actions are recorded in the special log, until all documents have been checked off as completed. While this procedure was approved on December 21, 2017, the ROV has yet to have the opportunity to put the procedure into practice. The procedure and sample tracking log have been attached separately as Supporting Documentation.

  • Completion Date: December 2017
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although Santa Clara has developed a procedure for candidates, voting districts, and others who submit documents to have them verify the accuracy of the electronic versions of those documents once it has formatted them, it has not yet implemented this procedure. Until Santa Clara can demonstrate that it has implemented this procedure, we will continue to report this recommendation as partially implemented.

  • Auditee did not substantiate its claim of full implementation

Recommendation #9 To: Santa Clara County Registrar of Voters

To ensure consistency in responding to election-related errors, Santa Clara should immediately implement a contingency plan or decision matrix that includes specific guidelines for the actions it will take based on the number of voters affected and the significance of the error. In instances in which it chooses to deviate from this plan, Santa Clara should document its reasons for deciding to do so.

60-Day Agency Response

Originally responded 12.22.17. On December 15, 2017, we implemented "ROV Policy - Decision Matrix_Implemented 121517," which requires ROV administration, after an error in voting materials has been discovered, to follow the guidelines listed in a written decision tool. This written decision tool we created, called the "ROV Decision Matrix," includes several action items that must be taken after any error that affects voting ability is discovered (see 12.22.17 Response #5), while specifying certain actions be taken depending on the severity or circumstances of the error. Any deviations from the guidelines in the ROV Decision Matrix will be documented in the ROV Error Tracking Log (see 12.22.17 Response #5). While this policy was implemented on December 15, 2017, the ROV has yet to have the opportunity to put the policy into practice. The policy has been attached separately as Supporting Documentation.

  • Completion Date: December 2017
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Fully Implemented

To the extent that Santa Clara follows its established decision matrix when responding to election-related errors, this recommendation is fully implemented.


Recommendation #10 To: Santa Clara County Registrar of Voters

To maintain the public's confidence in it and its functions, Santa Clara should immediately include in its postelection reports descriptions of any election-related errors, accounts of why the errors occurred, and explanations of how it plans to prevent similar errors from occurring in the future.

6-Month Agency Response

Since the last report of December 2017, our office has not conducted any elections nor prepared any Post Election Reports; therefore, we have not yet had an opportunity to include any descriptions of any election-related errors, accounts of why the errors occurred, and explanations of how we plan to prevent the errors from occurring in the future.

  • Estimated Completion Date: October 2018
  • Response Date: April 2018

California State Auditor's Assessment of 6-Month Status: Partially Implemented


60-Day Agency Response

Originally responded 12.22.17. On December 15, 2017, we implemented "ROV Policy - Decision Matrix_Implemented 121517," which incorporates the requirement to include information concerning an error in voting materials in the Post-Election Report published after each election. Information concerning the error will be based on specific details documented in the ROV Error Tracking Log. This information will include a description of the error, an account of the reason(s) for the error, and an explanation of how the department plans to prevent future errors (see 12.22.17 Responses #4, #5, and #9). While this policy was implemented on December 15, 2017, the ROV has yet to have the opportunity to put the policy into practice. ROV Policy - Decision Matrix_Implemented 121517 has been attached separately as Supporting Documentation.

  • Completion Date: December 2017
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Partially Implemented

Although Santa Clara established a policy which requires it to include information about errors in voting materials in its post-election reports, it has not yet demonstrated that it is following this policy. Until Santa Clara can demonstrate that it is following this policy, we will continue to report the status of this recommendation as partially implemented.

  • Auditee did not substantiate its claim of full implementation

Recommendation #11 To: Secretary of State, Office of the

The Secretary of State should adopt regulations establishing clear criteria for mistakes in election-related materials that constitute reportable errors and require counties to report these errors to it after each election.

6-Month Agency Response

The Secretary of State does not currently have the resources to develop and promulgate regulations during the 2018 election cycle, which has already begun; however, we plan to do so in the next non-statewide election year, 2019. For the 2018 election cycle, the Secretary of State has asked counties report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur. In addition, we will use any reported errors to develop best practices to share with all county elections officials to assist them in the preparation of their election-related materials.

  • Estimated Completion Date: June 2019
  • Response Date: April 2018

California State Auditor's Assessment of 6-Month Status: Pending


60-Day Agency Response

The Secretary of State does not currently have the resources to develop and promulgate regulations during the 2018 election cycle, which has already begun; however, we plan to do so in the next non-statewide election year, 2019. For the 2018 election cycle, the Secretary of State will request counties report to our office any substantive errors in election-related materials provided to voters so that we may assist in mitigating those errors as they occur. In addition, we will use any reported errors to develop best practices to share with all county elections officials to assist them in the preparation of their election-related materials.

  • Estimated Completion Date: June 2019
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Pending


Recommendation #12 To: Secretary of State, Office of the

Beginning in December 2018, the Secretary of State should implement annual risk-based reviews of a selection of county election officials' offices to ensure their compliance with state election laws and regulations.

6-Month Agency Response

The Secretary of State will begin conducting annual reviews of county elections officials' offices to ensure compliance with election laws and rules in December 2018 as resources allow.

  • Estimated Completion Date: December 2018
  • Response Date: April 2018

California State Auditor's Assessment of 6-Month Status: Pending


60-Day Agency Response

The Secretary of State will begin conducting annual reviews of county elections officials' offices to ensure compliance with election laws and rules in December 2018 as resources allow.

  • Estimated Completion Date: December 2018
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Pending


Recommendation #13 To: Secretary of State, Office of the

To inform and enhance the guidance it provides to county election officials, the Secretary of State should analyze error reports and its risk-based review results to focus its guidance on topics most relevant to improving elections throughout the State.

6-Month Agency Response

The Secretary of State has requested counties provide reports of substantive errors in the creation and distribution of election-related materials in 2018 and will review the reports in order to provide guidance and best practices to all county elections officials in the preparation of their election-related materials. After the promulgation of regulations, we will require error reports from counties.

  • Estimated Completion Date: June 2019
  • Response Date: April 2018

California State Auditor's Assessment of 6-Month Status: Pending


60-Day Agency Response

The Secretary of State will request counties provide reports of substantive errors in the creation and distribution of election-related materials in 2018 and will analyze the reports in order to provide guidance and best practices to all county elections officials in the preparation of their election-related materials. After the promulgation of regulations, we will require error reports from counties.

  • Estimated Completion Date: June 2019
  • Response Date: December 2017

California State Auditor's Assessment of 60-Day Status: Pending


All Recommendations in 2017-107

Agency responses received are posted verbatim.