SUBJECT
Title
OFFICE OF EVALUATION, PERFORMANCE, AND ANALYTICS: UPDATE TO THE BOARD OF SUPERVISORS AND APPROVAL TO WAIVE BOARD POLICY A-87 AND AWARD CONTRACT(S) FOR RESEARCH AND TECHNICAL ASSISTANCE TO UNIVERSITIES (DISTRICTS: ALL)
Body
OVERVIEW
Across the country, governments are facing rapidly evolving challenges and increasing demand for accountability. Properly used data provides evidence of which programs and policies achieve their intended results. For the County of San Diego (County), which serves a population of 3.3 million residents, along with an $8.2 billion budget, more than 20,000 employees, and hundreds of programs, this evidence can be an invaluable resource to help policymakers make decisions.
On May 18, 2021 (16) the County Board of Supervisors (Board) established the Office of Evaluation, Performance, and Analytics (OEPA) to serve an enterprise-wide function providing leadership, coordination, and capacity building to oversee County efforts around evidence-based policymaking, program evaluation, data sharing, collection, and metrics. Today’s item provides an update on OEPA’s efforts, highlighting its strategic approach to increase the use of data to inform decisions at the County. In addition, this item requests approval from the Board to award a contract or contracts for research and technical assistance services from universities.
RECOMMENDATION(S)
CHIEF ADMINISTRATIVE OFFICER
1. Receive presentation: Update on Office of Evaluation, Performance, and Analytics.
2. Waive Board Policy A-87, Competitive Procurement, and in accordance with Administrative Code Section 401, et. seq., of the County Administrative Code, authorize the Director, Department of Purchasing and Contracting, upon successful negotiations and determination of a fair and reasonable price, to award contract(s) with universities for Research and Technical Assistance for up to one (1) year, with up to two (2) one-year option periods , and an additional six (6) months if needed, and to amend the contracts as needed to reflect changes to requirements and funding, subject to the approval of the Director of OEPA.
EQUITY IMPACT STATEMENT
Equity is a priority area in the County of San Diego’s (County) Annual Strategic Research Plan, which guides OEPA’s research and evaluation work. Contracting with universities to receive research and technical assistance will significantly increase OEPA’s capacity to provide answers to the most important questions around how to improve equity in the policy areas of highest priority to the County.
SUSTAINABILITY IMPACT STATEMENT
Sustainability is a priority area in the County of San Diego’s Annual Strategic Research Plan. Contracting with universities to receive research and technical assistance will increase OEPA’s capacity to provide answers to the most important questions around sustainability and the environment.
FISCAL IMPACT
If approved, today’s action will result in costs of up to $300,000 in Fiscal Year 2023-24. Funds for this request are included in the Fiscal Year 2023-24 Operational Plan in the Chief Administrative Office, Office of Evaluation, Performance, and Analytics. The funding source is General Purpose Revenue. Future costs and revenue of up to $300,000 per year will be included in future Operational Plans and contract option years will be exercised upon availability of funding. There will be no change in net General Fund cost and no additional staff years.
BUSINESS IMPACT STATEMENT
Increasing business compliance with employment laws and regulations and reducing wage theft are among the priority questions in the Annual Strategic Research Plan. Contracting with universities to receive research and technical assistance will increase OEPA’s capacity to address these questions.
Details
ADVISORY BOARD STATEMENT
N/A
BACKGROUND
The pace and scope of challenges experienced by governments are rapidly changing. With an $8.2 billion budget, more than 20,000 employees, and hundreds of programs serving a population of 3.3 million residents, County of San Diego (County) leaders regularly need to make decisions on policy areas for which our constituents demand solutions that show tangible results, such as homelessness, housing affordability, justice system reform, equity, and many others.
When used properly, data can be an invaluable resource for decision-makers, giving them clear and objective evidence on which programs and policies work, for whom they work, and in what places and communities. On May 18, 2021 (16) the Board of Supervisors (Board) directed the Chief Administrative Officer to establish the Office of Evaluation, Performance, and Analytics (OEPA), to serve an enterprise-wide function providing leadership, coordination, and capacity building to oversee County efforts around evidence-based policymaking, program evaluation, data sharing, collection, and metrics.
To fulfill this direction, OEPA is implementing a strategic framework to increase the use of data to support decision-making. The framework involves three core strategies, described below.
Developing a Structured Approach to Use Data to Inform Decisions
This strategy seeks to ensure we prioritize studies that will support important decisions. In order to achieve this, OEPA must be deliberate in analyzing the information needs of County leadership. The first element of this strategy is the Annual Strategic Research Plan (ASRP), submitted to the Board in July 2023, which aligns our research and analytical work to County strategic priorities. The ASRP was developed through a collaborative process to ensure it reflects the priorities of multiple County stakeholders.
Second, in collaboration with its consultant, Impact Paradigm Associates (IPA), OEPA is building a program inventory that will result in a comprehensive list, across policy areas, of all programs throughout the enterprise. Moreover, for each program in the inventory, OEPA will research whether previous studies have found evidence that it works or not, which will allow us to prioritize the evaluation of those that have insufficient evidence.
Finally, OEPA is working with senior County staff across all four business groups to develop the Strategic Performance Framework, which will create metrics for the County’s Strategic Plan. These metrics will provide transparency regarding the County’s progress in achieving its strategic goals and will help prioritize analyses on areas with unsatisfactory progress.
Getting the Most Value Out of Data
In the conduct of their regular activities, governments collect a substantial amount of information. Because this data is collected for the operational needs of individual departments, it is often stored in ways that make it difficult to analyze. As evidenced by efforts like the Alternatives to Incarceration initiative, County departments are increasingly seeking ways to collaborate on collecting and sharing data. Properly used, administrative data can provide invaluable insights into the needs of the San Diego region and the impacts of County policies and programs on addressing those needs.
In partnership with the County Technology Office and the County’s Information Technology provider, OEPA is building the Enterprise Integrated Data Platform (EIDP), a state-of-the-art cloud-based system that will connect to and extract data from many of the County’s most important data systems and store it securely. Moreover, the EIDP will build and continuously update a master person index, enabling the linkage of individual-level data across different source systems. This will allow the County to leverage the value of the data and use it to benefit the community by improving County programs and policies to better address the needs of our region.
Innovations in technology and use of data should be supported by data management practices that are ethical and comply with data privacy, confidentiality, and security regulations. The County’s Chief Data Officer is leading an effort to modernize the County’s approach to data governance, which will establish a structure with clear and transparent data sharing rules and policies, ensuring we comply with regulations and ethical principles as we use data to benefit our community.
Increasing Capacity to Use Data to Make Decisions
The ASRP established an agenda for the use of data to inform County decision-making. Although the two strategies described above will allow OEPA and County departments to focus on high-value projects and become more efficient in using data, there is considerable work to be done. Moreover, as the last few years have shown, new challenges like public health emergencies, natural disasters, or policy changes can emerge at any time. To continue making progress, it is essential to increase the County’s capacity-staff skills, adoption of modern data analysis tools, etc.-to use data to inform decision-making. The County has a long and successful history of using data for operational planning and improvement; thus, OEPA must ensure that our capacity-building efforts focus on areas of real need. In collaboration with IPA, OEPA developed a survey to assess the County’s current capacity and needs around program evaluation, performance measurement, and other types of data analysis. IPA is analyzing responses and will summarize the findings in a report that will inform OEPA’s capacity building efforts.
The importance of building in-house capacity for program evaluation to foster a culture of learning was a major point of emphasis in the establishment of OEPA. This makes the County special among its peers. Most local governments rely on external parties for evaluation studies, which can be costly and inefficient, as separate contracts are usually needed for each evaluation, and some of the knowledge acquired by the evaluator during a study does not remain in the government organization upon completion of the study. OEPA developed a curriculum to train County staff on core evaluation concepts and the value of evaluation. This training will help build staff knowledge to engage as partners more actively in the evaluation process, support evaluation projects, and foster a shared language within the County. By the end of May 2024, OEPA will have trained over 120 staff from the County’s Land Use and Environment Group and the Health and Human Services Agency. Over the next several months, OEPA will proceed to train staff from the Public Safety and Finance and General Government Groups.
Regardless of the County’s success in building its capacity to use data, occasionally, access to external technical or subject matter expertise will be needed. OEPA is already benefitting from collaborations in current projects with external partners such as the Massachusetts Institute of Technology, the U.S. Office of Evaluation Sciences, and Casey Family Programs. Other departments have also developed strong and long-lasting partnerships with external researchers and evaluators. OEPA seeks to continue fostering these collaborations and to establish new ones to ensure its work provides the best possible answers to the questions County decision-makers and stakeholders have about the effectiveness and efficiency of County programs and policies.
Justification for Contracting with Universities for Research and Technical Assistance
The ASRP establishes an ambitious multi-year agenda for research and evaluation that will support the County’s strategic priorities. Even with OEPA’s depth of expertise and capacity-building efforts, it would be extremely difficult for the County alone to tackle the most complex topics in the ASRP.
Universities have unique depth of knowledge and skills on a broad range of policy areas covered by the ASRP, such as climate action, infrastructure, early care and education, homelessness, employment regulations, social programs, regulatory programs, employee recruitment and retention, child abuse and neglect, and justice system reform. Their contributions to support the County could include program evaluation, data analysis, survey research, studying best practices, community-based research, and technical assistance. Because the strength of expertise on specific technical and subject matter areas varies across universities (that is, one of them may be the strongest on homelessness but not on the justice system or the environment), agreements with several of them may be needed. As such, OEPA seeks a waiver of Board Policy A-87 and authority to pursue agreements with universities for research and technical assistance support.
LINKAGE TO THE COUNTY OF SAN DIEGO STRATEGIC PLAN
The requested action supports the County of San Diego’s 2024-2029 Strategic Plan Initiative of Empower through Innovation, Transparency, and Accountability, as it ensures we can properly maintain program and fiscal integrity through evaluation while fostering innovation and leveraging best practices that promote continuous improvement.
Respectfully submitted,
SARAH E. AGHASSI
Interim Chief Administrative Officer
ATTACHMENT
List of OEPA Enterprise-Wide Initiatives and Current/Potential Projects