The American Rescue Plan (ARP), the Bipartisan Infrastructure Law (BIL), the Inflation and Reduction Act (IRA), and the CHIPS and Science Act provide the U.S. Department of Commerce with unprecedented new resources that can be leveraged to advance the mission of creating the conditions for economic growth and opportunity for all communities.

To ensure these programs are delivered as effectively and equitably as possible, data practices play a critical role.

Improving program data practices is one of many aims of the Commerce Data Governance Board (CDGB). To support bureaus in their evidence-building practices, the CDGB chartered the Metrics Working Group in August 2022.

The Metrics Working Group was specifically set up to uncover obstacles and explore pathways for improving the use of data for program design, delivery, and evaluation.

It was also charged with proposing recommendations for facilitating large-scale observational studies for evaluating outcomes from DOC programs that have similar intended benefits.

Observational studies enable DOC decision-makers to understand the effectiveness of specific policy approaches intended to drive common outcomes, such as local job growth, business revenue, business formation, employment trends, educational attainment, and ecosystem habitat restoration.

In this blog post, we share more about this effort, as well as discuss major insights from the Metrics Working Group final report, “Innovation in Evidence-Building: Evolving Program Data Management Practices within the Department of Commerce.”

Breaking Down Silos: How the Metrics Working Group Coordinated Across Bureaus

To uncover obstacles and explore pathways together, the Metrics Working Group convened representatives from the following four bureaus that received new funding resources:

•  Economic Development Administration (EDA)
•  Minority Business Development Agency (MBDA)
•  National Oceanic and Atmospheric Administration (NOAA)
•  National Telecommunications and Information Administration (NTIA)

The Metrics Working Group also included representatives from other departments that support monitoring and evaluation efforts of these bureaus, such as those from the DOC Performance Excellence Office, the DOC Evaluation Officer, and the Commerce Implementation Coordination Office (CIC) within the DOC Office of the Secretary.

The group also conducted interviews with various stakeholders throughout the DOC, including program evaluation leads, program data managers, grants acquisition experts, GIS specialists, equity indicator experts, and economic projection experts.

Identifying Best Practices: How the Metrics Working Group Surfaced and Solved Challenges Together

The Metrics Working Group surfaced two program models that illustrate innovative approaches to evidence-based program delivery. Program models include a pathway for tracking outcomes at the individual-level (such as EDA’s Good Jobs Challenge) and a pathway for tracking outcomes at the community-level (such as NTIA’s- Broadband Equity, Access, and Deployment Program).

EDA’s Good Jobs Challenge program is particularly groundbreaking for the way data is collected, linked, and analyzed for measuring outcomes for participants. The $500 million program funded by the American Rescue Plan is designed to improve labor market outcomes for unemployed and underemployed Americans by providing federal funding for 32 local workforce development programs across the country.

Some of the evidence-building best practices adopted in this program include:

Collect high-quality data for tracking equitable program outcomes:

• The program validates core concepts such as a “good-paying job”.
• The program identifies specific underserved populations as intended beneficiaries.

Streamline data collection processes based on the needs of program stakeholders:

• The program was designed with multiple program stakeholders in mind, using online survey tools to improve data quality and reduce administrative burden and time lag.

​​​​​​​Link program data with statistical data for analyzing program outcomes:

• The program enables appropriate access to program data for operational insights and evaluation purposes by multiple parties, including with the U.S. Census Bureau’s Data Linkage Infrastructure and research partners.
• The program includes a “counterfactual” approach to evaluation, comparing outcomes of Good Jobs Challenge participants to the outcomes of comparable workers in similar labor market conditions through linkage with statistical data sets.

Linking program data with statistical data enables researchers to learn more about long-term program outcomes, such as earnings, business revenue, and employment. The EDA Good Jobs Challenge is particularly noteworthy for its use of this innovative research approach.

These are just some of the best practices identified in the Metrics Working Group final report. Read more here.

Moving forward, the Commerce Data Governance Board, and related teams are working to support bureaus in scaling next-generation evidence-building practices.

At the Department of Commerce, our duty is to ensure that we continuously improve the effectiveness of the federal programs we administer to meet the needs of the communities we serve. Data practices are a key enabler for effective program implementation. We’re excited to continue this work at the Department of Commerce in the year ahead.

This post is authored by Barbara Downs, Evaluation Officer, U.S. Census Bureau; Nancy Ritchey Archive Branch Chief, National Oceanic and Atmospheric Administration; and Shannon Arvizu, Senior Advisor to the Department of Commerce Chief Data Officer.