Baked sweets

TRCs and Sugar Cookies – a Holiday Review of Cost-effectiveness Testing Practices

by Alaina Boyle

As a home baker, I look forward to the holidays for the opportunity to dust off my cookie recipes — and to enjoy quality time with family and friends. Every December I comb through my recipe box to select the year’s favorites. To find the perfect mix of cookies to share with my family and friends, I like to refine the ingredients and directions in old standards while testing new recipes. Recently – bear with me on this – I’ve noticed that the Database of State Efficiency Screening Practices (DSESP) is a lot like my recipe box.

Here’s what I mean.
The National Efficiency Screening Project (NESP) team spent much of 2019 collecting the “recipes” for cost-effectiveness (CE) testing practices across all 50 states plus two jurisdictions. Key components of these recipes include the state’s underlying policies (like a recipe’s step-by-step directions, policies describe how a state should conduct its CE test) and the energy efficiency program impacts included in each state’s test (i.e., the recipe’s ingredients).

Similar to a baker’s signature gingerbread recipe, the combination of ingredients and directions for every state’s CE test is typically modified from the test’s theoretical definition such that it’s unique to each state. This is the case with the commonly used Total Resource Cost (TRC) test. The DSESP shares details of these CE test “recipes” including which impacts states account for, specific test assumptions, and underlying policies.

The DSESP serves as a portal to review states’ “recipes” as well as various guidance resources and can help inform interested states which ingredients are being used across the country, and in what quantities. One new feature of the DSESP is that it identifies which states have or are applying the National Standard Practice Manual (NSPM) to review and update their current practice relative to the NSPM’s economically-sound and policy neutral CE testing principles.

A core principle is to ensure that a state’s CE testing practices reflect the intent of the state’s applicable policy goals. For example, if a state identifies economic development as a goal for investing ratepayer funds in efficiency programs, then those impacts should be included in the state’s CE test. From a baker’s standpoint, this is like making sugar cookies – if one’s intent is to bake a sugar cookie, then sugar should be in the recipe!

The NESP team developed the DSESP to provide a repository of central, standardized, and accessible state CE testing information in response to such CE stakeholder needs.

What we Baked in 2019
The DSESP team – including Synapse Energy Economics and E4TheFuture staff – was busy in the kitchen in 2019. To complement the full population of the database with all US jurisdictions, we developed state-specific CE overview sheets released over the summer. These downloadable summaries of current state practices provide a quick glance at a state’s practice, similar to simplifying state’s recipes down to a visual grocery list without measurements or directions.

In July, we learned from ACEEE how states write their “recipes” and how the DSESP can inform updates to these practices in this DSESP topic brief. ACEEE based the brief on its analysis of state practices using the database and used DSESP’s new interactive map visualizations to show how state practices compare across utility and non-utility system impacts. They specifically examined whether and how wholesale price suppression effects and participant productivity are included in state tests.

United States map showing how states calculate productivity effectsACEEE is now sifting through state recipes to better understand how states account for key utility system impacts (the “Three Rs”: Risk, Reliability, and Resilience). In early 2020, ACEEE will publish a topic brief that reviews which states account for these impacts — and how.

Feedback from database users showed that the Three Rs are a key area of consideration as CE testing moves into 2020 and beyond; according to the NSPM, every CE analysis should account for the full range of utility system impacts as the foundation of CE tests.

Both ACEEE briefs were informed by DSESP user surveys in 2019. The DSESP team also solicited feedback via direct communication, and we plan to continue meeting user needs with updates in 2020. Improvements to date include:

  • highlighting states applying the NSPM framework
  • clarifying data and references
  • creating new interactive tabs providing visual representations of practices across states and by impact

Since fully populating the DSESP this fall and conducting QC to ensure the recipe book page numbers lead users to the right recipes, we’ve also added new guidance documents, such as the EPA’s Public Health Benefits per kWh of EE and Renewable Energy report, ACEEE’s Guidance on Measuring the Economic Development Benefits of EE, and papers from RAP and NARUC on distributed energy resource valuation.

The new DSESP version featuring these updates is now available on the NESP website.

What We’re Mixing Up for 2020

When searching for new recipes online, a helpful indicator of deliciousness is the number of people who have used the recipe. The DSESP is the only database of its kind and has been downloaded over 600 times since its launch in late 2018, by over 435 unique users. We’ve also heard through our recent DSESP survey that users appreciate this resource.
Quote survey respondent DSESP

To keep the DSESP valuable for users, we have begun working toward user-informed updates to be released 2020. What will comprise the 2020 updates? From valuable user feedback (thank you to those of you who shared your thoughts!), we determined key priorities. We’re putting on our aprons and chef’s hats to fold in navigation improvements including:

  • better definitions
  • updated data sorting and presentation
  • an expanded scope to address key questions on inputs, governance, and more

Throughout, accuracy remains a top priority; we will ensure that the state CE recipes in our recipe box – the ingredients, directions and measurements – are reflective of actual current practice.

Making Your Own Favorite Recipes
Just as I make my youngest brother’s favorite cookie (Granny’s cardamom hearts) using a completely different recipe from my own favorite (peanut butter blossoms), states can apply the NSPM to ensure their CE test will meet the policy context of their own jurisdiction (see these case studies of how states have applied the NSPM). The DSESP shares state recipes alongside resources for key stakeholders to consider and refine CE test ingredients and directions, updating their inputs to better match the desired flavors of the state.

We hope to support regulators, program administrators, evaluators, and other interested parties in their efforts to improve CE testing practices by:

  • making data available to support data needs
  • identifying states that are increasingly turning to the NSPM as a framework for refining and improving their practice
  • pointing to helpful guidance resources to inform ways to quantify impacts

I am excited to further improve the database and add more resources to better support DSESP users in the new year. Until then, may you have a joyful holiday and partake of your favorite baked goods to your heart’s content.

–Alaina Boyle is Research Associate, Clean Energy Valuation at E4TheFuture.