Fuzzy Math

Earlier this year, I sat on a panel with Tom Eckman, a long-time efficiency expert who helped establish the use of “deemed savings” in four Northwestern states.

The topic of the session was energy efficiency measurement — also known as EM&V (evaluation, measurement and verification).

Eckman’s presentation was focused on the history of EM&V and the emerging technologies that are transforming this important sector. Toward the end of his presentation, Eckman showed a slide with a powerful message. The slide asked: “Why deem it when you can measure it?”

This might seem like common sense. But it was a bold statement coming from an industry leader who played an important role in ushering in the widespread use of deemed savings.

What are deemed savings and why do they matter?

Deemed savings are a set of predetermined savings values for efficiency measures that are developed from commonly accepted data sources and analytical methods, and maintained by states or regional bodies.

These savings estimates are called “deemed savings values” and provide a savings estimate for each change made to a home or building. Deemed savings can also take the form of an algorithm, providing a formula of inputs for arriving at a savings estimate. Deemed savings have a multitude of benefits; they are fast, easy to calculate, constant and predictable. Because of those traits, deemed values are commonly used to estimate energy savings from efficiency programs.

But deemed savings values rely on multiple assumptions and can vary widely from the actual measured savings. In fact, a recent study stated that “static efficiency assumptions are inherently imprecise.”

The quality of deemed savings varies. While it’s important to update deemed savings values regularly with data from studies, those efforts are costly, time-consuming, and do not always happen. As a result, deemed savings provide questionable value when the primary research is conducted in another state, or if it becomes outdated or is not thoroughly vetted.

At a time when energy efficiency is being relied upon to replace aging power plants, reduce greenhouse gas emissions, and perform as a capacity resource on the grid, it is more important than ever that we apply sufficient rigor to how we measure energy-efficiency savings. This is also critical because savings measurement is a common metric for rewarding utilities and ensuring ratepayer dollars are spent prudently on energy efficiency.

Common flaws with deemed savings

Industry experts regularly critique deemed savings, and regulators experienced in energy-efficiency oversight are well aware of the issue.

“Deemed savings allow regulators and utilities to more easily measure and oversee energy efficiency programs but we need to be aware that using deemed savings trades accuracy and understanding the persistence of savings for ease,” said Dian Grueneich, a former California utilities commissioner.

Recently, the issue has been getting more attention.

As part of a Virginia State Corporation Commission (SCC) proceeding on standardizing EM&V, the SCC staff wrote a report on deemed savings values used in some Mid-Atlantic states. The investigation found the deemed savings used in the region were based on outdated information: “The vintage of the data supporting these assumptions dates from the early 2000s to as far in the past as 1986 in one case,” reads the report.

Staff also found that the deemed savings were based on studies conducted in places as far away as Illinois, California and Ontario, Canada — all states with questionable comparability to Virginia.

The commission staff dug into one particular deemed savings algorithm for a low-flow shower head. According to the report, the algorithm includes “a measurement of gallons per day per person for showering.” The gallons-per-day assumption is based on an EPA document that assumes 11.6 gallons per day per person for showering.

However, the commission staff reviewed the source document and reported that the report itself was based on pooled results, presented “for summary and comparative purposes alone.”

The SCC is not alone in noticing the issues with deemed savings. QuadROI, a business intelligence firm that aggregates public data on energy efficiency, recently presented a thorough study on the collection of deemed savings values and algorithms used in various states. The paper found that input values for similar savings estimates varied widely.

QuadROI analyzed faucet aerators with the hope that these devices were simple and consistent enough to provide the same result from state to state, regardless of weather, climate or other factors that might vary. However, it found that the expected lifespan of the faucet aerators varied from anywhere between five and 10 years.

In Pennsylvania, the expected life of a faucet aerator was 10 years. Just across the state border in Maryland, which uses Mid-Atlantic deemed savings values, the same faucet aerator is expected to burn out in five years.

These are not trivial differences. The lifespan of a device is an input that greatly impacts the expected savings, since the amount of years the aerator lasts will determine how many years the savings will persist. QuadROI’s paper concludes that without rigorous research into deemed savings values, there is a “risk that reported program results deviate from reality.”

Improving deemed savings and moving forward

Deemed savings were created to provide a savings estimation approach that was simple, cost-effective and replicable for regulatory compliance. Despite the flaws, deemed savings have value for planning purposes. Furthermore, certain types of energy efficiency programs, like point-of-sale rebates, would be impossible without using deemed savings as substitutes for actual measurement.

However, as an industry, we need to strive for precision in quantifying energy savings. Deemed savings should be used for EM&V only when necessary. We should employ more rigorous EM&V approaches wherever possible.

Put simply, Eckman’s statement — “Why deem it when we can measure it?” — should be an important guiding approach to EM&V. It is well timed with modern analytics solutions emerging in the industry that are reducing the cost, time and effort associated with measuring actual savings from the meter.

Modern measurement solutions are arriving just in time. Energy efficiency is being increasingly deployed to deliver reliable demand-side reductions. If efficiency is going to be considered a resource truly on par with other sources of generation, we will need the savings estimates to be factual and reflective of what is happening at the meter.

By: Jake Oster, Sr. Director of Regulatory Affairs

Originally published in Greentech Media