How to Make Heads or Tails of a Research Study on Methane Leakage

While researching and writing Natural Gas and Global Warming: A Review of the Evidence Finds that Methane Leaks Undercut the Climate Benefits of Gas, I read stacks of research papers on the climate impacts of natural gas. Parsing these academic studies is an acquired skill and I thought I’d share some of what I learned about how to read methane research for anybody who is just diving into this topic.

Different research papers on the climate impacts of methane draw very different conclusions. By asking the right questions about how each study was conducted, it is possible to partially explain some of the variation in the results of these studies and to understand how they relate to each other.

Here are some of the key questions to ask about each research paper.

Does the paper rely on a “top down” or a “bottom up” analysis?

There are two main ways to estimate the amount of methane that leaks from oil and gas production, storage and transportation. “Top-down” studies use measurements of the amount of methane in a region, obtained by flying a specially equipped airplane or driving around in a van full of measuring equipment while taking air samples. Atmospheric concentrations of methane can also be obtained through satellite observations or stationary monitoring facilities. This approach can measure methane from all known and unknown sources, but also might be unduly influenced by local sources of pollution or weather patterns.

“Bottom-up” analyses add together the measured or reported emissions from each step of natural gas production: how much methane leaks when a well is fracked and when it is in routine operation; how much methane is leaked per mile of pipeline or by each valve; how much methane escapes from different storage tanks, and so forth. Researchers add together the leakage from each step to arrive at an estimate of total leakage. This can miss unexpected sources of pollution.

For bottom-up analyses, how does the study address the issue of “super-emitters”?

One of the weaknesses of bottom-up calculations is that just a few faulty components in a natural gas system have the potential to leak huge amounts of methane. If researchers measure the hatches on 50 storage tanks, for example, to calculate an average emissions rate for hatches, they might miss the impact of the one valve in a thousand hundred that is failing. “Super-emitters,” the random pieces of failing equipment here and there, may dwarf the impact of other sources. An analysis conducted for the EPA “measured ~75,000 components and found that 58% of emissions came from 0.06% of possible sources” (as cited here). A bottom-up study that doesn’t discuss the problem of super-emitters might be missing a large source of emissions.

What parts of the natural gas system are being studied?

Natural gas production, storage and transportation involves lots of steps, which are typically divided into upstream processes and downstream processes. A study might look at both upstream and downstream emissions, or just upstream or downstream, or even a smaller portion. Further complicating matters, not all researchers agree on what counts as upstream or downstream. Typically, though, emissions at the well site and from processing natural gas are considered upstream, while storage, trans­mission and distribution are downstream.

What universe of wells was studied?

Two studies using the same method for collecting and measuring methane emissions might nonetheless produce very different results if they examined different wells, regions, or time periods.

Cooperative research efforts between academics and well operators enable researchers to access well sites and all the infrastructure involved in producing oil and gas. However, if industry selected which wells researchers could study, maybe they picked the wells that were likely to have the lowest emissions. It isn’t possible to know, but it may be a source of bias in the analysis.

Oil and gas industry practices may differ from one region to another, some formations may contain more unexpected pockets of methane, and oil fields and gas fields may have different methane leakage rates. For example, in an area without natural gas pipeline infrastructure, well operators have no way to get natural gas to market and have less incentive to capture it. Or if a well has been drilled to produce oil instead of gas, the owner may not care as much what happens to the methane. All these factors may contribute to regional variation in emissions.

Emissions may change over time, as well operators become more or less careful about capturing fugitive methane emissions and as drilling practices change. For example, if the price of natural gas is lower, producers have less incentive to invest in equipment to capture methane to deliver to market. A decade ago, fracking was relatively rare. Any analysis based on emission rates from the early 2000s isn’t necessarily relevant to understanding methane leakage today.

How much gas did the researchers assume wells would produce during all their years of operation?

The most methane is released during the first year a well is in operation, as it is drilled, perhaps fracked, and brought into production. Once the well is producing, daily emissions drop dramatically, despite the potential for leaks from storage tanks, pipelines and other sources that are now full of natural gas. If researchers assume that a well will produce natural gas for decades, the initial cloud of emissions is a relatively small portion of the total gas produced from the well. But if the well is going to produce gas for just a few years, leaked methane becomes a much bigger percentage of the produced methane. And that means the useful gas produced from that well has a bigger warming impact.

What number is used for the global warming potential of methane, and over what time period?

As scientists have refined their understanding of methane, they have increased their estimate of how harmful it is for the climate. The Intergovernmental Panel on Climate Change’s Fourth Assessment Report, released in 2007, calculated that a pound of methane had 25 times the global warming impact of a pound of carbon dioxide over 100 years. In the Fifth Assessment Report, released in 2013, the IPCC raised methane’s 100-year global warming potential to 34. Despite this, a number of emission inventories, emission calculators, and research studies still base their calculations on a global warming potential of 25, resulting in underestimates of the impacts of leaked methane.

Another major variation that appears in research papers is whether methane’s impact is assessed on a 100-year basis or a 20-year basis. Methane is removed from the atmosphere relatively quickly, meaning that a pound of methane has a greater impact in the short run than the long run. Over 20 years, methane causes 86 times more warming than does a pound of carbon dioxide, versus 34 times over 100 years. The choice by researchers to use a 100-year timeframe or a 20-year timeframe is neither right nor wrong; it merely prioritizes concern about long-term warming or worries about short-term tipping points.

If the methane impacts are presented with some “real-life” points of reference, how were those calculated?

Lots of assumptions are behind comparisons of methane emissions to other sources of global warming pollution, such as the assumed global warming potential of methane. Another example is if natural gas is compared on a lifecycle basis to coal burned for electricity, what’s the assumed efficiency of the coal power plant and the natural gas power plant? Nobody is building new coal-fired power plants, so the best reference point for coal plants is the average efficiency of plants currently in operation, not the most efficient new plant that could be built. For natural gas plants, choosing the efficiency of new gas plants implies that natural gas is enabling construction of new natural gas electricity capacity. That’s not an unreasonable assumption. (Though if the discussion is about adding capacity, then we should also be comparing emissions to the global warming impact of renewable energy.)

Not all natural gas is used to produce electricity. It is also used to heat water and buildings, to power industrial processes, and to drive vehicles. All those points of comparison involve assumptions and judgments, too.

(Note that if you are trying to calculate a real-life comparison about the impact of leaked methane based on a study you’ve read, be sure to compare unburned methane instead of combusted methane. EPA has calculators for methane leaks and for combustion. Note that the EPA leakage calculator uses an outdated 100-year global warming potential for methane of 25. You can fix that with a little math.)

And finally, what the heck are these units of measurement? I think I’ve encountered billion cubic feet, thousand cubic feet, therms, British thermal units, teragrams, and metric tons.

Some abbreviations you’ll see include:

• Bcf = billion cubic feet
• Mcf = thousand cubic feet (though I read one study that said this was million cubic feet, despite their sources and their math all treating this as thousand cubic feet)
• MMcf = million cubic feet
• Tg = teragrams
• Btu = British thermal unit
• BOE = barrels of oil equivalent, a measure to evaluate leaked methane in comparison to the useful energy obtained from the oil and gas wells that are the source of the methane leaks.