Transatlantic Policy Challenges of the Digital Energy Nexus

July 08, 2018
Photo Credit: Gyn9037 / Shutterstock

Photo Credit: Gyn9037 / Shutterstock

Digitalization has begun to transform the energy sector in both Europe and the United States, with all the economic, social, political, and even geo-political consequences that disruption entails.

This transformation is far from complete. At the macro level, digitalization is spurring the electrification of our energy sectors, and digitalization will help us integrate renewable sources of energy into the system on large and small scales. At the micro-level, the digital connectivity underlying the “Internet of Things” makes every electric device a contributor to the energy system (rather than just an electricity consumer) by bundling “distributed energy resources” into larger eco-systems. Digitalization is transforming the one-way electricity supply of yesterday into an omni-directional system for tomorrow.

Such changes open opportunities for new entrants — including those who used to be simple consumers — to provide new energy services to the system. This challenges the once staid business models of traditional electricity suppliers, and consequently will transform the ways Europe and the United States have regulated energy utilities. This overhaul of the energy system raises new policy issues that have not yet been fully addressed in either the United States or Europe, including on the ownership and use of the mountains of energy data now being generated, on privacy protection, on new infrastructure needs and new cybersecurity vulnerabilities.

This report attempts to provide insights into the state of the digital transformation of the energy sectors in both the United States and Europe. We analyze the implications of those changes to the sector and identify some key drivers behind the transformation, including climate concerns, efficiency, competitiveness and resilience, the weights of which sometimes differ on the two sides of the Atlantic. Finally, we look at how both sides are addressing the data, privacy, infrastructure and cybersecurity challenges digitalization is generating before suggesting a few recommendations for industry and policymakers to consider.

The State of the Transformation

Digitalization and the electrification of our societies go hand in hand, feeding on each other and together driving the transformation of the energy sector. In advanced economies like the European Union and the United States, electricity is now almost a quarter of all energy consumed, up from just over 10 percent in 1973. This electricity share is set to expand further as the Internet of Things grows and as transport — which now guzzles gas and diesel — is increasingly electrified. But even as the electricity sector grows in importance, it is changing at the same time. The traditional electricity supply model — where electricity from large scale power plants is fed through transmission wires and distribution systems to passive commercial, industrial, and residential consumers — is transforming into a more active and distributed model.

A critical part of this change is the burgeoning of distributed renewable energy power — solar and wind power generated at the home, office building, and factory. The spread of renewable solar and wind power in both Europe and the United States, however, while associated with the digital transformation, does not define it. In the case of both solar and wind, Europe leads the United States by nearly two times. Solar provides 3.7 percent of Europe’s electricity while wind kicks in 11.8 percent; in the United States, those numbers are 1.5 percent and 6 percent respectively. A considerable amount of this renewable power is generated by utility-scale wind-farms and solar installations that fit nicely into the traditional energy supply model, even though they are renewables. At the same time, however, government supports and expanded take-up have dramatically driven down the prices of wind turbines and solar panels, even as their technology and quality improves. As a result, more distributed electricity production from commercial/industrial as well as residential sites for solar and wind generation (including micro-wind turbines), is growing quickly in the United States, while installations in Europe have slowed from their former highs.

The expansion of renewables technologies has also highlighted problems associated with them: their variability (always understood as a problem); the mismatch between their time of production and when their energy is needed; their ability to overwhelm the local grid, leading to major curtailments. Further, there is the issue of their zero marginal cost, putting them first in line to supply mid-market and peak demand, which drives down wholesale prices and undermines the economics both of renewables and of traditional power sources. One example of the existing inefficiencies is the case of wind turbines in Europe, which already have the capacity to be the second largest source of electricity, but are active only a third of the time.

Into this system comes the battery and energy storage. This in some ways had been neglected in the push to renewables, but it critically smooths renewables electricity supply and demand across time, potentially putting variable wind and solar power on the same level as “dispatchable” nuclear, coal, and gas. This vastly improves their economics. While installed storage capacity in Europe and the United States is still low (2.6 gigawatts in Europe and 1.3 gigawatts in the United States), battery prices have declined dramatically (in part because of expanding electric-vehicle demand), and residential and other distributed demand is expanding rapidly. The last element of the battery transformation is the electric vehicle, which could both be a drain on the grid and an enormous storage asset to it.

In this new world of disbursed renewables generation and storage technology, digitalization is the new and crucial glue. Digitalization emerges first in the form of smart meters, but secondly and more importantly, as platforms that bring together and optimize these “distributed energy resources.” Second generation smart meters, which can both provide granular information on usage and communicate wirelessly with devices in the factory, commercial establishment, and home, now cover over half of America’s consumers, while the number is closer to 40 percent in Europe. Digitalization has introduced new actors into the energy system. Companies new to the electricity sector, but with deep experience in information technology, are applying high-powered computer analytics to data from smart meters as well as from thermostats and other smart controls to help consumers reduce and manage their demand and to sell self-generated electricity back to the system. Some platforms allow individual consumers to come together as networks through which small contributions to supply and demand can be aggregated into a significant force on the market

These new networks bring synergies to and magnify the effects of distributed energy production and storage. They represent, in effect, highly local mini-electric companies, and are sprouting most notably in the United States while large-scale pilots are taking off in Europe. While these new models are difficult to count, there are approximately 2,000 known larger scale “microgrids” (which can actually be disconnected from the main power system), with the United States leading Europe at 7 gigawatts capacity versus 1.8 gigawatts.

Impact on Business Models

Electric companies, especially at the distribution level, could (and perhaps should) provide the efficiencies that come from bringing distributed energy resources together — the companies have the contacts to the customers, often own the smart meters, and are increasingly deploying distributed energy resource management systems. But they are often hampered by regulatory barriers which restrict their ability to generate revenues from new services. And indeed, in some places in both the United States and Europe the companies that bring customers electricity over their distribution networks are explicitly enjoined from producing electricity, even from batteries.

Electric utilities are deeply concerned that “over the top” digital energy service companies, could woo customers off grid and allow them to avoid costs associated with maintaining and strengthening that infrastructure. Many are trying to combat this risk to their business in part by trying to limit the outside competition, or by ensuring it has to face the same regulatory burdens they do. Others are trying to beat the competition by joining in, either acquiring or teaming up with the new up-start electricity service companies.

Drivers of the Transformation

 While this transformation is happening in both Europe and the United States at the same time (which says something about the rate of technology diffusion, despite regulatory barriers), it is occurring largely for different reasons. In the United States, the chief drivers are profit motivation, as IT companies see a new market to bring their disruptive models, and resilience, as microgrids and other such arrangements provide backstops to potential grid failures associated with an aging above-ground electricity transmission and distribution network. In Europe, in contrast, the chief driver is the desire to combat climate change, an important policy goal, but one that does not always fit with economics.

Policy Issues Raised by Energy Digitization

The digital transformation of the energy sector raises numerous policy issues, including how the new market, where utility and new entrant meet, should be regulated. But it also raises issues that are outside the “normal” energy regulatory remit.

One of the first, and most consequential, is ownership of and access to the data that increasingly underlies the industry, as bytes become more valuable than electrons. In both Europe and the United States, this may be easier to handle in the commercial and industrial client base, where contracts that define ownership/usage rights are an accepted norm. At the residential (and small business) level, however, data ownership and access becomes more complicated. And this can become an especially difficult issue where the energy supply company has traditionally held the power over data related to energy usage. The consumer, who is increasingly also a producer, may dispute the energy supply company’s role and want to provide his/her data to whichever service supplier can offer the best deal. On both sides of the Atlantic, regulators are trying to find ways to ensure that the market is open. On both sides of the Atlantic as well, companies will invest substantially to win the right to eke out the efficiency benefits that the right access to data brings.

A related and critical policy issue is data protection. Smart meters alone can read deeply into what is going on in a household by its energy usage — whether bread being toasted in the morning or tea brewed at night (and according to some reports, the television channel being watched). This will become even more granular as devices in the home — the hot water heater, the washing machine, the heating and cooling system, even lights — are “smartened.” Consumer and privacy rights groups are concerned about how this data can and will be used. But “opt-in” approaches to guard privacy may lead to fewer consumers engaging in smart energy management, which would hamper a society’s ability to reduce greenhouse gas emissions. This “public purpose” is not yet sufficiently strong to overcome privacy concerns, but the tension needs to be addressed.

The need to build out and maintain an ever-smarter electricity infrastructure is running into economic problems. Electricity demand is mostly stagnant and could weaken further as additional efficiency is squeezed out of the system and as renewables plus storage increasingly out-compete investment in even low-cost gas-fired turbines. All these new trends work against the revenue models of the utilities that are responsible for the grid, a situation that could be exacerbated if energy communities like microgrids increasingly undermine the grid’s customer base. The trend is reminiscent of the situation in the telecommunications industry, where phone companies saw “over the top” and Internet service suppliers erode their revenues while avoiding obligations to maintain phone line infrastructure. The two sectors, however, are also merging — utilities often need spectrum and cables to support their own operations, and the “internet of energy” will also require new investments in 5G mobile as well as new internet protocols that are necessary to connect everything. The infrastructure needs of the two sectors are increasingly intertwined, and regulators unaccustomed to looking outside their silo will need to be more coordinated.

Newer capabilities also bring new risks. The emerging internet of energy creates new vulnerabilities in the electricity sector on both sides of the Atlantic. Critical energy infrastructure has been subject to significant attacks in both Europe and the United States. Both regions are striving to harden their systems, but have taken different approaches. The United States is focusing more on precise and detailed norms for cybersecurity in the electricity sector, while the EU has done more work on cybersecurity for low carbon technologies and electricity distribution. But in both areas, consumer devices remain vulnerable. Policymakers and industry need to work with consumers to teach them how to reduce vulnerabilities for themselves and their communities and to produce more secure devices.

Policy Recommendations

 Europe and the United States both have deeply embedded regulatory structures for their electricity sectors. The digitalization of the energy sector is putting these structures under stress. Industry and policymakers on both sides are striving to figure out how to adapt to the disruption digitalization is bringing. And despite the differences in their systems, they can learn from the experiences each side is having. Some of the thoughts they should keep in mind, however, include:

  • Accept digitalization — it won’t go away

    The disruption and dislocations brought by change are difficult. Those who are or think they may be harmed will highlight the potential costs of change. But the internet of everything is coming, and with it the internet of energy. Every electrical device will become a node connected to an AI-enabled platform that will drive efficiency and the demand for electricity down. More customers will become competitors, generating their own supply and sending surpluses back to the system. The accelerating trend toward the digital transformation of the energy sector will not go away.
  • But keep it in perspective

    The pace of change is accelerating, but often from a still-small base. Policies related to digitalization need to keep that in perspective. But it’s also worth bearing in mind that renewables and storage, empowered by digital systems, have had an enormous impact on wholesale prices and on decisions over billions of dollars of investment, even as relatively small players on the market.
  • Facilitate the integration of new models and platforms

    The end-point of a substantially decentralized energy system is certain. Policy-makers, the private sector and the public can and should anticipate that end-point and let it guide them. This implies, above all else, being willing to allow new ways of organizing distributed energy resources to come on the market. All players, including utilities, should be allowed to experiment with them.
  • Support markets rather than subsidies

    The new distributed energy resources — including e-vehicles and batteries — and the digital technologies that underpin them have their own economic and commercial rationale. They should be allowed to find their own way. Supporting research is one thing; subsidizing commercialization another.
  • Go for climate change benefits

    The digitalization of the energy sector can have a large impact on reducing greenhouse house emissions if the appropriate policies and planning are put into place. Cities, regions, the EU, and the U.S. state and federal governments should encourage systems-based approaches to redesigning energy grids so that distributed energy systems are integrated to help achieve decarbonization. In the EU, where meeting new energy efficiency targets will require sustained ambition over the next decade, policymakers and industry should work with citizens to illustrate how digital efficiency solutions, such as demand response systems, can help meet climate change goals.
  • Emphasize resilience as well as reliability

    As important as climate change considerations are, however, reliability and resilience are critical motivations too, and in some way respond better to economics. Microgrids, for instance, can provide communities with greater resilience against outages caused by extreme weather, technical disruption, or cyber-attacks on large infrastructure, but they are also the tool to achieve the greatest efficiencies in the digitalized energy world. And it’s clear that resilience to outages increasingly also implies resilience to climate change.
  • Stay focused on cybersecurity

    Policymakers and industry should prioritize developing security in the many small entry points into the digital-energy system. This will require a focus on setting and implementing minimum cyber security standards for IOT devices (such as smart thermostats or washing machines), electric vehicles and EV charging stations, and “smarter” traditional energy devices such as meters. Consumers need to be taught to be responsible for their cyber security and security of their communities.


The digitalization of the energy sector is bringing disruption that may return us to a more distributed energy model that resembles the past. With the efficiencies and controls digitalization brings, both the United States and Europe can build new systems that achieve diverse policy goals if they embrace, and encourage, creative destruction.

Download the policy paper »