Tech and energy are teaming up, creating a market that could grow 500% in the next 5 years

As the energy industry faces a time of reckoning — pressured by consistently low oil prices, high operating costs and a growing sustainable investing movement — oil and gas companies are increasingly turning to Silicon Valley for help streamlining operations and boosting efficiencies.

By some estimates, the addressable market for digital oil and gas solutions could grow 500% over the next five to six years, saving oil producers roughly $150 billion, while creating an ever-larger market for tech companies in the highly competitive — and high margin — business of cloud computing.

Opportunities for savings include cutting capital expenditures as well as selling, general and administrative operating costs and transportation operating costs.

“The digital age is finally dawning for Oil & Gas … We see a market poised to erupt over the next five years,” Barclays said in January in a note to clients. “The last 12 months has seen a dramatic shift in adoption, with numerous announcements of cloud and digital-platform partnerships that we think are just early signs of things to come,” the firm added.

In the last year, Microsoft has announced partnerships with Exxon and Chevron, among others, while in May Google parent company Alphabet renewed and significantly expanded its partnership with Schlumberger. Amazon Web Services offers digital services to the industry through its oil and gas division, and counts BP and Shell among its clients.

Energy giants have, of course, been using tech companies’ enterprise software for years, and oil and gas companies’ highly complex operating systems — including precise drilling techniques and rig management operations — have depended on sophisticated data-based decision making for decades.

But oil companies were traditionally somewhat reluctant to hand over their treasure troves of valuable data thanks to cyber security concerns and wanting to maintain competitive advantages, among other things. This meant that for the most part software was developed in-house or by companies within the oilfield services sector.

Now, however, driven by lackluster returns in the energy space and rapid advancements in the tech sector, the two sectors are increasingly coming together, creating partnerships between two industries that in other ways are very much at odds with one another.

“The magnitude of the capacity for processing and storage makes it possible to do things we didn’t dream of within the industry,” said John Gibson, Flotek chairman and CEO and former chairman of energy technologies for energy investment bank Tudor, Pickering, Holt & Co.

“The whole industry needs an uplift in performance, profitability and free cash flow, so working together with the data to improve industry performance has become a mandate … We need the tide to rise for everybody,” he added.

A number of factors are driving the transition, including years of lagging returns in the energy sector.

As recently as six years ago, when oil fetched more than $100 per barrel, producers’ costs weren’t looked at under a microscope. U.S. West Texas Intermediate began a downward trajectory in 2014 and while prices have rebounded from the extreme lows of 2016, WTI remains far from its prior highs, meaning oil and gas companies have had to adapt.

“The oil business here [North America] has gone from gold rush to austerity in a very short period of time,” Shaia Hosseinzadeh, founder of energy-focused private equity firm OnyxPoint Global Management, said. “In this new world, there are a lot of demands being placed on the oil industry. … The entire ecosystem is being asked to do more with less.”

Energy’s continued underperformance — it now accounts for less than 4% of the S&P 500, compared to more than 11% in 2010 — has coincided with major advancements in the tech space, including rapid iterations in areas like machine learning and data processing. At the same time, widescale adoption has led to steep cost declines for things like data storage.

Tech companies can harness insights from applications refined and tested across sectors. It’s difficult — if not impossible — for individual companies to fully replicate what they offer. In other words, partnerships where applications and technologies are co-developed can be the only choice.

“They [energy companies] are realizing that they’re not IT companies. They’re not software developers, but they are users of it,” IHS Markit director Carolyn Seto said to CNBC. “They are partnering with these [tech] companies to be able to gain access to these new technologies, as opposed to taking the development costs themselves of building out capabilities within their organization.”

Reid Morrison, oil and gas advisory leader at PwC, noted that as oil prices rebounded from 2016 lows it also created an opportunity for energy companies to advance these technologies from proof-of-concept to actually moving them into the mainstream where they can hit the companies’ bottom line.

Barclays also made this point, noting that “value creation over the next five years hinges on scalability as Digital moves beyond discrete applications to organization-wide implementation.”

As big oil looks to data services and cloud computing to help its performance and profitability, companies that provide these services could be in for a big payday.

Barclays estimates that the digital services market could grow to $30 billion annually over the next five years, from less than $5 billion today, with the potential market for cloud providers also growing to $30 billion annually. Given the potential size, tech companies are vying for market share.

Raymond James analyst Pavel Molchanov said in a 2019 note to clients that while the cost savings might not be all that pronounced for energy companies, “the sale of these products and services – to energy and other verticals, taken in aggregate — can be quite needle-moving for technology providers.”

“There is an enormous opportunity to bring the latest cloud and AI technology to the energy sector and accelerate the industry’s digital transformation,” Microsoft CEO Satya Nadella said in a statement in June while announcing the company’s three-party collaboration with Schlumberger and Chevron.

On the energy side, Barclays estimates that greater efficiencies will save producers roughly $150 billion annually, which translates to shaving $3 per barrel from the production price of oil.

Besides the oil producers themselves, Barclays said there’s a “golden opportunity” for oilfield services companies like Schlumberger, Halliburton and Baker Hughes to “regain relevancy.” These companies have deep industry experience, and also have their own digital offerings.

The firm said that in the near-term Schlumberger is best-positioned, but that Baker Hughes “may have the greatest upside of all.” The firm noted that these numbers are just estimates since it’s difficult to quantify given the secrecy surrounding the field.

Longer term, technological advancements will also be a way for energy companies to stand out in a cutthroat industry, said Rebecca Fitz, senior director at BCG’s Center for Energy Impact. “In an unhelpful oil price environment, companies could competitively differentiate themselves by growing their margins more than their peers. And that’s where technology becomes interesting.”

For obvious reasons, oil and gas companies are particularly vulnerable to the growing ESG movement, which is when environmental, social and governance factors are prioritized when making investing decisions. Against this backdrop, energy giants are leaning on tech companies to help them make operations cleaner and safer.

Remotely monitoring operations can help companies quickly identify leaks and therefore mitigate the environmental impact, for example. This also means that fewer personnel are exposed to dangerous conditions. Additionally, the very act of moving data to the cloud means that oil and gas companies can reduce the number of energy-intensive data centers needed.

If tech’s involvement helps to boost energy companies’ ESG ratings, it could come at the expense of the tech companies’ ratings. Some argue that since the world is still dependent on fossil fuels, tech companies should help oil and gas companies be as energy-efficient as possible. Others say that making the industry more cost-effective will delay the widespread adoption of renewable energy. When Exxon and Microsoft announced their partnership last February, the oil giant said it could lead to an additional 50,000 oil-equivalent barrels of production per day in the Permian by 2025, generating “billions of dollars in value over the next decade.”

Amazon and Microsoft have recently unveiled ambitious plans to become carbon neutral and carbon negative, respectively, and relying on power generated from renewable sources is just one of the ways in which they’ve sought to make their operations more environmentally friendly.

But still, the tech companies have faced backlash — most notably, perhaps, from employees — for their involvement in the oil and gas industry.

Despite the changes in the last few years, Barclays said that this trend is still in its infancy, although acknowledged that the market can be difficult to gauge due to the secretive nature of oil and gas companies.

But after looking at the sector for many months, the firm said this change in enabling technologies looks set to accelerate.

“Our research reveals a much more vibrant, complex and opportunistic digital oil & gas market than most investors realize; one that is just now starting to emerge,” the firm said.

Subscribe to CNBC PRO for exclusive insights and analysis, and live business day programming from around the world.

– CNBC’s Michael Bloom and Nate Rattner contributed reporting.