The oil and gas business was dramatically different in 1981—fundamentally the same, in that it was based on monetizing hydrocarbons, but how that effort is capitalized and how the value is added and made into money is much more sophisticated, transparent and accessible today.

What have been the game-changers, according to experts in the fields of exploration and production and the capital markets? There are several that have been key to transforming the way the U.S. E&P business works today.

Open trading of oil and gas futures—and in real time, creating a strip price and the ability to hedge future production, locking in economic prices and using those hedges as collateral in bank financing. Open auction and negotiated data-room processes run by competitive firms independent of the seller. The major oil companies' exit from the U.S. onshore and shallow Gulf of Mexico to deploy capital and human resources into the deepwater Gulf and abroad.

The creation of private-equity firms focused exclusively on placing institutional capital into start-up E&P companies, instead of E&Ps having to seek funding from each of thousands of potential capital partners.

And, the technology: 3-D seismic, horizontal drilling, specialty proppants and frac fluids in multi-stage fracture stimulations, ships for deepwater drilling and greater horsepower for ultradeep drilling.

Dr. Chandra Rai, director and Eberly Chair at the University of Oklahoma’s Mewbourne School of Petroleum and Geological Engineering, credits 3-D seismic as having the biggest impact on the industry in the past 30 years.

Organic growth

In 1981, virtually all U.S. oil and gas wells were drilled vertically, and most underwent some amount of hydraulic fracturing. Today, well more than half of the rigs at work are drilling horizontal wellbores and all of these—well, at least the commercial ones—are fraced, and multiple times.

For these completions to be successful today, proppants and frac fluids had to become more sophisticated too, says Dr. Chandra Rai, director and Eberly Chair at the University of Oklahoma's Mewbourne School of Petroleum and Geological Engineering.

"In the evolution of proppant, as the industry started drilling deeper and deeper, you needed material of higher strength, and bauxite proppant was one. The other one was sand coated with resin, which gives it more strength. Untreated sand will crush under higher pressure as you go deeper into the reservoir; it will become powder."

As for frac fluids, the principles of hydraulic fracturing were known much earlier than how to produce oil and gas from shale, he adds. However, it has become increasingly sophisticated from shale to shale in terms of type of fluid and the ratio of proppant to fluid, for example. "In the Barnett, operators use slickwater. The slicking agent reduces the friction. But, in other shales they are using more chemicals because slickwater might not be the most appropriate fluid to use in those."

Remarkable is that the technology has become so pervasive that it has been commoditized, he adds. "So many people are able to do horizontal drilling now, but few were able to do it 15 years ago."

Greater use of directional drilling onshore the U.S. has improved economics as well. It was primarily used offshore, as building production facilities for every well is more expensive in a marine environment than onshore. From one platform, offshore producers could produce from eight wells and, eventually, as many as 30 or 40 wells.

"That's the reason directional drilling came about," Rai notes. As producers moved to deeper and deeper water, directional drilling became even more key as infrastructure and resources farther ashore are exponentially more capital intensive.

Onshore, directional drilling has become essential in reducing costs as well—and the environmental footprint. "On the continent, you can operate a well that produces for 30 or 40 years. Someone can make money on it. But, offshore, you cannot afford to do that. The well life is much shorter."

Probably the greatest impact on industry in the past 30 years has been 3-D seismic, he says, contributed in large part by technology from outside industry. "In the late 1980s, computing power became cheaper. The price started to come down. Along with it came the ability to look at 3-D structures below the surface where 2-D imaging is not sufficient.

"In a structural trap, for example, you can drill with a 2-D dataset because that may be sufficient. For a more complex structure, you have to bring the drillbit very close to where your object is, and 3-D has shown over and over again that it is naturally a more superior image."

Handling the enormous amount of data has been helped as the cost of storing digital data has declined precipitously over the years—a personal computer might have had 4 million bytes of storage capacity in 1992, for example; today, a 1-trillion-byte external drive may cost $75.

"The cost of the storage, plus the advancement in seismic-data processing, enabled people to take large datasets and process them in a short period of time. As all of that came together, there came to be a realization in the mind of the geoscientist that a 2-D image of the subsurface is not sufficient to describe the complexity of where producers are drilling."

He notes too that, while the idea of 3-D imaging existed before the late 1980s, it may not have been deployed when oil and gas prices rose through the 1970s and into the early 1980s, because producers could drill just as many dry holes each year yet still make more money. Low oil and gas prices of the later 1980s and into the 1990s made better results necessary.

"Every company was desirous of cutting down the number of dry holes. How do you cut your number of dry holes but by imaging the subsurface as well as possible and with the data to be able to do as much interpretation as needed? 3-D made all that possible."

Jack Randall, co-founder of Randall & Dewey Inc., now part of Jefferies & Co. Inc., says one of private-equity firms’ significant contributions to industry is today’s increased competition for assets.

Inorganic growth

Jack Randall points to the evolution of private-equity investment in E&P as critical to the energy landscape today. Randall co-founded, with Ken Dewey, the transaction advisory firm Randall & Dewey Inc. in 1989 that is now part of investment-banking firm Jefferies & Co. Inc.

Among the plethora of formal energy private-equity firms' contributions is the increased competition for assets today, he says. Collectively, more than 100 well-funded E&Ps, most of them with the mission of deploying that capital on acquired assets and creating more value from them, are in data rooms across the U.S. now.

In his early career at Amoco Production Co., a subsidiary at the time of Standard Oil of Indiana, "our acquisitions program was largely based on properties we thought had additional potential and where the current owner might be a smaller company that just didn't have the capital or knowledge to access the potential. We now have this vibrant capital market out there where it is much easier for a smaller company, even a start-up, to make acquisitions. If you have a decent track record, you can go out and get money from EnCap (Investments LP) or NGP (Natural Gas Partners) or one of many others, go to a data room, buy some properties, build a company, sell it and do it all over again.

"There are people all along the way who will help you do that now."

Independent, well-organized firms that assist in putting majors' and large independents' assets in the hands of smaller E&Ps seems like an obvious idea today, but, in 1988, companies that were divesting properties usually ran their own data rooms or hired an investment-banking firm to handle it. Eventually, both were perceived to be less effective and more expensive for asset divestments that didn't involve exchanging securities, and that did involve in-house manpower to run a data room and vet the bids.

Simultaneously, large E&P packages were hitting the streets. The watershed event, as Randall and many others in industry deem it, was the 1989 divestment of Tenneco Oil Co. It heralded a new age.

"There was an event around the Tenneco deal that wasn't supposed to happen—that is, the way the majors reacted to the opportunity to buy Tenneco." Simply, some didn't bid at all, or bid on only one or two packages. He and his partner, Dewey, a colleague at Amoco at the time, discovered that the majors were preparing to heavily focus on selling U.S. properties—not on buying them.

"All of these large companies were deciding almost simultaneously to make the same move and that was to focus on international and downsize in the U.S. It was a watershed event in my mind because, in a low-key fashion, it announced that these companies, which were buying and strongly competing with each other for new assets, were now going to become sellers," says Randall.

The Tenneco properties and those within the majors' portfolios that were cut loose in the coming years were still full of upside opportunity, he adds. Previously, the majors' hand-me-downs had been held onto to near extinction. "Pretty much, prior to 1980, the world was one where the majors and large independents owned a property from birth to death. Now, it's a more efficient situation."

Since the1980s, stricter economics are run and underperforming properties—compared with the owner's scale or business model—are cut loose, ending up in the hands of a smaller operator with less overhead and production, and where an improvement in performance that would not be meaningful to a major oil may be a 10% increase in earnings to an independent.

"And it continues to be passed down the food chain. There are properties during the past 22 years of Randall & Dewey that we've seen in the data room numerous times. For all of this to happen, the market had to evolve. I like to think our company had a lot to do with that, as well as the private-equity firms and the mezzanine-finance guys."

And it's due to hedging too, he adds. In the 1980s and in 1990, Nymex trading in oil and gas resulted in creating a strip—a fungible oil- or gas-price forecast—that enabled asset buyers to bank their buys. Previously, large companies each had their own, internal forecasts. Bank loans were based on the price of oil or gas at the time. With the development of forward markets and the ability to hedge future production, "all of that changed.

"It's kind of academic now. You want to know what oil and gas cost five years out? Here's the futures market. And, you go and lock in the price in the derivatives market. It's not what a group of employees at a conference table some place with a crystal ball think."

The advent of independent asset-marketing firms has resulted in a seller's wider reach—even to buyers abroad—via a formalization of data rooms, negotiating processes and agreements, Randall says. "When we started, the typical data room was kind of a treasure hunt. 'Here are the files.' There may be an additional reserve report available. You would have to find the upside. So the company that could find the upside and properly risk it was typically the winner in the bid process."

Asset-transaction advisors stepped in to review all the data and state the potential upside. "We leveled the playing field and made the process much more competitive."

He isn't surprised to see the majors now buying back onshore the U.S., where 15 of the Top 20 gas producers and 12 of the Top 20 oil producers are non-super-majors today, according to a Chesapeake Energy Corp. review.

"It was very predictable. In about 1990, they all went the same direction at the same time. It wasn't a good sign and we knew they would be coming back eventually."

Exploration in virgin regions abroad has another set of risks. "It's always easier to conjure up in your mind a lot of reserves where there isn't a whole lot of data to support it. It's always harder for acquisitions to compete with exploration because exploration, by its nature, is based on a lot of theory and estimation about how huge the reserves could be. It's harder to dream that into an acquisition."

There are also more or different political risks abroad. "Exploration has a very wide range of potential outcomes, from a loss to a huge gain; acquisitions have a much narrower range of outcomes. So it's that challenge that management faces when looking at how to grow: Through acquisitions or exploration?

"It was not a surprise to see the majors coming back. It was not a surprise at all."

The new E&P

Joe Foster, chairman of Tenneco Oil when it was divested and who founded Newfield Exploration Co. in early 1989, employed each organic and inorganic advancement cited by Rai and Randall in growing Newfield from $9 million of start-up funding to more than $9 billion in market capitalization today. The company is No. 19 on the Top 20 list for U.S. gas production, making more than a half-billion cubic feet per day at year-end 2010.

In particular, Foster could have used today's ready private-equity funding. "It is tremendously easier to get start-up funding today," he says.

Joe Foster, founder of Newfield Exploration Co., employed both organic and inorganic strategies in growing the company from $9 million in start-up funding in 1989 to a $9-billion-plus market capitalization today.

His formation of Newfield played a role in that, he adds. Private-equity firm Warburg Pincus, led at the time by Howard Newman (who has since formed his own Pine Brook Road Partners), finally came into the Newfield investor fold on the company's second round of capitalization. And, it wasn't until after the University of Texas' endowment fund committed $3 million in the first round and, then, Duke, Princeton and Yale signed on in the second round.

"I stumbled across Howard Newman. At the time, there were very few private-equity firms and even fewer were invested in energy companies. I believe our success at Newfield in bringing Warburg Pincus in led to a lot of other private-equity firms becoming interested in investing in our kind of company," Foster says.

Warburg Pincus eventually made a niche business of energy investments. Others have followed, including EnCap, NGP, Lime Rock, Morgan Stanley, Riverstone, KKR, Metalmark and more. "A lot of people noticed what Warburg had done, and it also gave Warburg courage to make other private-equity investments as well. Now, there is a systematic search by private-equity people looking for operating people.

"You can see these guys in every phase of the business, from land acquisition to delivering products to markets."

With more buyers in data rooms, "you have a more liquid market and a greater probability of getting a better price," he adds. "The private-equity-backed companies have done so well that they have put pressure on the bigger companies to do things differently. It's been healthy for the industry."

Foster also deployed technological improvements, such as 3-D and horizontal drilling, at Newfield as these innovations developed and became accessible. "The great advantage 3-D has brought is that you can delineate prospects much more powerfully and with more detail than you could with 2-D seismic."

The technology alone contributed to bids for offshore leases beginning to escalate again, bringing the shallow Gulf of Mexico back to life after the 1980s and, later, contributing to risk-taking in deepwater exploration. "With more detailed data and more certainty that the oil and gas would be there, the risk of making these bids was lowered and the offers escalated significantly. 3-D versus 2-D is like day and night."

Initially, the majors deployed 3-D in the Gulf of Mexico, as they could afford the shoots, and independents joined in to pay a share to get a look. "So not just the majors had it then, but the smaller companies had it—the fastest movers had it—and it became a tool that resulted in a hell of a lot of oil and gas being found."

Foster sees improvements in virtually every aspect of the E&P business over the years and ongoing in real time today. When he started his career in petroleum engineering in the 1950s, drilling in maybe as much as 100 feet of water was considered deepwater, for example.

"The longer you live, the more you realize that just about everything is an evolutionary process, that one thing builds on another. That's what happened in the Gulf. That's how we got to drilling in 10,000 feet of water: It's because, first, we learned how to drill in 10 feet of water. There are generations of engineers, other industry members and capital partners that came along and improved on what was being done and extended the capabilities of the business."

Also on his short list of oilfield innovations has been open trading of oil and gas futures. "Nymex was one of the greatest things that happened. It simplified the valuation. When you're looking at buying a property, you have to look at what prices are going to be over a period of time. That was difficult to do. Nymex sort of takes that out of it. It gives you a strip. The strip may be right or the strip may be wrong, but it gives you a basis upon which to make an intelligent bid, based on a price that you think will be pretty similar to what your competitor will use.

"It really takes the oil and gas price out of the game and makes the emphasis more on the reserves and what you perceive the upside to be."

And, it was crucial to Newfield's start-up with $9 million, compared with private-equity commitments today to start-ups of as much as $1 billion. "Early on, we weren't exactly managing cash day to day, but we were managing cash pretty carefully. We locked in prices (via hedging), so we knew that we would have the capital needed to undertake the programs we were prepared to drill. Gas prices were pretty volatile back in the 1990s when we were doing this, so the fact that we could lock in prices was very helpful to us to manage our business."

Newfield would hedge two-thirds of future production and let the balance "ride"—that is, be exposed to the ups and downs of the spot price and also kick in to supplement the hedged or promised production if there was a major interruption in company-wide output.

"We had our downside covered with the floors and the upside open to us. It let us manage a small company. We were at risk of failure for the first five or six years. Nymex trading in oil and gas took a lot of that risk out in respect to the uncertainty of pricing."

The next great thing

Without pause, Rai, Randall and Foster each look to U.S. natural gas as being the next great industry game-changer in the coming years. Rai says, "This is a whole new source of fossil fuel. Shale gas. No one would have said this 10 years ago. This is a complete break of the paradigm."

That break is because shale is a source rock. Randall says, "I don't think it has really sunk in with people outside the E&P business what this ability to extract hydrocarbons from what, essentially, is the source rock means in terms of energy supply. It's a total game-changer.

"You have to get your mind around it to get your arms around it. And there is a lot of source rock in the U.S. and a lot of technology that is making it easier to produce it. I don't think we're anywhere near the end of exploiting U.S. hydrocarbon resources. This is almost a whole new beginning."

The phenomenon may impact energy dynamics across the globe. Foster says, "There is shale all over the world. Presumably a lot of it has gas in it. The exploitation of shale gas is ultimately going to have a huge impact on the energy equation—not only in this country but in others."

Further out, Rai says methane hydrates may come into play. "If someone could mine the gas hydrates, it may be another very large source of natural gas, which, of all the fossil fuels, we know is the cleanest out there."

He also expects that industry in the next 30 years will be surfacing far more resources with fewer personnel. "The industry will be able to drill wells better and frac them better as science progresses and better understands the whole system of rocks and fluids. We may not have to do 12 frac stages in shales. Maybe we will be able to do fewer. And thus the margin, the cost versus the price, will keep getting better."

What about the application of unconventional technology—horizontal drilling and multi-stage fracturing—to conventional oil and gas formations, such as the Midcontinent's Mississippi Lime? Foster says, "Again, I think we're in for a lot more evolution. I'm confident a lot of that is going to happen. I've been in this business since 1957 when I got out of college and, except for when the gas bubble was eating us alive (in the 1990s), I felt like this was a great business with a lot of opportunities. And, I feel that way now, 54 years later."

Will there be another golden age for the Bob Simpsons, Jon Brumleys, Steve Farrises and Jim Floreses—who founded or grew multi-billion-dollar E&Ps (XTO Energy Inc., Encore Acquisition Co., Apache Corp. and Ocean Energy Inc., respectively) on the backs of acquisitions from the major oils? Could the industry be in the next great golden age already with unconventional resources in play?

Randall says of Simpson's XTO, which was founded in the 1980s, "That company grew from absolutely nothing almost solely through acquisitions and the exploitation of those acquisitions. Hedging also played a huge role in enabling XTO's growth. It was sold in 2009 for $41 billion. That's a tremendous story. And there are others, too.

"It's a real good question whether that sort of golden opportunity is still ahead for some people. It's become a lot more competitive than it used to be."

And the future of the Gulf of Mexico? Randall sees tremendous opportunity for the redevelopment of the shallow-water Gulf. Also, ultra-deep prospects—at more than 30,000 feet—in the shallow Gulf, which Newfield tested several years ago and McMoRan Exploration Co. and partners are working to prove today, are very exciting.

"There is a huge opportunity there, if the right technology can be brought to bear, and I think it will happen."

Foster's advice to industry leaders of the next 30 years? "The best way to stay out of trouble is 'Don't get caught speeding.' There are a lot of ways to speed, besides on the highway. The point is,'Don't go too fast and don't believe that just because you've been successful two or three times, that you can't fail the fourth or fifth.'"