The corporate history of the U.S. shale renaissance was not some “Night of the Living Rich” in which every developer of every crappy reservoir woke up at the same time, got wealthy and bought a private plane with mud-stained cash. Harold Hamm, Mark Papa and George Mitchell succeeded for specific reasons: being early in a shale basin and being relentless in trying techniques until one worked. But there were diverse ways to win in the boom—and just as many to lose.

Through some combination of buying, selling and developing shale gas and oil reservoirs, E&P executives tried to get rich—or, in the anodyne Wall Street term, create value. (“I’m not European-castle-owning rich. I just created value.”) There was no pretested recipe for buying, developing and selling that worked to win part of the $1.35 trillion in new reserve wealth at year-end 2014.

There were successful land flippers, equivalent to vacant lot hunters who benefit when a hipster opens an artisanal barbershop in a formerly marginal neighborhood. These companies leased land from farmers for peanuts and then sold those leases without drilling a well when a land rush began. Other companies pursued a “dot map” approach, in which they leased land, drilled a few science experiment wells, and then sold their assets to another company with the anecdotal well results “derisking” the acreage.

Companies like Devon Energy Corp. or Anadarko Petroleum Corp. were like full-on real estate developers: they aggressively drilled wells and leased new acreage and swallowed neighboring companies like Mitchell Energy whole. In effect, they reinvested their profits to add buildings to their mixed-use developments and improve the value of their land with better tenants paying higher rents. These companies, if they were private, went public and cashed in after an IPO. If they were already public, executives and employees could see their wealth rise with their companies’ stock price, as less rich Harold Hamms.

Risks and rewards

At each stage, the risks were real. Much like for landowners, a company would face the choices of caution versus confidence, patience versus pace, with countless examples of lucky bastards who had gotten rich holding on and poor bastards who had lost by prematurely selling out—and vice versa. Some companies paid millions for acreage that would turn out worthless. Other companies saw ravenous rivals agreeing to terms that one could classify only as ATFD—Are They [Bleeping] Drunk?—and then watched as acreage in suburban Fort Worth that was leased for $26,500 per acre, with a 25% royalty rate, on a three-year term ended up actually making the drunk rival millions when the Barnett Shale wells worked.

Take Ohio, for example, which eventually roused the imagination of frontier-oriented E&P companies. Why Ohio? Well, why not Ohio: shale plays underlie—and feed—shallower reservoirs, and oil and gas production had dribbled out from shallow Ohio reservoirs for decades. Hell, the state was the largest producer of American crude in 1896.

So a few companies shook cobwebs off old Ohio well logs and geological maps. Eventually, they focused on the deep Utica Shale there. Pioneers began drilling vertical test wells, then a few horizontal multi-stage fracked wells. By 2010, the industry was abuzz with the possibility that the entire eastern half of Ohio was going to be an Abu Dhabi on the Erie. There were parts of the Utica that would give up oil, parts that would give up gas and parts that would give up “wet gas”—gas mixed with natural gas liquids like butane.

After more wells were drilled, some operators crowed about greater successes and others begged for patience. (“We’ve drilled only a few science experiments.”) Eventually, after enough wells, the industry had a better sense of the reservoir quality, the unit costs and profitability of each part of the Utica. It turned out that the “oil window” of the Utica was open only to marginal returns. The dry gas portion in the southeastern corner of the state, however, boasted insanely productive wells.

Over just two years in Ohio, companies could judge their decisions in retrospect, whether they were right to sell, stupid not to, or unlucky in delays. The winners would look with half-justified pity at the companies that lost money, because their me-too strategies were unable to replicate the financial results of those quicker to read the market or the technical results of more talented operators. Sometimes losing meant not selling quickly enough.

One company famously reported to its investors that it had a deal to sell all of its Utica Shale position for $6 billion. This meant $6 billion in profits, because the company had gotten the Utica acreage for free when it bought some shallow conventional wells above it years earlier. But the negotiations took so long that the billions-offering buyer changed its mind, likely spooked by some well reports. The jilted seller still holds most of its Utica acreage.

Sometimes losing meant selling too quickly. A few companies, I’m sure, smugly congratulated themselves after jettisoning acreage in the dry gas window of the Utica. It was a loser’s area. America needed more dry gas like it needed more cable TV stations. The sellers later kicked themselves as they read about the buyers drilling huge wells there and becoming new stars of the oil patch.

And sometimes losing meant buying too quickly. In 2013, had you told an oilman that you could get him acreage in the Cline Shale in West Texas, his heart would have started racing as if you were offering him a month of whiskey and fishing, with his boss’s approval. Now, if you offer him some Cline, he’ll pat you on the back and say that he’ll have to de-Cline. He would rather spend a month at the dentist. A company I know tried to sell Cline acreage to 174 different operators in 2014. Not one bothered to test the company’s desperation with a price.

Like a modern dance performance, the early days of a boom in any area were randomness and strange posturing. (These performances would be faster and less random with each successive boom.)

As each play matured, two patterns emerged. First, in most places, the core of the plays—where the oil and gas bearing rock is the thickest, where it is least interrupted by natural faults, where its rock characteristics are most responsive to fracking—turned out to be even better than people thought. The fringes of a play, on the other hand, were worse.

The Mississippi Lime play, once thought to cover a dozen counties in southern Kansas and a half dozen in Oklahoma (and none—don’t ask—in Mississippi), shriveled in a couple of years to parts of four or five counties in Oklahoma. Current estimates are that the cores, subjectively defined, usually represent the best 20% to 30% of any play.

Second, in every area of the boom, some operators consistently made better wells than others. Even the village idiots of the oil business can improve on shale well productivity by adopting neighbors’ techniques to deliver more stimulation—more fracks and more proppant—to more parts of the reservoir. But oil and gas production is no different than other endeavors. Some people are just better, more intelligent, more patient, more data driven: they drill better wells because they spend time in the field and understand every step of the process tactilely, not as a plan on paper; they drill better wells because they study intensively what’s working and what’s not, thinking clearly about geology, engineering and physics; they apply cold logic to the quality of acreage based on the underlying rock, not their longing to become the next Harold Hamm.

More nimble organizations have also been able to adjust well designs quickly, extracting lessons from a well just drilled or one currently drilling, rather than downloading every well plan from some bureaucratized template. And those more nimble companies, with engineers as buddies (or brothers) of landmen a few offices away instead of in a rivalrous department on a separate floor, have been quick to capture the opportunities to lease new land as implied by their own wells. Mark Papa, not surprisingly, divided EOG Resources into smaller, self-contained, play-focused companies—a collection of mini-EOGs.

Barrels of disruptions

In the shale revolution, the pattern of what Harvard Business School professor Clayton Christensen calls disruptive innovation—in which second-tier caterpillars morph into butterflies—applies not just to individual companies but to the oil and gas itself. In many areas of economic life, being in a second-rate line may not matter. Even if you’re not selling bespoke blouses made with rare Japanese threads, you can still make money selling offensive T-shirts at the seashore.

But in a commodity market, if there is enough supply from new gas wells that can be profitable at, say, $3 per thousand cubic feet, the market will settle—to simplify matters—with everyone getting $3 per thousand cubic feet. This is bad news for even boom-era shales that can profitably make widgets at only $4.50 gas.

For us in the oil industry, it still seems science fiction sometimes that in West Virginia, for decades a land of whispering wells drilled into shallow reservoirs, you can now extract 8 billion cubic feet of gas from a single honking well in the Marcellus. It’s like your ne’er-do-well son announcing he got into medical school.

It is equally incomprehensible that those unbelievable wells in West Virginia may not be profitable because there are so many more 12 billion cubic feet wells in Pennsylvania, driving down the price. It’s like your once ne’er-do-well son not getting into medical school after achieving straight A’s at Yale.

Critics of the shale revolution point to companies that lost money after oil or gas prices fell as evidence that the boom is nothing but a bubble blown up by the combined hot air of Wall Street sharps and Texas wildcatters. (Those critics love to mention the corporate origins of EOG.) Yet this left-wing gotcha has a twin in right-wing cynicism: my oil industry friends declare that the demise of solar panel manufacturers like Solyndra was the just punishment for liberal “crony capitalism.” But regardless of the subsidies for the solar industry, the collapse of the price of solar panels did not mean that the original impulse to invest was corrupt.

Nor does losing money in the oil and gas business mean that the impulse to invest in the shale revolution was an underhanded financial scheme. The lost money means that both impulses were too popular, bringing in too many competitors, leading to oversupply. Markets are often as innocent and as inevitable as that.

I believe that financial success from the shale revolution will be much harder to come by, even as shale production expands. It will not be impossible, for sure. Some companies will secure profitable drilling locations in core shale plays that other operators passed over, because earlier wells were completed poorly or because some crotchety farmer refused to lease prime acreage to oil companies but his children—“you’ll love the nurses at the home, Pops”—like the sound of being rich. There will continue to be entrepreneurial oilfield service companies that invent businesses and products that make drilling, fracking, and other parts of the process more efficient.

However, a booming, leaping, radically disrupting industry is a better creator of wealth than a normal industry facing normal competitive markets with normal oversupply threats and normal opportunities to generate investor returns. The late autumn orchard has fewer apples to be picked.

In the shale business, there is not enough land still held by crotchety farmers to provide opportunities to the hundreds of E&P companies looking for it. And no new major American shale play may be unlocked. For sure, I was not prepared for the emergence of the Scoop, a profitable but smaller play in Oklahoma, and not just because it shares its name with a type of Frito. Shale plays lie beneath where we’ve already discovered oil and gas. We may find some new plays in Oklahoma. We are unlikely to find them in Oregon.

And, yes, oil and gas prices could rise in the future. The world may need the “disrupted” barrels in the Canadian oil sands or Cline Shale or deepwater Angola five years from now or 20 years from now. But demand for oil and gas may have by then faced its own inflection point. The current onshore shales—shales of increasing efficiency and decreasing unit costs—could join production from existing wells and the usual low-cost Middle East sources to provide the reasonably priced hydrocarbons the world wants.

Gary Sernovitz is a novelist, self-described liberal oilman, and managing director at Lime Rock Partners in New York. This is adapted from The Green and the Black: The Complete Story of the Shale Revolution, the Fight over Fracking, and the Future of Energy, by Gary Sernovitz. Copyright © 2016 by the author and reprinted by permission of St. Martin’s Press LLC.