Abdalla El Badri, the Opec secretary general, who spoke in Dubai last week, made a comment that explains why oil prices have been so high in the past decade, as well as offering a clue to future strategy.

The International Energy Agency (IEA) and the US's energy information administration forecast Opec's share of output to increase steadily - but, in complete contrast, Mr El Badri mentioned that, as far off as 2035, the organisation would keep its share of world oil production at about 40 per cent.

This 40 per cent is the magic number - the key to understanding the past decade in oil, and the next.

At first sight, it seems obvious that Opec's market share must increase over the next quarter century. It holds 77 per cent of world oil reserves, almost double its share of production. Non-Opec competitors such as the UK, Norway and Mexico are struggling with increasingly mature fields and facing declining output.

The IEA has revised its forecasts repeatedly in recent years. In 2004, it forecast that by 2030, Opec would produce some 65 million barrels per day (bpd). Now this has been revised down to just 54 million bpd. But the projection of a greatly increased Opec market share, well over half of global demand, remains unchanged.

Yet these projections rest on the flawed assumption that Opec will increase its output to make up any gap between non-Opec supplies and steadily growing demand.

The projection of sharp increases in Opec market share runs against historical precedent and the organisation's self-interest. Only in two years have Opec's current members produced more than half of the world's oil: 1973 and 1974, during the first oil crisis.

Since 1993, and the recovery of Kuwait after the first Iraq war, Opec's market share has been essentially stable, at about 39 per cent. This is despite war in Iraq, violence in Nigeria, nationalisation in Venezuela, the collapse and resurgence of Russia, record low prices below $10 per barrel, record high prices at $147 per barrel, soaring Chinese demand, a global recession and all the other vicissitudes that have buoyed and battered world oil markets.

This outcome was both accidental and planned.

It was accidental because it depended on failures in various Opec members that left Saudi Arabia as the unchallenged, dominant force in the organisation, with the support of its Gulf allies the UAE and Kuwait.

Venezuela's production grew so strongly through the 1990s that its oil minister reportedly said at one fractious Opec meeting, "Yes, we are over-producing, but we are not cheating [on quotas]". In 1997, its strategic plan foresaw 2010 production at 8 million bpd. After the 1998 oil price collapse, the newly elected president, Hugo Chávez, struck a deal with the Saudis to limit oil output.

But Venezuela, suffering from mismanagement, hostility to foreign investment, and lavish social programmes paid for by oil money, bore the main burden of the cuts. Last year, Venezuela produced, not 8 million bpd, not even the 3.5 million bpd at the time of Mr Chávez's accession, but only 2.5 million bpd.

Similarly, because of a combination of sanctions, war and mismanagement, the two other plausible Opec challengers - Iran and Iraq - stagnated. And non-Opec production growth weakened as Russia faltered under a burden of heavy taxation, maturing fields and Vladimir Putin's reassertion of state dominance over domestic oligarchs and foreign investors.

Yet Saudi Arabia's revived dominance was planned - because Riyadh took advantage of its rivals' weaknesses to steer Opec in the most favourable direction. The Saudis executed major expansion projects but exercised self-discipline.

The Saudis' share of Opec output has actually fallen slightly during this millennium, as they carefully accommodated a new member, Angola, and the slow return of Iraq. They have been richly repaid by soaring prices.

Economic modelling suggests that Opec's current market share is optimal. If production were lower, its members would lose revenue, and the resulting high prices would drag even more seriously on the world economy than they do today.

If Opec increased its share of global oil output, though, the resulting lower prices would not be fully compensated for by higher volumes.

However the Saudis came to this conclusion - by some combination of careful analysis, and a series of ad hoc adjustments based on market and domestic imperatives - the implications are clear: global oil production will increase at about the same rate as non-Opec supplies.

There are two caveats to this. First, if non-Opec output ceases rising and begins to fall, Opec will have an incentive to lift output to compensate, and ease global economic damage. Second, if Opec's own discipline crumbles, the Saudis may again lose control as they did during the 1980s and 1990s.

That could be caused by a resurgent Iraq, or a change of policy in Venezuela.

If oil prices are to fall without either Opec breakdown or global economic recession, non-Opec output must rise, and oil use must be minimised by new technologies and improved efficiency. In a subtle, understated way, Mr El Badri has implicitly laid down the gauntlet to oil consumers.

Robin Mills is an energy economist based in Dubai, and author of The Myth of the Oil Crisisand Capturing Carbon