UK election upset isn't just a vote of no confidence in Boris Johnson

Prime Minister Boris Johnson, British newspapers are reporting, has suffered a "shock defeat" in a long-time Conservative-held seat in England and the "shock waves" are spreading through his entire political party. Well, possibly.

The Liberal Democrats certainly overturned a Conservative majority of 16,000 votes to win Chesham and Amersham, a prosperous area in south-east England, with a huge swing of 25 per cent. This is the Conservative heartland, and all kinds of excuses are being made for the party’s failure.

Local people, apparently, are incensed by Mr Johnson's plans for a new high-speed trainline, HS2, through the district. They don't like new planning rules that mean more houses can be built on their green and pleasant land. Undoubtedly some voters really do loathe new housing and new infrastructure, but before swallowing these convenient excuses, let us first consider something much bigger that has been going on across the country in the past decade.

More than 80 per cent of the UK’s population live in England. The most influential British media houses are based in England, with their centre of gravity in London. But at times, London media reporting outside the capital – especially in the north of England, Scotland, Northern Ireland or Wales – is a bit like reporters discovering lost tribes in the Amazon rainforest. They often miss one obvious, yet underreported fact.

The "United Kingdom" claims to be, in its name and structure, “united". But in the 2015 general election, each of the four parts of the UK was dominated by four very different political parties. That year, the Conservatives scored what some saw as – that cliche again – a “shock” victory, winning a parliamentary majority.

They ruled the UK, but they were the biggest party in England only. In Scotland, the largest party was – and is – the Scottish National Party. In Wales, it’s the Labour party. In Northern Ireland, the biggest party is the Democratic Unionist Party.

All this happens because the UK in the 21st century retains a voting system for the UK Parliament that was devised in the era of the horse and cart. The “First Past The Post” system (FPTP) is designed to function well with only two parties – government and opposition. But the UK now has at least six significant political parties. If they split the anti-Conservative vote – and they usually do – then the Conservatives can win easily with less than 50 per cent.

This system in the past benefited Labour, too, and in the 21st century, both Labour and Conservative governments have had big majorities – sometimes landslides – in parliamentary seats but with only a little more than 40 per cent of voters backing them.

Since 2019, for example, Mr Johnson has had an unassailable majority of 80 in Parliament, but he was the choice of only 43.6 per cent of voters. All the other parties combined achieved almost 57 per cent. In a fairer voting system, the UK would probably often have coalition governments, like Germany or Ireland and many other countries. Instead, Mr Johnson's minority of the vote and vast majority in Parliament means he could comfortably stay in power until the next election in 2024.

And that’s where Chesham and Amersham voters have outsmarted him and the Conservative party. They voted tactically to get the Conservatives out. The Liberal Democrats won because Labour and Green party voters put their own party preferences to one side in order to vote for the best anti-Conservative candidate. And it worked.

Tactical voting, or a more formal alliance of opposition parties, is not just a vote of no confidence in Mr Johnson. It is a vote of no confidence in an FPTP voting system that is unfair and outdated, and has generally been abandoned in many advanced countries.

Arguably this old system still works in the US, but they retain mostly a two-party system – Republicans versus Democrats – with a few independents and marginal groups. The UK is now definitively a multi-party democracy. The real story of Chesham and Amersham, therefore, is not selfish people voting against housing developments. It is selfless voters voting against the government.

It could happen again on Thursday, in another by-election, involving a Labour-held seat in the north of England, Batley and Spen. Some Liberal Democrats there have committed to sacrifice their own party to vote Labour. There are even those who hope for a formal pact between opposition parties so that in the next general election, candidates will step aside to allow the anti-Conservative majority in the country to become an anti-Conservative majority in Parliament. This would be the real "shock" for Mr Johnson and the Conservatives.

A more radical – and more logical – step would be for all opposition parties to agree that FPTP is unfair and out of date.

As someone who has for years argued for a fair proportional voting system of the type used in other European countries, I remain hopeful. But as someone who has also witnessed supposedly massive “shocks” in British by-elections, which fade into history, I am not yet optimistic.

Gavin Esler is a broadcaster and UK columnist for The National