On January 6 this year, the 91-year-old former publisher Diana Athill was announced as the winner of the biography prize in the 2008 Costa Book Awards for her memoir Somewhere Towards The End. Athill now looks like a very strong contender to take the overall Book of the Year prize at tomorrow's awards ceremony in London. If she does so her victory, as well as being a personal triumph, will be another token of the high place biography now holds in the affections of the reading public. But it also provides us with the occasion to wonder: what exactly do we mean by biography?
The Costa Prize has always raised problems of comparing like with like - or, rather, comparing unlike with unlike. In considering the final Book of the Year prize, for example, how are the judges to compare the winner of Best Novel with the winner of Best First Novel? If the latter deserves Book of the Year, surely it should have won the Best Novel prize, debut or not. And how does one weigh a book of poetry against a children's book? And so on.
The Biography category presents that sort of problem, in a pretty profound way, even before you reach the final round. Consider the 2008 shortlist. In the first place, we had two down-the-line old-fashioned biographies - the fruits of years of work and research, taking their subjects from birth, or a bit before it, to death and legacy. There was Jackie Wullschlager's acclaimed biography of the painter Marc Chagall, and Judith Mackrell's Bloomsbury Ballerina, a life of the Ballets Russes star Lydia Lopokova, who went on to marry the economist John Maynard Keynes.
Then we had two memoirs. One - If You Don't Know Me By Now: A Memoir of Love, Secrets and Lies in Wolverhampton - was a memoir of youth by a young man, Sathnam Sanghera. The other was a memoir of being elderly by an old woman: Athill's own slim and exquisitely shaped description of her turning to face what Larkin called "age, and then the only end of age." These are such different categories of book that it's hard to see how they can usefully be compared. Athill and Sanghera, having lived their stories, could facetiously be said to know more about their subjects than their two competitors. But the sheer work that the conventional biographer is required to do of necessity outstrips the memoir. Are the judges of a prize for biography to consider prose style, accuracy and diligence of research, originality of arrangement, lasting significance of subject, or what?
Funnily enough, that shortlist, and Athill's success in it, incarnates a massive movement in publishing towards one sort of life-writing - even though Athill herself is a veteran of an age when publishers tended to favour another. Life-writing as a publishing proposition has, these days, divided into three or more parts. In the first place there are the conventional biographies. These are still being produced - though it seems only a matter of time before the only people who can afford to do so will be tenured academics, retired oil billionaires and bookish heirs to manufacturing fortunes.
Writing traditional biographies of eminent dead people - the sort of book whose senior practitioners include Claire Tomalin, Hilary Spurling and Richard Holmes - is a deeply time-consuming, and therefore a deeply expensive activity. The market for it is small, and - with rare exceptions - shrinking. One of the best literary biographies of recent times was John Haffenden's magnificent two-volume life of the poet and critic William Empson. It took him years to write, following Empson's trail through libraries and private papers all around the world - in England, America, China and Japan. The finished product ran to around 1,500 pages and was prominently and admiringly reviewed. Everything you need to know about this fascinating man is in there. If, like me, you love and revere Empson, you'll have whooped with delight when it finally appeared. But if, like most people, you've barely heard of him, you'll say: "William who?" If it sold more than a couple of thousand copies I'd be astonished.
These books still come out - only last year Patrick French's biography of VS Naipaul was handsomely published by Picador. But the squeeze that the recession has put onto publishers' advances across the board means that fewer are being commissioned. Most publishers simply aren't willing to invest - at a price that will pay a writer's mortgage for the time it takes him to do the research - in a book that will start earning out, if it ever does, five or even 10 years after it was commissioned.
Yet even as these books wither on the vine, biography in a broader sense is coming on by leaps and bounds. What, exactly, is filling the gap? One answer - and this is why Athill's Costa success is a bellwether - is memoir. At one end of the market, there are the so-called "misery memoirs" - the point at which autobiography intersects with self-help and, in several cases, fiction. Though the market in these has tailed off slightly in recent years - largely due to oversupply on the part of stampeding publishers - it remains a huge area of the industry, often commanding its own shelf in bookshops: Tragic Life Stories.
But into the broad category of memoir there also fall the ongoing, more writerly serial memoirs of Alan Bennett (Writing Home and Untold Stories) Simon Gray (The Smoking Diaries, Year of the Jouncer, Coda) and of course, Athill herself (Stet, Yesterday Morning and Somewhere Towards The End). The hunger for memoir also means that an increasing number of non-fiction books incorporate an autobiographical element. On the one hand, there are the geek memoirs - autobiographical stories, usually by men, told through the history of an obsession. Nick Hornby's Fever Pitch - about Arsenal football club - can probably be regarded as the paradigm. But others have done the same for collecting (Collections of Nothing, by WD King), or watching Dr Who (Dalek I Loved You by Nick Griffiths), birdwatching (Birdwatchingwatching by Alex Horne) or adoring Judy Garland (My Judy Garland Life by Susie Boyt).
Then there are the comedy stunt memoirs - in which the author travels round Ireland with a fridge (Tony Hawks), or across England in a milk float (Dan Kieran), or travels the world in search of people with the same name as him (Dave Gorman), or says yes to everything (Danny Wallace) or beats the Moldovan national football team at tennis (Tony Hawks again). The other thing that's selling, and selling by the ton, are biographies of living people. The Christmas bestseller lists are filled with the memoirs of sporting and television celebrities - many, but not all, ghostwritten. Russell Brand's memoir My Booky Wook - which he wrote himself, and it shows - was the triumph of the Christmas before last. This year it was the turn of Paul O'Grady, whose At My Mother's Knee and Other Low Joints sold in scads, as did Dawn French's Dear Fatty.
These books are purposely ephemeral. Printed in huge volume, highly discounted and sold very fast, they generally vanish from all but remainder shops within a year or two of publication. In a couple of years time, after all, the way needs to be clear for the author's next autobiography. The model Katie Price - better known as Jordan - has published three volumes of autobiography in as many years. Yet at the same time, a number of writers are starting to break the mould of conventional biography using techniques more commonly associated with fiction. This is not an entirely new thing - one thinks of Gertrude Stein's The Autobiography of Alice B Toklas, for example. But its modern pioneer in English is Peter Ackroyd, whose Life of Dickens was unashamedly subjective, even including imaginary conversations between the author and his subject.
Writers like Roger Lewis - author of a fabulously hostile Life of Anthony Burgess, of whom he was initially an adoring acolyte - have worked to make the subjectivity of the biographical process more visible to the reader. So has Alexander Masters, the author of Stuart: A Life Backwards. This biography of a troubled homeless man was assembled with its subject's collaboration and structured back to front - casting it, at Stuart's suggestion, as a sort of murder mystery. What killed the child he was? The year Stuart was shortlisted for the Samuel Johnson Prize for non-fiction, it was beaten to the main prize by another, equally unconventional biography - Jonathan Coe's Like a Fiery Elephant, about the experimental writer BS Johnson.
In the garden of biography, a million flowers bloom. Let us wish Diana Athill luck in this year's Costa Book of the Year prize. And let all readers and writers of biography, finally, take encouragement from the fact that two well-crafted and absorbing volumes of memoir are even now sitting on top of the worldwide bestseller lists. Their author? A well-mannered American lawyer called Barack Obama.
Sam Leith is the former books editor of The Daily Telegraph.