On a sun-drenched morning last week in Palermo, the Sicilian capital, a bunker-like courtroom in the Ucciardone prison was the scene of a rare challenge to the mafia. Bosses and low-ranking “soldiers” stared fixedly from their steel cages as seven shopkeepers – hidden by shaded glass – identified those who had allegedly collected extortion money from them for years.

To date, 18 Palermitans – owners of bars and pizzerias, shops and car showrooms, even a street vendor who sells olives – have picked out their tormentors, for whom extortion rackets are the key instrument to control a neighbourhood.

One witness had paid the same mafioso for 22 years; a refusal would have meant a gangster holding a gun to his temple or burning his shop down. At best. One man, the son of an ironmonger, recently refused to pay – and subsequently found a video cassette of the Agatha Christie film A Murder Is Announced on the seat of his scooter.

Italian newspapers hailed the courtroom confrontations as a “revolution”. The wall of omerta, the law of silence imposed by the mafia, was crumbling, said one. You might be forgiven for thinking that Cosa Nostra, as the Sicilian mafia calls itself, had been beaten. Certainly, the government of Silvio Berlusconi has trumpeted the mafia arrests in the past two years – particularly the prize catch of Bernardo Provenzano, the capo di tutti capi, after 43 years on the run.

The government is now saying its priority is terrorism. And the media, in Italy and abroad, are more interested in the mafia’s cousins on the mainland: the Camorra in Naples and the ’Ndrangheta in Calabria, who litter the streets with the bodies of gangsters and informers’ relatives.

The truth is that although the mafia is far less visible than in the days of Provenzano and his Corleonese clan, it is more powerful than ever. As so often in its history, the mafia has undergone a swift transformation – and a younger, flashier breed of bosses has emerged.

When The Sunday Times sent me to Sicily to cover the arrest of Provenzano in 2006, the antimafia fighters I met marvelled at the shabbiness of his end. “All those sacrifices, all that risk, all those deaths – just to end up in a smelly shack? I almost feel sorry for him,” as one prosecutor put it.

The hut was in a valley a five-minute drive from the dismal town of Corleone. Here the leader of both the mafia and the Corleonese clan, the richest and most powerful crime family in history, had spent his last weeks of freedom in a squalid home that smelt dankly of ricotta cheese.

The godfather looked like a peasant, with his silver hair and weather-beaten face. He had been living off simple food such as boiled chicory; the only hints of his wealth were seven Scottish-made cashmere sweaters and £30,000 in cash. Where, I wondered, were the gangsters of the Godfather films with their beautiful women, luxury villas, limousines and partying?

In true rural mafia tradition, Provenzano simply did not believe in flaunting his fortune (the state has clawed back only £4.5m from him and his lieutenants, a tiny fraction of his true worth).

Since the mafia was born as a private security arm for landowners under Sicily’s Bourbon rulers, it has been rooted chiefly in the countryside. The Italo-American heroes of the Godfather films – Mario Puzo, the writer, named Don Vito Corleone, the fictional godfather, after the town – pale in comparison with the real-life Corleonese family. One of its favourite killers, Giovanni Brusca, confessed to torturing victims, committing “more than 100 murders, but less than 200”, and dissolving bodies in acid or roasting them on grills.

My research for a book on the Corleonese clan – which included interviews with a dozen prosecutors in Palermo and Rome and a trawl through 8,000 pages of judicial documents, wiretap transcripts and supergrass testimony – made me realise that a younger, urban generation of bosses is rewriting the mafia script. It is much closer to the villains in The Godfather than the defunct Corleonese clan was.

Foremost among those tipped to head the mafia is Matteo Messina Denaro, 46, a boss’s son from Trapani, in western Sicily. He likes to boast: “With the people I’ve killed, I could create a cemetery.” Drug trafficking has made him rich enough to indulge his passions for Versace, Rolex and Cristal.

Informers say that after being inspired by a cartoon, he toyed with the idea of ordering a pair of machineguns that could spring out of the boot of his armour-plated Porsche. The millionaire playboy gave up on the idea and instead uses the Porsche to ferry girlfriends to nightclubs on the coast.

In one letter that has come into the hands of the authorities, a girlfriend called Maria wrote breathlessly to him: “Please don’t say no. I really want to give you a present . . . I read in a magazine about video games that the Donkey Kong III cassette has come out and I can’t wait for it to be in the shops so I can buy it for you. You’re the most beautiful thing there is.”

There could hardly be a more striking contrast with Provenzano. Messina Denaro – who keeps one step ahead of the police – is in the mafia not only for the power and the money, but also for the joy of flaunting his wealth. He has seen off competition from his closest rival, Salvatore Lo Piccolo – another boss with urban roots, who is now behind bars – but investigators believe he has not yet won overall control. For the time being, the mafia bosses have agreed to share control, with each concentrating on the affairs of his own clan.

Meanwhile, police believe they have prevented the outbreak of a new mafia war by arresting Sicilian gangsters aiming to fill the vacuum at the head of Cosa Nostra.

The reigning mafia bosses are unanimous in their determination to follow a strategy of minimum risk and maximum profit. Indeed, this line was laid down by Provenzano before his capture: “Enough massacres; enough murders. We have to set up businesses,” he ordered. Killings simply brought police out on the streets and were bad for profits, he warned.

The stakes are staggering: the mafia and its two junior cousins, the Camorra and the ’Ndrangheta, earn £42 billion a year, or 10.5% of Italy’s GDP – not including another £35 billion or so from drug trafficking. The roots of such wealth are protection rackets and public works contracts. The mafia ensures that businesses under its control win lucrative contracts to build roads and public buildings such as hospitals or even law courts, or to supply hospitals with medical supplies.

A new bonanza is on the horizon. Mafia clans are eagerly awaiting the start of work on a £5 billion two-mile bridge – championed by Berlusconi – that will link Sicily and the mainland. A study by a senior prosecutor has shown that if a bridge is built, the mafia will control everything from work on local roads to catering for workers.

Berlusconi has brushed aside such concerns. “What is the mafia anyway? A ten-thousandth, a millionth. How many mafiosi Italians are there compared with 57m citizens?” he has said.

Now Italy’s richest man, with a fortune of £5.6 billion, Berlusconi was accused by a supergrass of sending Salvatore “The Beast” Riina – Provenzano’s predecessor as godfather – £53,000 a year in the 1980s. Berlusconi denies this. He has also ended the isolation of mafia prisoners: they can now meet fellow inmates, making it easier for them to rule from behind bars.

Will the mafia ever die? Only twice in the past century can the state be said to have taken it on. First, in the 1920s, when Benito Mussolini used brutal methods to make mass arrests; and, second, in the 1980s, when Judge Giovanni Falcone sent 338 mafiosi to jail in just one “maxi-trial”.

In 1992 Brusca blew up explosives hidden under a motorway, killing Falcone, his wife Francesca and three bodyguards. The wave of outrage that resulted has long evaporated and most Sicilians are resigned to living with the mafia.

Today the burden of fighting it rests on a tiny force of prosecutors and police in Palermo. The prosecutors live as outcasts – their guards never leave their sides and they never socialise, apart from with close friends, for fear of being compromised.

Falcone’s “maxi-trial” was held in the same courtroom where the 18 shopkeepers are confronting the mafiosi. There is no questioning their bravery – eight have been given police escorts. However, with an estimated 80% of businesses in Palermo paying protection money – and 70% in the rest of Sicily – the war against the mafia has yet to be fought, let alone won.

The Last Godfathers by John Follain is published by Hodder & Stoughton at £20

Times Recommends

• Games industry sets its lawyers on downloaders
• Obama close to naming running mate
• Scientists grow blood in lab