Billionaire newspaper owner tells Luke Harding about his previous bouts in Russia and his opinion of David Cameron

It has become a video sensation: the moment when Alexander Lebedev punched a fellow businessman live on Russian TV. Ever since Lebedev – the billionaire owner of the Evening Standard and Independent – floored tycoon Sergei Polonsky, speculation has swirled: where did Lebedev learn his pugilistic skills? The KGB, perhaps? And why did he swing first?

In an interview with the Observer, Lebedev discloses that his dust-up with Polonsky on a show about the global economic crisis wasn't his first brawl. The oligarch said he had been involved in numerous street fights in Moscow during the 1970s and 1980s – the final years of the Soviet Union.

It was an era characterised, he said, by excessive boozing, hot tempers and social disorder. "Moscow was full of hooligans back then. If you went for a walk in the evening in the street you always got someone asking for money," he recalled. "Everyone in the Soviet Union was doing boxing or karate. I was party to a lot of street fights. People were always drunk."

Sometimes he came off worse, but sometimes he bested his opponent. He said on several occasions his face got covered in bruises and "a couple of times" he woke up in hospital. These fistfights occurred not just when he was a young student but also when he was an "officer" – after he had joined the KGB.

At home in Russia, Lebedev has found his attempts to enter government repeatedly blocked. But since the punching incident on Russia's NTV channel his credibility has soared. The tabloid portal Lifenews.ru has even hailed him as a national hero. Prime minister Vladimir Putin – a black belt in karate – seems less impressed, however. Last week he suggested that veterans from the Soviet war in Afghanistan should be sent in to sort out the country's brawling oligarchs.

Speaking in private Mayfair offices belonging to his son Evgeny in London, Lebedev said he had no idea that Polonsky was going to be on the show, and that Polonsky had then needled him off-camera. Polonsky had behaved obnoxiously, Lebedev said. During the broadcast the construction magnate was "extremely agitated", according to Lebedev.

The brawl came at a delicate moment in the shadow world of Russian politics.with Putin – Russia's pre-eminent politician – deciding to stand in the presidential elections in the spring of 2012 and return to the Kremlin. The press baron said he thought Putin wasn't relishing the idea of a comeback because two decades after the demise of communism Russia was in poor shape. "The crisis in Russia is different from the crisis in Italy or the US," Lebedev said. "The infrastructure is 50 years old. The industrial base is completely eroded. Russia is not producing anything. Corruption is beyond any normal sensible proportion."

During his visit to Moscow this month, David Cameron, the British prime minister, recounted how two charming young Russians approached him on his gap year trip to the USSR in 1985 and struck up a conversation. They were, Cameron suggested, KGB. Lebedev, who served as a KGB spy in the Soviet embassy in London in the late 1980s, said that he thought that this unlikely. He said that the KGB's reputation for efficiency had been grossly exaggerated. "It could have been anything. Maybe [they were spies], maybe not."

Lebedev met Cameron briefly in Moscow at a party in the British embassy. Of the prime minister, he said: "I think he is qualified for the job." According to Lebedev, when his son Evgeny recently met Hamid Karzai in Afghanistan, Cameron telephoned Evgeny to find out what was said – a tantalising hint of Downing Street's continuing backroom contacts with newspaper proprietors in the wake of the hacking scandal.

Lebedev was reluctant to talk about recent management changes at the Independent or the removal of the paper's veteran editor, Simon Kelner, in July. Lebedev also said he didn't know anything in advance about the Independent's serialisation of Julian Assange's unauthorised autobiography, declaring: "Now you see! I'm not interfering."

He added: "We asked Simon [Kelner] whether he would become number one in an international federation devoted to supporting investigative journalism. That is the job he has been offered. I think he likes the idea." Asked about the editor's departure, Lebedev said Kelner had occupied the position for nearly 14 years and it was time for something new.

Lebedev was speaking before a fundraising gala held on Thursday night at Hampton Court. The event to raise money for the Raisa Gorbachev cancer foundation was hosted by Evgeny Lebedev, together with Graydon Carter, the editor of Vanity Fair, and Colin Firth.

The publisher's fraught relationship with the Russian state extends to the security services – he is suing Russia's Federal Security Service (FSB)the successor agency to the KGB, after it raided his bank last year. He said: "The case is being heard in the arbitration court. We are asking for $10m (£6.4m) in compensation."

• The first Russian motherboard based on ARM-processor Baikal-M