The headlines are all too familiar. The BBC has been exposed as having deeply unsatisfactory internal processes for investigating problems. Rival media organisations gloat over the inadequacies. Loyal BBC employees squirm with embarrassment. We have seen it all before.

The issue this time is the notorious interview with Princess Diana 25 years ago, conducted by Martin Bashir, a junior reporter on the BBC’s flagship weekly current affairs programme, Panorama. The interview attracted 23 million viewers. But a retired Supreme Court judge, Lord Dyson, has just issued a devastating verdict on how the interview was secured – indirectly, by deceit and trickery – and on how the internal inquiry into that deceit, once it was exposed, was handled – in a “woefully ineffective” fashion. Dyson came close to accusing the BBC of orchestrating a cover-up of Bashir’s misdeeds.

Dyson’s remit did not include two other relevant issues. First, why BBC managers chose to ignore their settled protocol on how to seek royal interviews. Second, why Bashir was re-hired by the BBC as a religious affairs correspondent long after the facts of his deception in obtaining the Diana interview had become known, not least to the man who had investigated his behaviour in 1995 and was BBC director-general at the time of Bashir’s return.

The first of these issues – bypassing established rules – was, at the time, treated as something of a triumph of enterprise over convention. For reasons never explained (and his death in 2017 leaves the questions unanswered), Steve Hewlett, the much-respected editor of Panorama in 1995, made two controversial decisions.

He agreed to let Bashir hijack a running discussion between the BBC and Buckingham Palace as to when and by whom Princess Diana might agree to be interviewed. (She was known to be keen to have her say; her husband had confessed to his adultery in an ITV programme one year earlier.) And Hewlett strongly vouched for Bashir when he managed to secure a meeting (on his own, with no producer present) with Diana’s brother, Earl Spencer. Bashir had presented Spencer with faked bank statements that suggested his former head of security was being secretly paid by a newspaper and the secret services. Spencer had phoned Hewlett to check Bashir’s credentials.

The faked documents and Bashir’s promise of further revelations about his staff were enough for Spencer to arrange a meeting between the reporter and his sister. Bashir reportedly regaled the brother and sister with dozens of bizarre, mostly invented stories of betrayal and espionage. Spencer started to regard Bashir as a fantasist and immediately cut off dealings with him. Diana, however, was persuaded that Bashir was a true friend. To maintain the sense of ‘us against the world’, Bashir arranged with Diana to film the interview in extreme secrecy. It proved to be an explosive encounter.

With his scoop in his bag, Bashir presented the BBC hierarchy with a dilemma: how much to say before transmission, and to whom. Rather than inform the Board of Governors of the interview (its chairman, Marmaduke Hussey was married to one of the Queen’s ladies-in-waiting), director-general John Birt kept them – and the Palace – in ignorance until just before transmission.

Whether or not disclosure might have risked losing the scoop, for whatever reason, became a moot point. But because of all the subterfuge involved, BBC management became all the more vulnerable when Matt Wiessler, the graphics artist who had been asked by Bashir to concoct the fake documents expressed concern. Wiessler raised the issue of whether he had been tricked into a deception with various senior people at Panorama. Hewlett’s reaction was to tell all who raised the matter with him to mind their own business.

When multiple newspaper stories about the documents forced an internal inquiry months later, Bashir repeatedly lied about showing the fakes to Spencer. When he finally admitted having done so, Bashir came up with an unlikely story. Diana herself had provided him with the key information used in the mocked-up bank statements, he said, though he had not met her when he commissioned the fakes and it is doubtful that she could have known the bank details of her brother’s security officer.

Nor could Bashir come up with any credible excuse for having created the documents in the first place. Diana did, however, send the BBC a handwritten note saying she had no regrets about doing the interview and had not been shown anything by Bashir that might have misled her .

Several BBC managers struggled with the prospect of having to tell the Board of Governors – and the British public – that the much-vaunted scoop had been secured by an initial act of deception, committed by a self-confessed liar. Comforted by the handwritten note, Tony Hall, head of news and current affairs, told the governors that despite a “lapse”, Bashir was honest and honourable. But Wiessler, the graphics designer who had blown the whistle, would never be employed by the BBC again. The Panorama team members who had raised the matter of the fake documents were weeded out, labelled as leakers of information to a hostile press.

John Birt, BBC director-general from 1992 to 2000, tried to justify Hall’s actions to Dyson. Birt said Hall’s extraordinary willingness to take Bashir on trust was because there was no “counterfactual”. This was acerbically dismissed by the judge, who pointed to Spencer as an obvious source of a “counterfactual”, but who was never approached by BBC managers for his version of events.

John Ware, a veteran Panorama reporter who presented a corrective episode of the show on the day of Dyson’s publication of his report, identified “incuriosity” as the fatal flaw in the BBC’s management method. In doing so, he echoed the Pollard Review from seven years ago on another BBC failure. The editor of Newsnight had inexplicably blocked coverage of the Jimmy Savile scandal, yet mounting concern about this had seemingly failed to reach the top levels of the BBC. Nick Pollard, former head of Sky News, said the lack of interest by then director-general Mark Thompson had been matched only by the efforts of those below him to shield him from unwelcome knowledge.

There is a regular element of circling the wagons whenever the BBC finds itself under attack. Acknowledging fault almost never happens, except when an outside agency – with no institutional interests to protect – investigates.

In 2003, a government scientist, David Kelly, died by suicide. He had been identified as the source for a hotly contested item on BBC Radio 4’s Today programme. The story related to alleged duplicity in Tony Blair’s administration’s account of Saddam Hussein’s weapons of mass destruction. A judicial inquiry by Lord Hutton was scathing about the BBC’s editorial processes as well as its seeming inability to admit error.

That episode led to the ousting of the BBC’s chairman and director-general – Gavyn Davies and Greg Dyke – as well as to the replacement of the Board of Governors by a more distant oversight body, the BBC Trust. Yet the change of governance did little to improve BBC accountability.

A report to the Trust by BBC management on a scandal involving fake phone-in participants to programmes was exposed as self-servingly incomplete by a parallel inquiry conducted by Ofcom. The media regulator was created in 2003 to oversee all broadcasters other than the BBC. (The BBC conceded that Ofcom could oversee complaints about fairness.)

Then the Trust mounted its own investigation into a Panorama episode on child labour in textile manufacture and concluded that the programme had almost certainly been unfair to Primark, the clothes retailer. However, the Trust was unable to make production team members participate in the inquiry, nor sanction them for refusing. A campaign by the Trust to force BBC management to reduce salaries at the top of the BBC led to an unseemly public row over the scale of pay-offs to departing executives. The model was not working.

The Newsnight-Jimmy Savile fiasco was compounded by the programme’s attempt to expose another paedophile scandal. However, that report mistakenly identified a Tory peer as a perpetrator, resulting in hefty libel damages, the departure of another director-general barely two months in office , followed not long after by his chairman and the whole Trust structure.

The 2015 BBC Charter review saw a combined board of executives and non-executives placed at the top of the BBC. B ut this proved no more successful when a glaring breach of privacy was committed by BBC TV News. Acting on a tip-off to a junior journalist, who used it to pressure South Yorkshire police into reluctant co-operation, the BBC believed it had a scoop when it learned that a flat belonging to Cliff Richard was about to be raided by police. Richard had not lived in the flat for some time. The police were following up on an anonymous complaint of sexual abuse. The BBC deployed a helicopter above the flat, named Richard as its owner and ran extensive live footage at the top of a lunchtime bulletin.

No arrests or charges were ever made, and it later transpired that the top BBC managers who took the decision to use the pictures should have known they were unlikely . When Richard failed to secure a public apology, he sued the BBC, and won. T he trial judge was so dismayed by the BBC’s behaviour that he awarded hefty damages. Combined with the payment of costs on both sides, the BBC shelled out more than £2m. It was Tony Hall, now Lord Hall of Birkenhead, who as then director-general of the BBC took the decision not to apologise. Instead, he chose to criticise the trial judge for ignorance of the law of privacy. No BBC News executives were disciplined, let alone fired, over this affair.

By then, Hall had also nodded through the re-hiring of Martin Bashir, and his subsequent promotion to BBC religion editor. But the growing pressure from various newspapers, now briefed by Earl Spencer as to the skulduggery in 1995, was putting mounting pressure on the BBC to hold some sort of inquiry. Hall sensibly relinquished his post in August last year, and his successor, Tim Davie, asked Lord Dyson to report on what had happened in 1995/6 (though not on later events). Various ideas are being floated as to how to avoid further such disasters, including a separate BBC subsidiary board. It would be made up primarily of independent editorial experts, to supervise BBC editorial output. The model cited is the main Ofcom board and its subsidiary Content Board, chaired by one of its main board members.

But there is a much simpler answer: treat the BBC just like any other broadcaster and make it accountable directly to Ofcom. Since 2015, Ofcom has been the backstop adjudicator of complaints about the BBC, but only after complainants have been taken through the BBC’s internal procedures, which can take many months. Of course, directing all complaints about the BBC’s output to Ofcom will greatly increase its workload. However, this extra cost can be borne by the licence fee as the BBC itself will be able to shed scores of unnecessary jobs.

Whatever its other failings (notably, in its oversight of public service broadcasting), Ofcom is widely respected for the objectivity and scope of its complaints procedures. It also has powers to order inquiries into broadcaster behaviour connected to the making of programmes. Ofcom’s predecessors in the commercial sector – the Independent Television Commission and the Independent Broadcasting Authority (IBA) – had similar powers. The Windlesham/Rampton inquiry into ‘Death On The Rock’, which examined the deaths of three Provisional IRA members in 1988, was undertaken at the behest of the IBA. I was the director of programmes who authorised transmission of that programme .

Broadcasting governance in the commercial sector is both vastly superior to the BBC model, and completely non-contentious as an issue. Why the BBC clings to its outdated internal structure is hard to understand. Given that Ofcom is there as an appeal court anyway, the best the BBC can hope for if a complainant invokes Ofcom is that the regulator’s response is no different to its own.

The Dyson report is the ideal opportunity for the BBC to embrace the transparency that being accountable to Ofcom affords. Next year’s mid-term review of the BBC Charter is the ideal moment to grasp that option.