Thursday, May 07, 2020

The very model of a modern scientific man

Your humble Devil was thoroughly amused by Neil Ferguson's fall from grace, and is very pleased to have found the time to outline Ferguson's manifest and repeated failings—for posterity, you understand.

And it seems that the media scrutiny of Ferguson's sordid personal life has put the wind up some of the government's scientific advisers...
One scientific adviser to the government said Ferguson’s resignation had created “an awful lot of concern” and that the mood in the community was “very depressed”. The events revealed how university academics who lent their advice to government were having to cope with an increasingly difficult situation, the adviser added.

“He’s an academic researcher. He doesn’t make decisions. He’s not paid for any of this. We are being drawn into a political situation which is very unpleasant,” they said.
Oh dear, what a pity—how sad. Hey, science bods—you know you might avoid this kind of media scrutiny? Yes, that's right: don't take political appointee jobs and refuse the fat taxpayer-funded salaries. You don't want to be caught up in politics? Then don't play at politics.

Are these people simple, or what?

In the meantime, a programmer has finally got around to looking at the Imperial College modelling code—and her assessment is not pretty. [Emphasis mine—DK]
I wrote software for 30 years. I worked at Google between 2006 and 2014, where I was a senior software engineer working on Maps, Gmail and account security. I spent the last five years at a US/UK firm where I designed the company’s database product, amongst other jobs and projects. I was also an independent consultant for a couple of years.
So, a reasonably credible source then. I wonder what she found? Let's cite some choice extracts from the assessment, shall we?
Clearly, Imperial are too embarrassed by the state of it ever to release it [the original model code] of their own free will, which is unacceptable given that it was paid for by the taxpayer and belongs to them.

Due to bugs, the code can produce very different results given identical inputs. They routinely act as if this is unimportant.

[...]

Investigation reveals the truth: the code produces critically different results, even for identical starting seeds and parameters.

I’ll illustrate with a few bugs. In issue 116 a UK “red team” at Edinburgh University reports that they tried to use a mode that stores data tables in a more efficient format for faster loading, and discovered – to their surprise – that the resulting predictions varied by around 80,000 deaths after 80 days...

[...]

Because their code is so deeply riddled with similar bugs and they struggled so much to fix them that they got into the habit of simply averaging the results of multiple runs to cover it up… and eventually this behaviour became normalised within the team.

[...]

Although the academic on those threads isn’t Neil Ferguson, he is well aware that the code is filled with bugs that create random results.

[...]

Imperial are trying to have their cake and eat it. Reports of random results are dismissed with responses like “that’s not a problem, just run it a lot of times and take the average”, but at the same time, they’re fixing such bugs when they find them. They know their code can’t withstand scrutiny, so they hid it until professionals had a chance to fix it, but the damage from over a decade of amateur hobby programming is so extensive that even Microsoft were unable to make it run right.

[...]

The Imperial code doesn’t seem to have working regression tests. They tried, but the extent of the random behaviour in their code left them defeated.

[...]

Much of the code consists of formulas for which no purpose is given. John Carmack (a legendary video-game programmer) surmised that some of the code might have been automatically translated from FORTRAN some years ago.

[...]

This code appears to be trying to calculate R0 for “places”. Hotels are excluded during this pass, without explanation.

[...]

R0 is both an input to and an output of these models, and is routinely adjusted for different environments and situations. Models that consume their own outputs as inputs is problem well known to the private sector – it can lead to rapid divergence and incorrect prediction.

[...]

Despite being aware of the severe problems in their code that they “haven’t had time” to fix, the Imperial team continue to add new features; for instance, the model attempts to simulate the impact of digital contact tracing apps.

Adding new features to a codebase with this many quality problems will just compound them and make them worse.
Yikes. And the conclusion...?
All papers based on this code should be retracted immediately. Imperial’s modelling efforts should be reset with a new team that isn’t under Professor Ferguson, and which has a commitment to replicable results with published code from day one.
Well, I think that this fucking debacle goes some way to explaining why Ferguson's models have been such a complete and utter failure from the get-go.

And let us remind ourselves that our government were stupid enough to believe this fucking team of charlatans, and that they are busy cratering the economy on the strength of a computer "model" (I use the term advisedly) that produces complete garbage.
On a personal level, I’d go further and suggest that all academic epidemiology be defunded. This sort of work is best done by the insurance sector. Insurers employ modellers and data scientists, but also employ managers whose job is to decide whether a model is accurate enough for real world usage and professional software engineers to ensure model software is properly tested, understandable and so on. Academic efforts don’t have these people, and the results speak for themselves.
Indeed they do.

It's odd though. There's something at the back of my head, something niggling at me—a real sense of familiarity about this situation...

Where ales have we encountered a commentary on a computer model that has huge political and economic consequences but which, having been written by a bunch of amateur fuck-wits, provides absolute fucking garbage...?

Oh yes—it's Harry again.

Do you remember, in November 2009, that there was a leak from the University of East Anglia's Climate Research Unit (CRU)? Most of the media spent their time exposing the dirty tricks revealed in the emails between the "scientists" who are the main proponents of the Catastrophic Anthropogenic Climate Change (CACC) theory—including using collusion and blackmail to prevent dissident papers appearing in "reputable" journals.

But what was less widely reported was that, along with the emails, the computer "models" (again, advisedly) were released—alongside a very long commentary by an unfortunate programmer who was tasked with making sense of them.

The programmer was called Ian Harris, and his HARRY_READ_ME.txt file was, for those of us who like to delve into these things, an absolute treasure trove—revealing the incompetence of these so-called "scientists", and the utter invalidity of their much-vaunted "climate models".

And here we are again: with a government fucking our economy and freedoms, all on the basis of useless, garbage-spouting models.

Dear Boris (and every other government): for fuck's sake, stop giving any credence at all to these models. Models are not evidence and they are not science: even the most well-coded model would be nothing more than a theory—and, as we have seen with both COVID-19 and CACC, the people building said programmes are nowhere near being competent.

These so-called scientists are not: they are hobbyist coders (and bad ones at that). And where they attempt to sell their models as reliable, these people are frauds—and they should be prosecuted. If not, then a class-action lawsuit might find a large number of backers—especially if a case carries the prospect of personally bankrupting Neil Ferguson. Certainly, I would happily donate.

UPDATE: Tim Almond explains why "it's stochastic" is no excuse at all, and the Streetwise Professor is as incensed as your humble Devil...

Wednesday, May 06, 2020

Vaccine futility

As some of you may know, your humble Devil studied (in a desultory way) Microbiology at university, and has always had a special interest in human pathogens. So, today I would like to talk about why a "permanent" vaccine for the novel coronavirus is pie in the sky; and cannot, as Boris keeps maintaining, be a pre-condition of removing the lockdown—not, at least, on a permanent basis.

The first point to make is that the novel coronavirus (nCV) is a single-stranded RNA virus (more about that in a moment) and we do not have an effective vaccine against any coronavirus.

Nor do we have, as some have claimed, a permanent vaccine against 'flu: what happens in that case is that predictions are made about what 'flu variants are likely in that year, and then people are given a jab that aims to cover all of them. Sometimes this works (e.g. UK 'flu deaths in 2018/19 were circa 1,700); sometimes the predictions are wrong, or a new strain emerges, that is sufficiently different that the vaccine is largely ineffective, and deaths soar (e.g. UK 'flu deaths in 2014/15 were in excess of 28,300) [Source for both figures.]

To start with, let's sketch, at a fairly high level, why this should be.

High mutation rates


The first issue to address is why single-stranded RNA viruses mutate so quickly...
  • advanced cells (such as in humans) reproduce using DNA which, as we all know (right?), is a tightly wound, highly-stable, double-stranded helix;
  • when the cell reproduces, the DNA helix unwinds and cell machinery moves along the strand to duplicate it;
  • the second DNA strand is then used to check and validate that the original one has been duplicated correctly;
  • this validation step is why DNA-driven cells mutate incredibly rarely.
In a single-stranded RNA virus, not only is RNA less stable than DNA, there is no second strand—and thus no validation mechanism: this leads to far more transcription errors on replication, i.e. a far higher mutation rate.

Given that viruses replicate many millions of times, then one can see that mutations will happen pretty frequently. As we can already see in the new coronavirus...
Researchers in the US and UK have identified hundreds of mutations to the virus which causes the disease Covid-19.

But none has yet established what this will mean for virus spread in the population and for how effective a vaccine might be.
So, why would these mutations matter?

Antigens and antibodies


Again, very broadly speaking, human cells are not smooth: they are studded with structures that enables resources to be attached to, and absorbed into, them—vitamins, minerals, oxygen, enzymes, etc. Pathogens, including the coronavirus, attach onto one or more of these structures in order to pass into the cell.

In order to do this, viruses develop their own structures—in the case of coronaviruses, these are the S-spike structures that give them their name—which act as the "key" to let them into the cell. These "keys" are known as "antigens".

When combating pathogens, your body manufactures "antibodies": these bind to the pathogens antigens and they are highly specific. If the shape of the virus antigen changes in any significant way, then the existing antibodies will no longer work—and your body has to start all over again.

How vaccines work


A vaccine works by introducing a something that looks exactly like the virus, but does not do the same damage, into the body. The "shell" has the viral antigens, but the dangerous bit—the RNA—is damaged, disabled, removed or otherwise "attenuated". The body then recognises this fake virus as a foreign invader, reads its antigens, and builds antibodies that will attack anything that looks like it—all without you getting ill.

(Actually, whether or not you have a likelihood of getting ill or not rather depends on whether the virus uses negative-sense or positive-sense RNA—there is a higher likelihood of illness with a positive-sense RNA virus vaccine, such as nCV (influenza viruses are negative-sense)—but I am not going to get into that just now.)

However, if the viral antigens have mutated significantly enough that the existing antibodies do not recognise it, but little enough that the antigen can still grant it access to the cell, the vaccine will no longer work.

Summarising the problem


So, with sufficient changes in antigens, the vaccine-generated antibodies will not work, and those infected will suffer the ill-effects of the virus (many of which are actually caused by your body's reaction—particularly dangerous, and often fatal (particularly in COVID-19 patients), is a "cytokine storm").

Viruses with a high mutation rate (particularly single-stranded RNA viruses) tend to lead to a vast number of significant changes in antigens—rendering vaccines temporary at best, as with 'flu.

Coronavirus is a single-stranded RNA virus—so no vaccine is going to be permanent.

What now


As such, this new variant will be with us for a very long time—will we shut down for weeks every time that it comes back around? Well...

In my next post, I shall discuss various treatment pathways. But, please, do not think that "a vaccine" is going to be the way out of this mess—and if Boris and co start to make that a condition of lifting lockdown, call them out on it.

They are either dangerously ignorant, or lying. So, no change there then.

Neil "lockdown fucking" Ferguson finished on the face of her it

Neil Ferguson, the useless Imperial College charlatan whose ridiculous modelling—which predicted 500,000 deaths from COVID-19 in the UK—led to the lockdown, has resigned from the shadowy Scientific Advisory Group for Emergencies (SAGE) government advisory committee after inviting his married lover over for a good seeing-to on a couple of occasions.
Prof Neil Ferguson has quit as a government adviser on coronavirus after admitting an "error of judgement".

Prof Ferguson, whose advice to the prime minister led to the UK lockdown, said he regretted "undermining" the messages on social distancing.

It comes after the Daily Telegraph reported a woman had visited his home twice during lockdown.
Aaaaahahahahaha! Hahahahaha...
His modelling of the virus's transmission suggested 250,000 people could die without drastic action.
No, it didn't: his modelling estimated that 510,000 people would die without drastic action, and he doubled down on this figure very clearly.

Neil's past successes


Okay—enough hilarity. On a serious note, and as usual, it's one rule for Neil and his mates, and another for us. So, I am thoroughly glad that he has resigned from SAGE: the downside here is that this scare-mongering fuckwit is causing huge amounts of damage—as neatly outlined some time ago by Steerpike in The Spectator (£)...
In 2005, Ferguson said that up to 200 million people could be killed from bird flu. He told the Guardian that ‘around 40 million people died in 1918 Spanish flu outbreak… There are six times more people on the planet now so you could scale it up to around 200 million people probably.’ In the end, only 282 people died worldwide from the disease between 2003 and 2009.
Well, look, epidemiological modelling is quite difficult, and computer models were in their infancy. Neil will learn next time though, right?

In 2009, Ferguson and his Imperial team predicted that swine flu had a case fatality rate 0.3 per cent to 1.5 per cent. His most likely estimate was that the mortality rate was 0.4 per cent. A government estimate, based on Ferguson’s advice, said a ‘reasonable worst-case scenario’ was that the disease would lead to 65,000 UK deaths.

In the end swine flu killed 457 people in the UK and had a death rate of just 0.026 per cent in those infected.
Anyone can make a mistake though. I mean, despite the similarity of bird flu and swine flu, these were completely different circumstances and a chap can't be asked to get it right every time, eh?
In 2001 the Imperial team produced modelling on foot and mouth disease that suggested that animals in neighbouring farms should be culled, even if there was no evidence of infection. This influenced government policy and led to the total culling of more than six million cattle, sheep and pigs – with a cost to the UK economy estimated at £10 billion.

It has been claimed by experts such as Michael Thrusfield, professor of veterinary epidemiology at Edinburgh University, that Ferguson’s modelling on foot and mouth was ‘severely flawed’ and made a ‘serious error’ by ‘ignoring the species composition of farms,’ and the fact that the disease spread faster between different species.
Pffft. Look, Neil was an epidemiologist not a bloody farmer, for god's sake—how could he be expected to know that there are different species of cattle, or that the infection rates might be different.

Besides, it was really difficult for Neil to code that. His research student would have had kittens trying to make that work in his Python script.

Neil will get it right next time—just you see if he doesn't.
In 2002, Ferguson predicted that between 50 and 50,000 people would likely die from exposure to BSE (mad cow disease) in beef. He also predicted that number could rise to 150,000 if there was a sheep epidemic as well. In the UK, there have only been 177 deaths from BSE.
Oh. Right. He doesn't.

Ah, well, surely—with everything that he has learned, Neil will get it right this time...
Ferguson’s disease modelling for Covid-19 has been criticised by experts such as John Ioannidis, professor in disease prevention at Stanford University, who has said that: ‘The Imperial College study has been done by a highly competent team of modellers. However, some of the major assumptions and estimates that are built in the calculations seem to be substantially inflated.’
[...]
On 22 March, Ferguson said that Imperial College London’s model of the Covid-19 disease is based on undocumented, 13-year-old computer code, that was intended to be used for a feared influenza pandemic, rather than a coronavirus.
I mean... I guess that 510,000 does seem quite high. But who really knows...?
Has the Imperial team’s Covid-19 model been subject to outside scrutiny from other experts, and are the team questioning their own assumptions used?
Well that is a good question. As a matter of fact, some weeks after releasing their models, Imperial College London did, indeed, Open Source their COVID-19 modelling code on GitHub—which is a decent step towards transparency.

The trouble is, what they did not release were the configuration variables—the assumptions that they made when they ran the model. Which means that no one can properly replicate and validate their outcomes. Which seems to be par for the course in the scientific community these days.

So, what to do?

The Swedish model


Well, as we know, Sweden has not been following the same lockdown model as ever other country: they have introduced social distancing measures (along similar lines adopted by the UK government—before Neil came along with his Doomsday predictions) and their outcomes are... well... pretty good. Now, they are a sparsely populated country, for sure, but even in cities such as Stockholm, the death rate is comparatively low and herd immunity is predicted within a couple of weeks.

So, the University of Uppsala, in Sweden, came up with a good wheeze: into Neil Ferguson and team's model, they fed in variables that applied to Sweden—to see if the outcome matched reality. Can you guess what comes next...?
The Uppsala team’s presentation appears to closely follow the ICL approach. They presented a projection for an “unmitigated” response (also known as the “do nothing” scenario in the ICL paper), then modeled the predicted effects of a variety of policy interventions. These included staying the course on the government’s alternative approach of remaining open with milder social distancing guidelines, as well as implementing varying degrees of a lockdown.
So far, so good.
The model stressed its own urgency as well. Sweden would have to adopt a lockdown policy similar to the rest of Europe immediately if it wished to avert catastrophe. As the authors explained, under “conservative” estimates using their model “the current Swedish public-health strategy will result in a peak intensive-care load in May that exceeds pre-pandemic capacity by over 40-fold, with a median mortality of 96,000 (95% CI 52,000 to 183,000)” being realized by the end of June.

Their proposed mitigation scenarios, which followed lockdown strategies similar to those recommended in the ICL paper and adopted elsewhere in Europe, were “predicted to reduce mortality by approximately three-fold” while also averting a catastrophic failure of the Swedish healthcare system.
So, according to the modelling, Sweden should also have locked down in order to "avert catastrophe"—what could be clearer?
The Swedish model laid out its predicted death and hospitalization rates for competing policy scenarios in a series of graphs. According to their projections [...], the current Swedish government’s response – if permitted to continue – would pass 40,000 deaths shortly after May 1, 2020 and continue to rise to almost 100,000 deaths by June.
Oh my god—the poor Swedes! Quick, quick, lock down every bloody thing and... Wait, what?
So how is the model’s projection performing? Sweden’s government stayed the course with its milder mitigation strategy. As of April 29th, Sweden’s death toll from COVID-19 stands at 2,462, and its hospitals are nowhere near the projected collapse.
Oh. Right.

So, Neil and co's model over-projects the terrible consequences, and has an in-built bias towards a particular course of political action? Gosh.

How incredibly surprising.

(For a special gold star, can we think of any other poorly documented, immensely flawed computer models that behave in a similar way, children...?)

Let's be fair


On the other hand, Neil and his cronies did predict that a full-on lockdown might reduce deaths to around 20,000; this is, in the UK, proving to be on the low side (with COVID-19 deaths standing at 29,427, at time of publication)—although there are complications in reporting, which I shall discuss another time.

But, was the 510,000 figure ever credible? Well... Your humble Devil did call this crisis wrong, thinking that it would blow over: however, given the profile of the vast majority of deaths, I think that half a million plus deaths remains a gross over-estimation.

A summary of Neil's career


I think that it is fair to say that Neil Ferguson's most high profile epidemiological models have been failures—and hugely expensive failures at that.

Or, at least, hugely expensive for the taxpayer—no doubt incredibly lucrative for Neil and his buddies at Imperial College London. It is certainly true that, despite his very public fuck-ups, Neil remains employed.

And whilst it is amusing to see any arsehole brought down by personal malefactions—especially one so hypocritical and hubristic as this—it is hugely unsatisfying to see Neil resign for opportunistic personal fucking rather than being sacked, in disgrace, for his many eye-wateringly expensive professional fuck-ups.

There is, of course, still time.

The BBC on Ferguson


The BBC's summary of Neil Ferguson's career (or "Analysis" as it is hilariously entitled) is written by "health and science correspondent", James Gallagher—who, unusually for a BBC science journalist, actually has a science degree (biology).

I reproduce it in full, below.
Prof Neil Ferguson is one of the world's most influential disease modellers.

He is director of the MRC Centre for Global Infectious Disease Analysis.

The centre's mathematical predictions advise governments and the World Health Organization on outbreaks from Ebola in West Africa to the current pandemic.
It was that group's work, in early January, that alerted the world to the threat of coronavirus.

It showed hundreds if not thousands of people were likely to have been infected in Wuhan, at a time when Chinese officials said there were only a few dozen cases.

But he shot to public attention as "Professor Lockdown".

In mid-March, the maths showed the UK needed to change course or a quarter of a million people would die in a "catastrophic epidemic".

Those calculations helped transform government policy and all lives.
I think that you will agree that this is most certainly not "analysis"—and nor is it in any way impartial.

Does anyone else reckon that James Gallagher and Neil Ferguson are good friends...?

DK's final word


Ferguson is a proven failure—he certainly does not deserve the encomium delivered by Gallagher.

The eventual outcome of this pandemic is unknown; plus, of course, it is difficult to "prove" a counter-factual. Nonetheless, the Swedish experiment seems to suggest that, once again, the UK government has been persuaded into taking hugely expensive and illiberal measures based on wildly pessimistic models supplied by a man who has a history of producing wildly pessimistic models.

The decision to believe Ferguson, of course, must lie with the government: however, I believe that said government should release the minutes of all meetings concerning these models, and whether there were dissenting voices—perhaps from those who knew of Ferguson's past fuck-ups. If there were no dissenting voices, then we must ask "why not?"

In the meantime, Ferguson sexual peccadilloes may have tarnished his personal probity—but his professional reputation remains inexplicably intact.

One can only wonder as to why.

NHS Fail Wail

I think that we can all agree that the UK's response to coronavirus has been somewhat lacking. In fact, many people asserted that our de...