The BMJ has just published a cracking series of articles online, including a revised Cochrane Collaboration review, that throw out some hefty challenges to influenza policy makers, pharmaceutical companies, researchers, governments and drug regulators.
The articles raise many questions about the evidence base that has been used to guide influenza policies, especially the billion of dollars invested in stockpiles of antiviral drugs like Tamiflu, and the adequacy of drug regulatory processes.
I’ve summarised the findings in this article for today’s Crikey bulletin. Below is some more of the background – this is a fascinating story at many levels – as well as an edited transcript of an interview conducted this morning with Professor Chris Del Mar, one of the Cochrane review’s authors.
In 2005, the Cochrane Collaboration conducted a review of the use of antiviral drugs in influenza, which found, amongst other things, that oseltamivir (Tamiflu) was effective in reducing important complications of influenza such as pneumonia. This conclusion was published in 2006 and again when the review was updated in 2008.
But an alert Japanese paediatrician Keiji Hayashi wasn’t convinced. Japan has been one of the world’s biggest users of Tamiflu, and concerns have been raised there about side effects, including rare but severe ones affecting children.
In July of this year, Hayashi contacted the Cochrane reviewers with his concerns. He noted that their finding was based upon a single peer-reviewed study by Kaiser et al, published in Archives of Internal Medicine in 2003. The Kaiser study itself had meta-analysed 10 manufacturer funded trials from the late 1990s, of which only two were published in peer reviewed journals. The remaining eight were apparently either unpublished or published only in abstract form.
He was also concerned that four of the review’s authors were employees of F Hoffman-La Roche Ltd; one was a paid consultant to the company at the time of the Kaiser analysis; and only one had no financial relationship to Roche.
Hayashi suggested that the unpublished trials were central to the review’s conclusion about oseltamivir’s ability to reduce lower respiratory tract complications of influenza, and challenged the reviewers to “appraise the 8 trials rigidly.”
So began what reads very much like a detective story. The Cochrane reviewers contacted the researchers involved in the Kaiser publication, and Roche but were unable to verify the data cited in the Kaiser study. They ran into multiple dead ends, and also found plenty of mystery and unresolved questions, including a series of inconsistencies in how the data had been reported.
It’s worth noting that the Kaiser study has been widely cited by Roche as proof of Tamiflu’s benefits, and also by policy makers involved in antiviral stockpiling and related pandemic policies.
One of the Cochrane reviewers, Peter Doshi, says their investigations discovered, however, “that the public evidence base for this global public health drug is fragmented, inconsistent, and contradictory”.
“We are no longer sure that oseltamivir offers a therapeutic and public health policy advantage over cheap, over the counter drugs such as aspirin,” he said. “If the public is to trust in public health policies, the scientific basis informing knowledge of the harms and effects of those interventions must be public and open to independent analysis.”
It’s powerful stuff and the BMJ series of articles are well worth reading (some are freely available online, though you will have to pay or be a subscriber to access others).
Meanwhile, here is an edited transcript of a rather rushed phone interview conducted this morning with Professor Chris Del Mar, who has issued a mea culpa on behalf of the Cochrane Collaboration. Del Mar is Coordinating Editor of the Cochrane Acute Respiratory Infections Group, and Dean of the Faculty of Health Sciences and Medicine at Bond University.
Q. What are the implications of the review?
CDM: We need to insist on looking at data more critically when we’re basing quite important policy decisions which cost billions. The truth is that we have to take a lot on trust and when we analysed the data very carefully, we found insufficient hard data to enable us to make the conclusions that everyone else has drawn from that (Kaiser) review.
We made a mistake in the Cochrane Collaboration by taking on trust the review, which included several studies which remain unpublished. Because they are rather heterogenous, we can’t draw conclusions properly without looking at the studies in their entirety. At the very least we need to see the numbers of patients in each trial, who they were.
Q. Were the difficulties in obtaining the data the result of a conspiracy or a cock-up?
CDM: The drug company hasn’t been forthcoming with us. We’ve been asking them for the data since August. We started by talking to Kaiser the professor in Switzerland who wrote this piece, and he told us he didn’t have the data and referred us to Roche who just gave us the run around basically. We said we had a deadline of the end of the year, and now Roche is saying, ‘why won’t they work with us?!” They’re saying, ‘it’s all fine, trust us, it’s OK”.
These trials should be done publicly, very openly, very transparently. It’s one thing to have a drug you might want to purchase yourself, but when it’s a drug that’s going to be bought by the warehouse, we need a different level of transparency.
Q: Have governments been conned?
CDM: They’ve been naïve. In a way its our fault, we’ve been part of this. The Cochrane Collaboration took the summary of these data on trust and it wasn’t until a Japanese person Hayashi wrote to us, saying you’re wrong, you’re making some terrible assumptions about these data and we said, ‘he’s right, we must go and look at this’.
We were going to do the update anyway. The Cochrane Collaboration in the first half of the year said we’ve got to respond to this influenza pandemic and update whatever we can in the Cochrane Collaboration, so the world has the best available evidence. The Cochrane Collaboration has been criticised for being slow, so we’ve said we’ve got to do this fast.
So we started the review. Simultaneously, the National Institute for Health Research in Britain asked us to update it and the NHMRC had a funding round. So we cobbled together little bits of money to make this happen. We took on board the Hayashi criticism and he was quite right, we shouldn’t have trusted that data. It may be that we will get individual patient data from Roche and we will be able to meta-analyse it.
Q: What should the Australian Government do?
CDM: It’s very easy to be smart after the event. This pandemic influenza doesn’t seem to be more dangerous through the population as a whole than seasonal flu. There is no increased overall deaths, although there are more in different parts of the population and less in others. In seasonal flu, it’s mostly the elderly at risk, and with this it’s younger fat people and pregnant women and people with respiratory illness, so some worrying sub-groups. But the overall threat to the population isn’t a lot different to any other flu season.
We have to ask what the opportunity cost is of having warehouses full of drugs of modest effect We probably should use the drug in seriously ill people even though the data is not convincing. But we had a stage here where the drug was being handed out by non doctors. In Victoria, the nurses were handing it out as a public health measure. You could ring up and pick it up. So this is quite a different level of drug use. In the scramble to try and deal with what might have been a much more serious threat, it was probably a reasonable thing to do but now we’ve had time to analyse the data and the risks with more sang froid, my feeling is we probably ought to start tinkering with the policy. We should resesrve oseltamivir for people who are seriously ill or those in high risk groups. I don’t think we should use it as a public health measure. It makes very little difference to influenza like illness.
Q. What about the safety concerns around Tamiflu?
CDM: There are some odd reports of patients getting ill in some of the trials that don’t seem to have made it into the FDA database. We had to invoke Freedom of Information to get some of the data. There’s a Japanese dataset as well. More than 70 per cent of the world’s Tamiflu has been eaten in Japan. They’re worried about some of the psycho-neurological issues, particularly in kids. We just can’t get our hands on the data properly, so this needs to be better managed. As independent researchers we couldn’t find it.
Q. The BMJ seems to be running very strongly on this review?
CDM: The BMJ’s sense of outrage is that we just don’t have enough information for something that is a public resource that we’re paying a lot of money for.
Q. What could come out of this?
CDM: We need to have the rules changed a bit. Its very complicated for the commercial sector because they want to protect their commercial interest and product so they tend to be very secretive, which is understandable, to protect shareholders. But at the same time they will have to release enough information. They’re going to have to be transparent.
We have a situation now where two-thirds of the patients in Kaiser data from 2003 have never been published. And that has been cited and used to guide recommendations. So we need to do something different about that. The relationship between pharma and regulators has to change.
The TGA needs to respond to what’s happened. In a way it’s the fault of us guys. The TGA relies on us academics and experts. This is a mea culpa for the Cochrane Collaboration. The Cochrane Collaboration is supposed to be the gold standard for synthesising information and we screwed up. We took this at face value and we probably shouldn’t have. By our own standards we failed. There was one slip up but is there just one slip up or is there is more systemic slip up?
Q. What about other Cochrane reviews in the influenza area?
CDM: Tom Jefferson (lead author on this review) has done three important reviews in this area. Including one of the influenza vaccine where the benefit is difficult to determine in the elderly. We’ve also done a review about physical barriers, such as masks, gloves, handwashing which seems to be incredibly efficient. It seems to us the world has gone loopy that it only ever talks about Tamiflu and vaccines, and much the most important things are barrier methods. We’d much prefer to see handwashing stuff all over the place rather than warehouses full of drugs that has modest effects.
It would be reasonable now to mount a really big, publicly funded trial with some placebos and measure actual effect. Governments should now fund that. Roche is screaming that would be unethical but I don’t think that would be. We probably also need that with the influenza vaccine. Vaccine trials are tricky because it varies from season to season. We do need more data. We don’t have enough data. In general we should be doing more testing of things. If we’re rolling out big policies we should be doing this in the light of gathering more data.”
You can read Roche’s response to the review and BMJ investigations here and here (full access is not free). The company is rejecting the review’s findings in media statements, as per this Financial Times report. Another report that may be of interest, from the LA Times,