Agnes Vitry, Senior Research Fellow, School of Pharmacy and Medical Sciences, University of South Australia, has sent in this detailed, thoughtful response to the recent Croakey survey on the issues surrounding the controversial, commercially-funded Australian and NZ guidelines for blood clot prevention (for more background, see previous Croakey posts).
Vitry’s comments are well worth a read:
1. Should health departments, hospitals, safety and quality groups be reviewing their support for these particular guidelines in response to the concerns raised in the MJA and elsewhere? If so, are there any particular actions you would like the various agencies to take?
Alasdair Millar describes a well researched case-study of what may happen when appropriate development procedures for clinical guidelines are not followed. Unfortunately, it’s not a rare situation.
In my research on the quality of Australian guidelines, we found that guidelines that followed the NHMRC processes scored significantly better than those not approved by the NHMRC in all domains except for editorial independence and clarity and presentation (the declaration of conflicts of interests was not a requirement for NHMRC-approved guidelines when we did our study). Low-quality scores in the rigour of development domain raise concern about guideline validity.
In contrast with Prof Chris Baggoley’s opinion, I don’t believe that guidelines that don’t follow appropriate processes are preferable to the absence of guidelines. Promotion of invalid guidelines may result in ineffective or unsafe treatments, or inefficient use of resources and it is the point that Alasdair Millar is making.
First, we must make clear that Alasdair’s criticisms focus on the use of preventive treatment for medical patients. He is not dismissing the usefulness of preventive treatment in many surgical patients especially in high risk surgical procedures. There would be no issue if the guideline development process was as simple as implied by Prof Baggoley: you have a good look at the scientific evidence and other international guidelines to produce your own guidelines. Unfortunately, this is not the case.
First, the quality of the evidence and the uncertainty around the validity of the results need to be reflected in the wording and grading of the recommendations. There are a lot of uncertainties around the evaluation of preventive treatment of medical patients for the reasons explained very well by Alasdair.
Clinical trials tend to use asymptomatic DVT as the main assessment outcome and the value of this surrogate has been questioned because of the discrepancy between high incidence of venographic DVT and low risk of clinical outcomes such as death, e.g. in elective hip and knee replacement surgery (see many references here).
Second, guideline developers have to make decisions about the generalisability of the results to sub-groups of patients different of the patients enrolled in the trials. There are two options here that guideline developers may consider: either you match your recommendations with the inclusion criteria of the trials (“evidence-based” option favoured by Alasdair).
Or you believe that the benefits shown for particular subgroups in the clinical trials could be extrapolated to other groups at high risk of DVT as shown in epidemiological studies (option favoured by the Australian Working Party).
This later option could be a huge gamble as demonstrated recently for other guidelines. The most recent example where this later option has been shown to be wrong was the recommendation in many guidelines to give low-dose aspirin to all type 2 diabetes patients based on the assumption that patients with type 2 diabetes will benefit from low-dose aspirin as type 2 diabetes is a very important risk factor for cardiovascular diseases but in the absence of good trial evidence showing that aspirin will benefit specifically patients with type 2 diabetes. Recent evidence has shown that it was not true.
Patients with type 2 diabetes without additional risk factors are more likely to be harmed than to be saved by aspirin. The point that Alasdair wants to make is that by widening the indications of preventive treatment for medical patients without good evidence you may end up harming people that you want to protect.
At the end of the day, making recommendations involve value judgements: is the expected benefit worth the potential risks? It is an area that the guideline developers must make much clearer than it is currently done. Health professionals need to be able to say clearly to their patients which are the risks of the different options.
It is where the conflict of interests (funding provided by the drug company who has a financial interest in the medicine recommended in the guideline, personal conflicts of interest of working party members) come into play.
How can health professionals and the public know whether the Australian Working Party made a “genuine” decision when they decided to widen the indications for preventive treatment of medical patients or whether they were consciously or unconsciously influenced by their ties with pharmaceutical industry?
Research has shown clearly that declaration of conflicts of interest will not be “the” answer. It is also the point made by Psaty for the interpretation of industry-funded clinical trials in his recent JAMA editorial.
I believe that in the case of guidelines, it is the same as for clinical trials: you need to have a critical look at the guidelines! It is what Alasdair did very nicely as a very concientious health professional. Also thanks to the MJA to have allowed the publication of his dissenting view. In my experience, it is not so easy to hold “dissenting” views in Australia as everybody knows everybody and you don’t want to hurt your friends/colleagues/employers…but healthy debates need to happen!
I would suggest that agencies that have endorsed the Working Group guidelines review as a matter of urgency their recommendation of treatment of medical patients over 60. These patients may be at high risk of DVT but also at high risk of bleed. What is the strength of the evidence for this group of patients?
2. Should such agencies also review their approach to endorsing and disseminating guidelines generally?
The issues for Australia are how to develop good guidelines with limited resources?
Professional organisations (including the Working Party) are given a hard time: they are criticised for the quality of their guidelines if they don’t follow the NHMRC processes but they cannot get appropriate funding for making guidelines in most cases. I am so admiring of organisations such as the Australian Cancer Network which try so hard to produce NHMRC-approved guidelines with so little financial (and emotional!) support!
So which are the ways forward? We proposed some in our article. Adaptation of international guidelines is an option (chosen by the Working Party) that avoids duplicating efforts when high-quality international guidelines are already available on similar topics.
However, this needs to be done well and there is no validated process for adapting guidelines produced in one cultural and organisational setting for use in another. NHMRC guidance in this area would be welcome to support the efforts of professional societies.
Second, a question that may need to be debated is whether the NHMRC methodological requirements for guideline developers have become too tough and too bureaucratic (see here for example) by comparison with so little attention devoted to the important issue of conflicts of interest?
Without appropriate public funding, guideline developers don’t have many other options other than relying on drug company funding with all the dangers that it involves.
In response to our article (and our criticisms of their guidelines), the Heart Foundation has called for a strong and robust national framework for the development and implementation of guidelines.
At the NHMRC, NICS will be in charge of the clinical guidelines from July this year. Will they have more resources than their predecessors? What are their plans? Do they plan to consult publicly the different stakeholders?
In the meantime, agencies need to be very careful before endorsing guidelines. The best advice I would give is to do what we do at the Australian Medicines Handbook, have a look around for other “good” independent (no industry funding) international guidelines. Some such as the ones produced by the UK organisations, NICE and SIGN, have very good development processes.
Then, you need to have a closer look at the areas where there may be a lot of clinical uncertainty or conflicting recommendations between guidelines. It’s not to say that “international consensus” offers some guarantee as most guidelines could be wrong at the same time (see aspirin example above) and it may be so hard to be the “dissenting” voice and to be able to make your disagreement published in medical journals!
3. Should health and medical groups producing guidelines accept funding from interests with a commercial stake in the guidelines’ recommendations?
Health and medical groups producing guidelines should not accept funding from interests with a commercial stake in the guidelines’ recommendations. The risk of bias is much too high and there are two many examples of biased guidelines because of industry funding. It is already so difficult for independent experts to make an appropriate assessment of the scientific evidence because of the importance of publication bias. In my opinion, it is better not to do “Australian” guidelines (but critically assess independent international guidelines) than to rely on industry funding.
I could make a long list of recommendations in Australian non-NHMRC guidelines that don’t stand a critical look.
4. Would you like to make any other comments?
It is time for professional and charity organisations in Australia and clinical experts involved in these organisations to review their policies with regards to industry funding.
Industry funding does not only increase dramatically the risk of bias but it increases the risk of suspicion from the public. It is not so easy to do in a country like in Australia where “partnership with industry” has been presented as the best way forward to deliver the best health and industry outcomes for the country (see the National Medicines Policy).
It’s time to have a good discussion on what is acceptable in partnership, what we cannot expect from partnership, etc. I believe that this is an urgent issue to look at for the newly created National Medicines Policy Committee.
Personally, I don’t think that we can take the risk to have the pharmaceutical industry involved in the production of medicine imformation for consumers and health professionals.
Professional and charity organisations which accept and request such funding are at high risk of becoming compromised at one stage or another. They are at high risk of being dragged in a position (most often unconscientiously and that is the problem) where they would recommend harmful treatment or treatment to a much wider population than justified by medical evidence.