Regular readers may remember there was some vigorous discussion at Croakey earlier this year after a SA study comparing the outcomes for homebirths and hospital births generated misleading headlines. The coverage largely reflected the slant put on the study by the Medical Journal of Australia’s press release and an accompanying editorial by the AMA president, Dr Andrew Pesce (see here, here and here, for example of the previous Croakey posts).
Well, the discussion didn’t stop there. I am writing a piece for the Crikey bulletin on related issues, some of which are also aired in the current issue of the BMJ (extract available here, but the full article costs).
Below are three sets of comments that helped inform the BMJ and Crikey articles. Firstly an email comment from Drs Steven Woloshin and Lisa Schwartz from the Center for Medicine and the Media at the Dartmouth Institute for Health Policy and Clinical Practice in New Hampshire. They are international leaders in efforts to improve media coverage of health and medicine.
Secondly, an e-interview with University of Minnesota journalism academic Associate Professor Gary Schwitzer, blogger and publisher of HealthNewsReview.
Finally, there is also a detailed analysis of the SA study by Dr Andrew Bisits, an obstetrician and epidemiologist, which reinforces some of the concerns raised about how the findings were reported in the press release, editorial and media.
1. Drs Steven Woloshin and Lisa Schwartz write:
“We don’t think the press release did a good job communicating the study findings.
It exaggerates the danger of home birth: the title says “higher risk of perinatal deaths” when that didn’t differ between the groups; the lead par highlights “intrapartum related” deaths when overall deaths didn’t differ, it presents only those scary odds ratios without providing the absolute risks. And it minimizes the potential benefits of home birth by qualifying that home births were associated with less specialized neonatal care, hemorhage, tears were lower but that these differences were not statistically significant.
The editorial isn’t better. Sadly, neither is the journal abstract. Where are the absolute risks for the OR’s of 7 and 27?
We think press releases do more than just draw attention to studies — they also shape how journalists think and write about them. Consequently we’ve been arguing for structured press releases (like journal abstracts) with clear rules about giving absolute risks and noting limitations.”
***
2.Email interview with Associate Professor Gary Schwitzer:
Q: Do you think the press release provides a fair or useful summary for journalists? Do you think it adequately acknowledged some of the uncertainties surrounding the study’s findings?
A: I’m in fits even with the first sentence. “A retrospective population-based study has added to previously published evidence…” What are the limitations of a retrospective population-based study? The news release doesn’t address this. What is the quality of the evidence of the previously published studies? The news release doesn’t address this either.
Deeper in the news release, it states: “Prof Keirse said that while the data showed that planned home births had a perinatal mortality rate similar to that of planned hospital births, they had a sevenfold higher risk of intrapartum death and a 27-fold higher risk of death due to intrapartum asphyxia (lack of oxygen during childbirth).” Huh?
Journalists – and the general public – are going to need a little help breaking down this statement. Sevenfold and 27-fold higher risks? 7 times what? 27 times what? If I’m reading the study correctly in a rush, the intrapartum death rate per 1,000 births was .8 for planned hospital births and 1.8 per 1,000 births for planned home births. Deaths attributed to intrapartum asphyxia were at the rate of .3 per 1,000 births for planned hospital births and 2.6 per 1,000 for planned home births. These frequency data are far more useful than the terms used in the news release.
The news release is simply inaccurate in stating “The study also found that low Apgar scores were more frequent among planned home births.” As I read the study, the rate was 1.1% for planned home births and 1.4% for planned hospital births.
It’s ironic that the final quote in the news release has no ending. It says, ““Those who argue that planned home birth in Australia can be safe will have to show this on the basis of…..” On the basis of what? The same kind of mixed evidence that this news release promotes?
This was, in short, an awful news release. It projects certainty where a great deal of uncertainty exists and emphasizes certain findings to the exclusion of others that may not fit a certain policy agenda.
Q. What do you think would have been an accurate journalistic representation of the study’s findings – for eg what would have been a fair lead or fair
headline? (most reports I’ve seen concentrated on the “sevenfold higher risk of intrapartum death and a 27-fold higher risk of death due to intrapartum asphyxia” highlighted in the release).
A: Honestly, I think a fair headline would have been a mealy-mouthed, “This study shows a mixed bag of outcomes, perhaps raising more questions than it answers.”
Q: There has been a lot of discussion about the potential pitfalls of the media simply reporting relative risks and benefits, when it comes to
medicines. Should we be having similar conversations about other types of interventions? Does it matter that the reporting of this study would make it very difficult for journalists to determine the absolute risks of homebirth versus hospital birth?
A: The importance of providing absolute risk/benefit data – or natural frequencies as Gerd Gigerenzer calls for – applies to any situation where a claim is being made about efficacy of an intervention or an approach – not simply with medicines. Journals are inconsistent in what they demand from authors. Some allow relative and absolute data to be used within the same paper – sometimes with relative data used for benefits (because it looks more impressive) and absolute data used for harms (because it looks less harmful).
The wide-as-the-ocean confidence intervals in this study are cause for concern. Did the journal address this with the authors? Why not educate journalists and the public about what these confidence intervals mean?
Q: Do you think journal press releases should be structured to provide journalists with a balanced interpretation of a study’s findings, for eg: what the study involved? What are the strengths and weaknesses of this design? What did the study find? How confident can we be of the reliability of these findings? What are their clinical/policy implications?
A: I think that journal press releases should follow the criteria employed by the Media Doctor or HealthNewsReview.org websites. Among other things, readers should be able to easily see if the study answered questions such as:
* How often do benefits occur – in absolute terms?
* How often do harms occur – in absolute terms?
* How strong is the evidence?
* Is this condition exaggerated (disease-mongering)?
* What’s the total cost of the intervention and how does it compare with
alternative options?
* Are there alternative options and how does this approach compare not
only in cost but in benefits/harms?
* Is this really a new approach?
* What is the current availability of the approach?
* Who’s promoting this?
* Do they have a conflict of interest?
There is a complete dissonance between the damning tone of the news release and this important sentence from the paper itself: “Although our study has shown few adverse outcomes from planned home births in SA, small numbers with large confidence intervals limit interpretation of these data.” Indeed, small numbers with large confidence intervals make any meaningful comparisons difficult.
Q: At the Croakey blog, the MJA editor Martin Van Der Weyden argued that the role of a journal press release was simply to attract journalists’ attention and it was up to them to then read the full study and make up their own minds. Do you think this is a useful assessment? If not, why not?
A: This is perhaps why journals should stop issuing news releases. If it’s all about attracting attention, not helping journalists and the public they serve to evaluate the quality of the evidence, which in this case is appropriately in question, then what public good is being served by journal news releases? Journals know that many journalists need help scrutinizing the evidence, yet they use news releases to get more publicity for their journal. This, of course, serves the journal’s goals of increasing the prestige of the journal and making it more attractive to advertisers. But it doesn’t do much to improve public understanding of science, does it?
Q: Another issue is the peer review process. There is so much focus on multidisciplinary care and research etc. Is it appropriate for medical journals to ask only obstetricians (the reviewers were three obstetricians and one statistician) to review an article like this, esp given the multidisciplinary nature of maternity care?
A: Evidence is evidence, whether one is an ob-gyn or a primary care provider or a journalist. So it shouldn’t require an ob-gyn to review an ob-gyn related manuscript. Who the peer reviewers are in any given journal on any given manuscript is an important question – one that gets very little attention so you’re wise to bring it up. It would seem wise to include a non-obstetrician in the mix if one truly cared about an even-handed evaluation of the evidence.
Q: What lessons can be drawn from this case (if any) for journal editors and readers, journalists and their readers, and researchers?
A: Journalists – and people who read news based on medical journal articles – need to realize that what is published in journals was not etched in stone tablets and handed down from a mountaintop. There are flaws in the way journals review and accept manuscripts for publication. So flaws in the underlying studies or analysis can be missed. When this happens, a manuscript may be allowed – within this flawed system – to make claims that go beyond what the evidence shows. Then, when a clear conflict of interest arises in the selection of the editorial writer, the process is further clouded. Worse yet, when the journal writes news releases that trumpet the findings and minimize the limitations, the scenario sinks deeper into an ethical abyss. Journal editors, researchers and journalists need to realize that the credibility of the publication process and of medical research itself is jeopardized in such instances.
***
3. Analysis of the study by Dr Andrew Bisits, obstetrician and epidemiologist
“The paper in the MJA 18/1/2009 reports the results of a study that compared outcomes of women who planned a homebirth with those who planned a hospital birth in the state of South Australia between the years of 1991-2006.
The objective is clear; no hypothesis is proposed for refutation because the study is an exploratory or hypothesis generating exercise. The study group consisted of those women who intended to have a homebirth in South Australia between the years 1991-2006; the control group consisted of those women who intended to have a hospital birth in the same period. The classification of these women seems to be clear cut. The source for the data about both groups comes from the SA perinatal database and confidential inquiries into perinatal adverse outcomes. No mention is made of the accuracy of these sources of data.
The key study factor was intention to birth at home. The key control factor was intention to birth at hospital. There is a veneer of simplicity about these factors; however, they are both complex interventions with many possible variations.
The authors analyzed the data with the following methods
The frequency of perinatal mortality, intrapartum death and death due to intrapartum asphyxia was reported for the two groups
Perinatal mortality was reported as the number of babies lost (up to 28 days post partum) per thousand births
The study and control groups were compared for important baseline differences
Odds ratios for the above outcomes with the reference group being hospital births
Odds ratios adjusted for important baseline differences between the study and control groups. Only those baseline differences reported on the data base were used for adjustment. No consideration was given to the unmeasured baseline differences e.g. how fearful or confident were the women, what was the skill of the carers, what was the available transfer infrastructure, what type of cooperation was there between the homebirth practitioner and the local hospital. In order to do this they used logistic regression. Because the numbers of outcomes were small in the study group the data would have best been analyzed with exact logistic regression methods rather than standard logistic regression methods which rely on assuming that the results will hold in a sufficiently large sample. The study report says that this could not be done due to the limits of the software however there is a software package Logxact which can deal with such data sets. With the methods used in the study, the odds ratios are likely to be biased.
There are some cursory details mentioned about each case and the factors that led to the adverse outcome. It would have been more useful if the authors dedicated a whole table to the nine cases with the pertinent detail. The results are dominated by the summary statistics.
The odds ratio of 27 for Intrapartum death is impressive but has to be interpreted in the light of the width of the confidence interval which is very wide. This adjusted odds ratio was obtained by trying to ensure that the two groups were more comparable on the basis of risk factors that were reported in the data base. If more appropriate confounders were included in the adjustment, then there is a likelihood that the confidence interval will be wider and therefore less stable.
The other contentious area in the analysis is the choice of factors used to adjust the odds ratios. It is clear that simply by throwing in more factors this can lead to biased results. This can easily happen with the methods reported by the authors.
One approach to analysis in these studies is to use propensity scoring. Each subject in the study is given a propensity score for the outcome. Homebirth subjects are then compared with those women in the hospital birth group who have a similar propensity score. While this method has its critics, it probably does lead to more accessible and presentable results compared to logistic regression, which for most readers is a “black box” in the statistical analysis process.
Trying to understand the results of this paper is reminiscent of the term breech trial where the initial interpretation was solely determined by the summary statistics showing an impressive excess risk in those women who were randomized to a planned vaginal breech birth. This was so impressive and frightening that breech vaginal delivery was no longer an option for women with a breech baby at term. As the initial excitement (fuelled by a very supporting editorial in the Lancet) and horror settled, more critical thinking took place about the individual cases where there were adverse outcomes. Questions were asked about the appropriateness of management. It became clear that the management in the three cases of perinatal mortality was totally substandard and could not be explained away with the intention to treat approach. Further critical thinking took place about the ability of centres to provide adequate and competent care for women with a planned vaginal breech delivery. Five years down the track in the AMJOG there was a call from one of the participating centres that the results of the trial be withdrawn.
The point to be made here is that valuable critical information was obtained, not from the various summary statistics but rather the actual details of the cases where there were adverse outcomes and inside knowledge about the conduct of the trial. The homebirth paper does not report enough details about the adverse events in the home birth group to allow a critical evaluation of how important the place of birth was in contributing to the adverse outcome. Looking at other studies that report favorable outcomes with homebirth it is clear that the following are important:
• A good working relationship with the referring hospital
• Skilled and credentialed midwives
• Guidelines for referral
• Robust transfer arrangements
• Distance from referring hospitals.
• Confidence that the woman and her partner have in their system of care.
The authors chose to analyse the data using a fixed-effects assumption about the risk of homebirth as opposed to a random effects assumption. The fixed effects assumption is convenient for this data because the incidence of outcomes in the control group is so low. The fixed effects assumption says that the risk of homebirth in the population at large is a fixed value which we aim to infer with our sample data. However, the more plausible assumption is that the risk of homebirth in the population at large is not a fixed effect but rather will be a range of effects given differing conditions for homebirth.
Confidence intervals surrounding a random effects assumption tend to be wider. More than likely had a random effects assumption been used to analyse the data, the confidence intervals surrounding the odds ratio of 27 would be wider and would have crossed unity – indicating that no clear effect of homebirth was demonstrable in this population data set.
The study is observational and the statistical methods yield strong associations between place of birth and adverse outcome. However, there is still a big step to establish causation. The odds ratio is very impressive because a rag bag collection of homebirths where there were considerable variations in practice yielded a particular set of numbers. The rarity of the outcome event makes it difficult to do any meaningful subgroup analysis that could identify those circumstances which will maximise safety in homebirth settings.
My conclusion for consumers and the general public is that this paper presents conclusions consistent with most other studies of homebirth; adverse outcomes associated with homebirth often occur because women with risk factors choose to deliver at home despite advice to the contrary. It is clear from the Dutch and Canadian observational studies of homebirth that safety of homebirth can be maximized where there are systems of health care that incorporate homebirth into their infrastructure by
• Clear guidelines
• Skilled midwives
• Informed consumers
• Good working arrangements with local hospitals
• Robust transfer arrangements
• Continuous and open audit of outcomes.
• Improving the birthing environment in public and private hospitals.”
And a final word from Croakey:
Some of those I interviewed raised concerns about Dr Pesce speaking about the study’s findings in the media last year, ahead of publication. However, the MJA editor, Dr Martin Van Der Weyden told me that he had given Dr Pesce permission to do this. Usually, the MJA, like most journals, is rather strict about discouraging researchers and journalists from publishing findings ahead of them appearing in the journal. One of the study’s authors, Marc Keirse, Professor of Obstetrics and Gynaecology at Flinders University, told me he had not known that Dr Pesce had been speaking of the results ahead of publication, but had no problems with this.
Great post – congratulations too on the article in the BMJ! One of the most unfortunate consequences of the AMA/MJA’s irresponsible promotion of this research is that by trying to scare women out of homebirths (assuming that that was what they were trying to do) they will probably end up pushing some women into having a home birth who probably shouldn’t. When women looking for some objective information on this issue can only find selective and sensationalist reporting such as this, they can lose trust in any research findings and ignore evidence which actually does show them to be high-risk. And of course, another consequence for the rest of us is that we continue to pay the high cost of hospital births for all those low-risk women who could safely have a homebirth and, if given reliable and objective information about it, would choose to do so.
Thank you Doctor Bisits and others for your insight into this ‘research’
It is no coincidence that this sensationalist study and media release happened right when the Senate was about to debate the new maternity laws. If these laws are passed, they would give Doctors veto rights over women’s birth choices – this includes a woman’s right to employ a private midwife to care for her at home during pregnancy and birth.
I would like to ask Minister Roxon and PM Rudd:
Private practice midwives will now be required to find Doctors to ‘collaborate’ with as required by your new maternity laws currently in the Senate – how will they be able to do this when the Head of the AMA, Doctor Pesce and the Medical Journal of Australia seem determined to demonise home birth both in professional journals and the mainstream media?
This is despite the overwhelming international evidence which show home birth with a skilled practitioner to be as safe (and in some cases safer) than a hospital birth for many women.
This low intervention apprach to birth has significant cost savings for the Government and the taxpayer. Why is the Federal Government not embracing this model of care which has a proven record of improving outcomes for both mothers and babies and the health budget?
Australian women want the choice to be able to birth at home with a private midwife – over 2000 submissions to each of the recent Senate Inquiries into this legislation clearly told the Government this is what is wanted by women and their families. What will it take for them to listen to the consumer and not the Australian Medical Association (which is essentially a trade union for Doctors that exists to protect their interests)?
It is not up to Kevin Rudd or the President of the AMA to decide how and where a woman gives birth – it is a woman’s right to decide.
As a simple bean counter, without delving into the details of the statistical analysis, it would appear that homebirth is a more risky activity than having a birth in hospital. If medical spin doctors exaggerate the risks for commercial advantage this is to be regretted.
However while homebirth advocates are indulging themselves with homebirth, who is looking after the interests of the unborn child? As some of the consequential but avoidable birth abnormalities require community support for the whole of the child’s lifetime, the issue is broader than “the woman’s right to decide”. A woman placing her own body at risk under such circumstances is no different from a scuba diver or downhill skier taking personal risks, but the unborn child has nobody as an advocate.
It is therefore vitally important that mothers undertaking homebirth acknowledge the higher risks and indemnify the community, including emergency health care providers, concerning the avoidable consequences of their self focused action, by being required to formally acknowledge retention of responsibility for this increase risk exposure for both themselves and their unborn child.
Greg
Can you please point out the source of your claims that homebirths cause “avoidable birth abnormalities that require community support for the whole of the child’s lifetime?”
More than 40% of Australian women now have an operative birth which includes caesareans and instrumental deliveries (forceps/vacuum extraction) – these carry high risks to the baby and the mother and can be very traumatic for both.
What these figures show is that the obstetric model is broken.
Marsden Wagner (for 15 years a Director of Women’s and Children’s Health, World Health Organisation) is quoted as saying: “Having a caesarean birth also affects the future reproductive possibilities of the woman, because having a cesarean section means she has a decreased chance of ever getting pregnant again. And if she does get pregnant again, she is at higher risk that her pregnancy will occur outside her womb, a condition that will never result in a live baby and is life threatening for the woman. If in her subsequent pregnancies she succeeds in making it to the end of pregnancy and goes into labour, she is also at higher risk of two serious complications during the birth, both of which can threaten her own life and the life of the baby: a placenta that blocks the outlet for the baby or a placenta that detaches itself before the baby is born.”
This is why many women choose to stay out of hospitals to birth – to avoid risky interventions that are often forced on them when they are at their most vulnerable. There are many many Australian families dealing with children today who have been permanently damaged due to medical interventions that occurred during their birth in hospital.
Let’s please focus more on the thousands of babies dying during childbirth in the hospital system being cared for under the obstetric model instead of comparing homebirthing mothers to scuba divers.
Thank you Crikey for such intelligent coverage and discussion of this issue. It is mindblowing the way that medical “statistics” can be manipulated for individual agendas.
I don’t think the penny has yet dropped with the government or the AMA that they will not win this war they are waging against women’s rights – because women who choose to birth at home do so because they have taken the time to do the research on what is safest for their babies and themselves. They are not passive health consumers who sit in a doctors room nodding their heads. They make informed choices and decisions. They are intelligent, motivated, confident women and their families who take responsibilities for their actions. Supporting women to birth at home with skilled, supported midwives is good for babies, good for women, and good for our groaning hospital system.
And they are choosing to birth the way that women always have, with the assistance of skilled caregivers, other women and (sometimes) men. It is no coincidence that the word for midwife in French is “sage femme” or “wise woman”.
Greg – perhaps you would like to speak to the fellow who rang me out of the blue the other day whose wife had the most horrendous birth experience at Blacktown Hospital. He has spent the last 4 years trying to get someone to be accountable. Listen to him, and others like him and you will no longer question why women choose to birth at home.
Dear Greg
Where to start! As a taxpayer we have both contributed to the billion dollar (and growing) premium subsidy for Australian doctors (High Cost of Claims Scheme). Obstetricians score 80c in the dollar support, oh and after that great score they lobbied hard and reduced the common law rights of plaintiffs. So we pay to get less, what a scam. You have paid NOTHING towards homebirth insurance or poor outcomes, which their are basically NIL. 1 million dollars in claims in the insured history. Calandre Simpson however is the largest payout in med neg history and on appeal (awarded $14M lowered to $11M on appeal) was still twice that of the previous highest award. How was Calandre injured? At a private hospital under the care of a private obstetrician. The likely cause of cerebral palsy was an overdose of syntocinon (used for induction and augmentation of labour). The overdose caused the uterus to contract uncontrollably bashing Caladre’s brain in (yes nice). Oh have obstetrics been held to account for the mis-use of syntocinon? No aprox 60% of Australian women have it as an induction or augmentation agent. The majority with no clinical need. As a consumer advocate I want a safe system across the board, but this is not a level playing field. Homebirth midwives would be the most scrutinised health professionals and as such their practices are in the vast majority excellent. Women are informed and the care is 1 to 1, this removes all those ‘falling through the cracks’ issues in the hospital system where oner busy midwife runs between 3, 4 or more labouring women, hoping to catch when a normal labour turns abnormal. In the case of serious prematurity and pathology hospitals provide valuable care. For the majority of healthy women they are a case of Russian roulette. Most women and babies come out alive (gee what a bench mark), but take a look at the high rates of morbidity from medicalised childbirth. It is hard to find because no one cares enough to compile a report. As someone who has immersed themeselves in childbirth reform for 10 years I can assure you the women and often babies are being suffering high rates of iatrogenic injury. Tragically they are being lied to and believe the harm was just bad luck, rather than as a result of an intervention that in the most part was not clinically indicated.
Does this mean I’ll be able to find a bulk-billing doctor? I don’t have a health care card, and the stats are schewed as most doctors bulk-bill concession card holders only.
What about my Chiro or Physio? Will I they be able to bulk bill as well? What about the sky high price of dental?
I want primary health care, so I don’t end up in hospital. I fail to see Rudd’s idea changing much.
Greg (March 2) – I would like to understand where your fear of homebirth has come from – do you have knowledge of homebirths gone wrong or have you researched the issue in depth???
I did not have either of my homebirths for my own benefit – I had both of my homebirths because I wanted my babies to be born to a low risk, healthy mother, who, accompanied by qualified midwives (who between them had half a century of helping women birth in both hospital and home settings) and after extensive research and preparation around possible unforseen issues such as prolapsed cords, umbilical cord entanglement and issues of bleeding in third stage – would birth them without unnecessary medical interventions and pressure into an environment far less infested with dangerous disease than a hospital.
I love the way homebirthing women and midwives are painted as irresponsible – we do nothing but take responsibility for EVERYTHING in terms of ensuring as best can be done, a healthy and safe place for a baby to be born. I didn’t even take panadol during my pregnancy – if a woman researches the pros and cons of taking powerful opiates during labour and makes an informed decision about her choices – I applaud her responsibility – but to take powerful drugs at a vulnerable time for the mother and baby with no idea what the impact will be on either – this scares me more and it is concerning that many women demand ‘drugs’ during labour as a matter of course without having looked into before the event.
I have saved the community many thousands of dollars and delivered for my babies the best start I could possibly provide. How irresponsible of me…