Related Articles

One Comment

  1. 1


    Commenting on the section: “However, there are weaknesses in the SR approach. For example, SR literature searches are often more than 12 months old at the time of SR publication, and trials published in that period won’t have been included. At least 75 health care trials are published daily, so that’s a lot of available evidence not incorporated in new reviews.”

    There are reasons for all this, and I don’t agree these are limitations of the SR approach.

    1) FUNDS: There is a general idea that anyone can do a systematic review. This means experience in the area is not given value. Which then means no one thinks this work deserves money.
    The 75 RCTs published daily are all funded, for several years and several people. With SR work, the general thought process is: we hire one who knows how to do this (often part time), and one from our existing people can do the helping bit, this should be finished within 6-9 months.

    2) TIME: Anyone who has carried out a Cochrane review knows it takes longer than that to do a good review. The collaboration itself says that (
    My experience says, if you spent less than 2 months on the protocol and less than a month refining searches I can guarantee you will end up with a lot of clean up needed at the end of the review.
    As for updates- when you are under-resourced or working for free you can’t expect a good and timely product. Now in the third update, within past 5 years, of a Cochrane review, I can tell you it won’t have happened if the lead author wasn’t 100% committed to the task, and the guarantor had not moved many boulders to make sure the lead author got consistent funding.
    3) PLANNING: SR is all about planning. I can also tell you that a poorly planned review often gets passed on at the data extraction stage to someone new and that delays things to at least another 12 months. I have recently finished two such reviews- someone planned these in a hurry and gave these to researchers inexperienced with SRs to do. When these were passed on to me and a colleague, several parts had to be redone because there were no logs of any decisions.

    The only way a search can be within past 3 months of a submission is that you finalise your write up for a review in January for example, then run a search again, get two people to quickly scan/data extract/RoB assess/analyse and update the write up within the next 8 weeks and then submit to journal 1 hoping they won’t reject because you found “there is not enough good evidence”, which would take them a month to tell you (at best) and then you send to another journal and so on. Even within Cochrane groups, the time frame from a submission to publication can be between 3 to 6 months. This means a review is expected to be finished within 6- 9 months if the search is expected to be only a year old at publication. Is it not self-contradiction?

    So I guess my point is, these are not limitations ‘of the SR approach’, these are consequence of ‘the approach to SRs’.

    I read a long time ago somewhere that one can only do three types of jobs: a good job, a quick job, or a cheap job, but only two of these are possible together, if its good and cheap it will take time, and if its good and quick it won’t be cheap.

    disclaimer: I have been carrying out systematic reviews for a decade now. Half of my contribution to date has been voluntary.


Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

© 2015 – 2020 Croakey | Website: Rock Lily Design


Follow Croakey