The question in the title was prompted by an editorial in the European Journal of Anaesthesiology concerning a Cochrane Review that is stated to have "uncritically included" data under suspicion. We write together as an author of that review (J Carlisle) and the editors (J Cracknell, AM, NP, TP, MZ) of the Cochrane Anaesthesia Review Group (CARG). Our purpose is to correct some misunderstandings about the writing and editing of this review. We also challenge The Cochrane Collaboration to authorise a working group to develop and implement a methodology that estimates data integrity and standards for the inclusion or non-inclusion of studies under suspicion but not yet retracted. Systematic reviewers can potentially play a crucial role in the scientific community's struggle against the pollution of fraud. But potential is all that we will have until we help ourselves.
In June 2005 a draft version of the review, on drugs for preventing postoperative nausea and vomiting, included allegations of fraud and fabricated data against an author of some of the included studies. Now, in 2013, 183 (of 193) studies by that author, Dr Yoshitaka Fujii, have been retracted, making him the most prolific scientific fraudster in any discipline, ever. But in 2005, the review presented the CARG and others with two problems. Firstly, it was large. Secondly, there was the problem of the allegation of fraud against Fujii. In 2005 no journal had questioned or retracted a Fujii-authored publication, and no employer had investigated his integrity. After much discussion between the authors, the editors, and the Collaboration's then leadership, it was decided to change the text to convey the authors' meaning in a way that reduced the risk of successful litigation. However, the review included analyses of any interaction between intervention effects and authorship (Fujii versus others).
The retraction of 183 Fujii studies resulted from a combination of statistically unlikely data distributions (age, height, sex, etc), the lack of regulatory approval, and the absence of any experimental study records or data. In a previous scandal, again involving an anaesthetic researcher (Professor Joachim Boldt), 88 retractions were based largely on failures to gain appropriate ethical approval. The retention of details of the investigation's results by Boldt's employers meant that the consortium of journal editors that retracted these papers had to state "the research results are [not] per se fraudulent". Boldt's studies, both retracted and active, continue to be incorporated into systematic reviews. One of us (J Carlisle) has applied the techniques used on Fujii's studies to 79 publications naming Boldt as an author, 44 of which have been retracted. Of the 35 studies not retracted, 10 displayed aberrant statistical characteristics that undermine the assumption that they are genuine (in comparison with 13/44 retracted studies). An additional study from the same researchers, but without Boldt as a co-author, was also retracted on the basis of procedural irregularity; this study also displays statistical anomalies.
The Cochrane Collaboration should therefore consider the following questions:
We also support closer collaboration with journals against this common enemy, the Collaboration helping to remove published fraud, the journals helping to prevent its publication in the first place.
John Carlisle1, Jane Cracknell2, Ann Møller3, Nathan Pace4, Tom Pedersen5, Mathew Zacharias6
1John Carlisle (email@example.com), Consultant Anaesthetist and Intensivist, Torbay Hospital, Torquay, UK; 2Jane Cracknell (firstname.lastname@example.org), Managing Editor, Cochrane Anaesthesia Review Group, Rigshospitalet, Copenhagen, Denmark; 3Ann Møller, Co-ordinating Editor, Cochrane Anaesthesia Review Group, and Professor, Department of Anaesthesiology, Herlev Hospital, Herlev, Denmark; 4Nathan Pace (email@example.com), Co-ordinating Editor, Cochrane Anaesthesia Review Group, and Professor of Anesthesiology, University of Utah, Salt Lake City, USA; 5Tom Pedersen (firstname.lastname@example.org), Director, Head and Orthopaedic Center, Rigshospitalet, Copenhagen, Denmark; 6Mathew Zacharias (email@example.com), Specialist Anaesthetist, Dunedin Hospital, Dunedin, New Zealand.
How to cite: Carlisle J, Cracknell J, Møller A, Pace N, Pedersen T, Zacharias M. What should the Cochrane Collaboration do about research that is, or might be, fraudulent? [editorial]. Cochrane Database of Systematic Reviews 2013;5:ED000060. dx.doi.org/10.1002/14651858.ED000060
3. Wilkes MM, Navickis RJ. The Boldt affair: a quandary for meta-analysts. Anesthesiology News 2013;39:4.
5. Ochmann C, Tuschy B, Beschmann R, Hamm F, Röhm KD, Piper SN. Supplemental oxygen reduces serotonin levels in plasma and platelets during colorectal surgery and reduces postoperative nausea and vomiting. European Journal of Anaesthesiology 2010;27:1036-43. dx.doi.org/10.1097/EJA.0b013e32833b04e4
Competing interests: The authors have declared (1) no receipt of payment or support in kind for any aspect of the article; (2) no financial relationships with any entities that have an interest related to the submitted work; (3) no other relationships or activities that could be perceived as having influenced, or giving the appearance of potentially influencing, what was written in the submitted work.
Image credit: Volker Steger/Science Photo Library
Contact the Editor in Chief, Dr David Tovey (firstname.lastname@example.org): Feedback on this editorial and proposals for future editorials are welcome.