At the Joint Colloquium of the Cochrane & Campbell Collaborations in Keystone in October 2010, we ran a workshop about the problems of detecting research misconduct, and had a wonderful discussion with participants. The US Office of Research Integrity defines research misconduct as: "fabrication, falsification, or plagiarism in proposing, performing, or reviewing research, or in reporting research results; fabrication is making up data or results and recording or reporting them; falsification is manipulating research materials, equipment, or processes, or changing or omitting data or results such that the research is not accurately represented in the research record; plagiarism is the appropriation of another person's ideas, processes, results, or words without giving appropriate credit; research misconduct does not include honest error or differences of opinion". The Committee on Publication Ethics (COPE) also outlines publication and research misconduct in its flowcharts for editors, and highlights redundant (duplicate) publication, changes in authorship, undisclosed conflicts of interest, and ethical problems as additional types of misconduct. Cochrane Review authors, as they analyse the entirety of primary research evidence in a specific area, are well placed to identify many of these types of research and publication misconduct. Indeed, Professor Sir Iain Chalmers urged systematic reviewers, not so long ago, to harness their unique opportunity to detect plagiarism.
Misconduct can seriously distort the scientific research record, and several articles have indicated that misconduct is widespread. Fanelli et al. found that 2% of scientists surveyed admitted to having fabricated, falsified or modified data or results at least once, and up to 34% admitted other questionable research practices; in surveys asking about the behaviour of colleagues, rates were 14% for falsification, and up to 72% for other questionable research practices. Other articles have also highlighted misconduct, including erroneous presentation of significant results in abstracts, selective outcome reporting, and other issues.
Our workshop considered whether Cochrane Review authors do enough to detect and report research and publication misconduct – important tasks they may do better than anyone else. Searching The Cochrane Library in July 2010 (using the search term "data NEAR/2 falsifi* in All Text") we found that only five Cochrane Reviews mentioned data fabrication,,,,, and two reported data falsification., In all instances the misconduct had already been discovered by other authors and editors, mostly in journals. Only one Cochrane Review clearly stated (in a table) that a primary study was excluded because of plagiarism. By contrast, redundant publication was reported in 440 of 4372 Cochrane Reviews included in Issue 9, 2010 of the Cochrane Database of Systematic Reviews. This considerable number may reflect that guidance to authors on how to handle duplicate data is described in the Cochrane Handbook; however, the Cochrane Handbook does not guide authors on how to report and highlight instances of redundant publication. Additionally, while the Cochrane Handbook highlights that authors should be aware of fraudulent studies, retracted publications, errata and comments, authors are not guided on whether and where they should report these issues when they are identified.
Risk of bias is now addressed extensively in current Cochrane Reviews, and GRADE provides a methodology for grading the quality of evidence. Both the risk of bias and GRADE encourage a considered assessment of the limitations of primary studies: however, these assessments focus primarily on identifying poor methodology rather than poor ethical integrity. Only assessment of incomplete outcome reporting and selective reporting can identify actual misconduct. Discussants at the workshop noted that experienced systematic reviewers become sceptical about studies from certain authors, sites, and even countries known for having poor research integrity records, and can be particularly cautious when appraising trials where too many participants have been recruited from a single site. When Cochrane Review authors think they have discovered misconduct, it is not clear what they should do; they have no guidance on how to report misconduct within their Cochrane Reviews, and how and when to contact editors of primary journals. In some cases authors may exclude a suspect study because it does not meet the prespecified inclusion criteria, and their concerns about the study may remain unheard.
The Cochrane Database of Systematic Reviews now has authors, editors, Editor-in-Chief, publisher, impact factor, and content such as Editorials, and therefore is similar in many respects to medical journals (although the author role is often partly fulfilled by editors and Cochrane Review Group staff, as staff work more closely with authors than editors of 'traditional' journals). Yet despite its journal status, the Cochrane Database of Systematic Reviews provides no guidance for authors via the Cochrane Handbook and Cochrane Policy Manual on reporting misconduct. By contrast, all Cochrane Review Groups are members of COPE, which provides editors with specific advice on both unresolved and proven cases of misconduct and, when necessary, on referring cases to authors' institutions and licensing bodies. Several Cochrane editors have now used the COPE forum meetings to discuss specific anonymised cases, but is COPE's advice shared with other Cochrane Review authors so that they can learn from it, and are subsequent actions reported anywhere in The Cochrane Library? And could The Cochrane Library publish notices of concern when appropriate, which might lead to the retraction of one or more primary studies if necessary?
Workshop participants agreed that it is time for the Cochrane Handbook and Cochrane Policy Manual to include guidance on detecting and reporting misconduct in primary studies within Cochrane Reviews. First of all, the Cochrane Handbook must clearly acknowledge the possibility of finding misconduct in primary studies. Secondly, it should provide examples of misconduct so that authors can clearly recognise it. Thirdly, it should advise on how and when to describe and report misconduct (for example, selective reporting is highly prevalent in primary studies but will not always amount to serious misconduct or warrant exclusion of a study from a review, and duplicate publication may sometimes be warranted). Options for reporting proven misconduct might include prominent flagging of relevant reviews; using special tables to list studies where there was misconduct; and providing a short glossary of terms for research misconduct that differentiates appropriate practices such as data transformation procedures from inappropriate data manipulation. Using the right terms would ensure consistent reporting within Cochrane Reviews and would enable reliable searches of completed Cochrane Reviews.
COPE provides algorithms which recommend graded responses to different types and degrees of misconduct. The Cochrane Handbook could provide similar algorithms about reporting, handling, and referring cases of misconduct, recommending actions including:
Other options debated in the workshop included appointing a Collaboration committee of fraudbusters, creating some other Cochrane independent body to report to (anonymously?) and to judge and handle cases. Internal referrals to such a body would relieve Cochrane Reviewer authors of the associated ethical, professional, legal, and practical burdens and would guard against making malicious accusations. Referral of anonymised cases to an external body (e.g. COPE) was also discussed, as was a more radical proposal to build a 'name and shame' website for collating proven cases of misconduct, and contacting authors' peers in confidence to discuss potential misconduct.
We believe that Cochrane Review authors should report newly discovered and previously known cases of misconduct in primary studies, should avoid using euphemisms, and should have proper support and guidance from The Cochrane Collaboration on how to handle misconduct. Care must be taken to avoid tarnishing reputations without cause or due process, advice in the Cochrane Handbook and Cochrane Policy Manual should be ethically and legally sound, and such delicate activities require oversight from the Cochrane Editorial Unit. The Collaboration might also encourage editors or other journals to send alerts about retractions and notices of concern, so that together they can work towards clean, unbiased systematic reviews and a reliable research record.
1Vasiliy Vlassov; 2Trish Groves
1Vasiliy Vlassov (firstname.lastname@example.org), President, Russian Society for Evidence Based Medicine, PO Box 13, Moscow 109451, Russia; 2Trish Groves (email@example.com), Deputy Editor, BMJ, BMA House, Tavistock Square, London WCIH 9JR, United Kingdom.
How to cite: Vlassov V, Groves T. The role of Cochrane Review authors in exposing research misconduct [editorial]. Cochrane Database Syst Rev. 2010 Dec8;2011:ED000015 http://www.thecochranelibrary.com/details/editorial/886689/The-role-of-Cochrane-Review-authors-in-exposing-research-and-publication-miscond.html.
 Groves T, Vlassov V. WS67: The role of Cochrane reviewers in exposing research misconduct. Joint Colloquium of The Cochrane and Campbell Collaborations, Keystone, Colorado, USA, 18–22 October 2010.
 Office of Research Integrity. Definition of Research Misconduct. http://ori.hhs.gov/misconduct/definition_misconduct.shtml (accessed 2 December 2010).
 Chalmers I. Role of systematic reviews in detecting plagiarism: case of Asim Kurjak. BMJ 2006;333:594–6.
 Fanelli D. How many scientists fabricate and falsify research? A systematic review and meta analysis of survey data. PLoSONE 2009;4:e5738.
 Gotzsche PC. Believability of relative risks and odds ratios in abstracts: cross sectional study. BMJ 2006;333:231–4.
 Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004; 291:2457–65.
 Berger VW,Ioannidis JPA. The Decameron of poor research. BMJ 2004;329:1436–40.
 Moore RA, Straube S, Wiffen PJ, Derry S, McQuay HJ. Pregabalin for acute and chronic pain in adults. Cochrane Database of Systematic Reviews 2009, Issue 3. Art. No.: CD007076. DOI: 10.1002/14651858.CD007076.pub2.
 Bulley S, Derry S, Moore RA, McQuay HJ. Single dose oral rofecoxib for acute postoperative pain in adults. Cochrane Database of Systematic Reviews 2009, Issue 4. Art. No.: CD004604. DOI: 10.1002/14651858.CD004604.pub3.
 Bhattacharjee J, El-Sayeh HGG. Aripiprazole versus typical antipsychotic drugs for schizophrenia. Cochrane Database of Systematic Reviews 2008, Issue 3. Art. No.: CD006617. DOI: 10.1002/14651858.CD006617.pub3.
 Clarke MJ, Hopewell S, Juszczak E, Eisinga A, Kjeldstrøm M. Compression stockings for preventing deep vein thrombosis in airline passengers. Cochrane Database of Systematic Reviews 2006, Issue 2. Art. No.: CD004002. DOI: 10.1002/14651858.CD004002.pub2.
 Gibson L, Lawrence D, Dawson C, Bliss J. Aromatase inhibitors for treatment of advanced breast cancer in postmenopausal women. Cochrane Database of Systematic Reviews 2009, Issue 4. Art. No.: CD003370. DOI: 10.1002/14651858.CD003370.pub3.
Competing interests: The authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request) and declare (1) no receipt of payment or support in kind for any aspect of the article; (2) no financial relationships with any entities that have an interest related to the submitted work; (3) that TG is employed as Deputy Editor and Senior Research Editor at the BMJ, and that the other author/spouses/partners/children have no financial relationships with entities that have an interest in the content of the article; and (4) that TG served on the council of the Committee for Publication Ethics from 2008 to 2010 (an unpaid task), but there are no other relationships or activities that could be perceived as having influenced, or giving the appearance of potentially influencing, what was written in the submitted work.
Image credit: Mehau Kulyk/Science Photo Library, F002/9891
Contact the Editor in Chief, Dr David Tovey (firstname.lastname@example.org): Feedback on this editorial and proposals for future editorials are welcome.