Preclinical evidence, like clinical evidence, is used to inform decisions about the safety and efficacy of treatments for participants in clinical trials. Unlike clinical studies, most preclinical animal studies are not systematically reviewed, even though systematic reviews of preclinical animal studies can contribute significantly to creating more transparency regarding the possible translation from preclinical animal studies to clinical trials.[2-6] The expected benefits of systematic reviews of preclinical animal studies encompass stimulating better science and improving translation. Fewer than 250 systematic reviews of preclinical animal studies were published prior to 2010, whereas almost 6000 Cochrane Reviews have been published to date.
The quality of methodology and reporting of preclinical animal studies is currently inadequate,[1,7,8,9] and the use of quality and risk of bias assessments in systematic reviews has contributed to the exposure of this problem.[1,10] Few preclinical animal studies report the use of randomisation, allocation concealment, blinding of personnel, or blind assessments of outcomes. To improve reporting for preclinical animal studies, guidelines have been developed and are being implemented by journals and publishers.[12,13] Another problem is that negative results are often not published, leading to publication bias, blurring the interpretation and validity of the research findings. Even though the current situation identifies serious shortcomings in primary studies, there are examples that demonstrate the value of systematic reviews.[2,4,8,15] One unique challenge with animal studies is the large amount of heterogeneity between studies. Systematic reviews and meta-analysis help to make heterogeneity transparent, creating clues for causative factors and possible mechanisms, and new hypotheses, and practical methods already exist for exploring this heterogeneity within meta-analysis of preclinical systematic reviews.
Systematic reviews and meta-analyses of preclinical animal studies create an overview of what has been published and how, including judgements on possible translation.[1-6, 8] Based on systematic reviews of preclinical animal studies, Horn and colleagues found no evidence to justify the start of clinical trials of nimodipine for focal cerebral ischaemia in humans. However, this review was conducted after 7665 patients participated in clinical trials and a Cochrane Review with the same conclusion was published a year earlier. Similarly, Pound and colleagues demonstrated that drug side effects (excess risk of intracranial hemorrhage after thrombolysis treatment for acute stroke) found during the clinical trial could have been identified beforehand if a systematic review of preclinical animal studies had been performed. There are also examples showing that reliance on narrative reviews of preclinical animal studies to justify the start of clinical trials in humans can be misplaced.[8,18] Therefore, systematic reviews of preclinical animal studies carried out prior to the start of clinical trials can save resources and can lead to improved safety for participants in clinical trials, and ultimately better health care.
For all these reasons, a small but growing international community of researchers is conducting systematic reviews of preclinical animal studies. The Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES; www.camarades.info) and the Systematic Review Centre for Laboratory animal Experimentation (SYRCLE; www.SYRCLE.nl) are at the forefront. CAMARADES is routinely performing systematic reviews of preclinical animal studies in stroke and other neurological disorders and has formed a worldwide network. SYRCLE has focussed on the development of methodology and guidelines and offers teaching and training internationally, in addition to performing collaborative systematic reviews. SYRCLE took the initiative for the first international symposium on systematic reviews in laboratory animal science (Nijmegen, 2012), and the second international symposium was organised by CAMARADES in cooperation with SYRCLE (Edinburgh, 2013). The conclusion from these meetings is that more systematic reviews of preclinical animal studies are urgently needed. Moreover, much more effort is needed in the field of education, and good reporting and conduct of systematic reviews, in order to achieve more high-quality systematic reviews.
For conducting preclinical systematic reviews, Cochrane methodology was the starting point and it has been transformed to be specifically for preclinical animal studies. A number of tools have been developed such as the search guide, search filters,[20-22] meta-analysis methods, reporting guidelines for preclinical systematic reviews, and risk of bias assessment. Although good progress has been made, methods and tools still need to be developed and improved upon for the conduct of systematic reviews in preclinical animal studies. In addition, much more education, teaching, and training is necessary because animal researchers have little or no training in systematic review methodology. For further development of tools, methods, education and guidelines an animal methods group would be the way forward.
During the Cochrane Colloquium in Quebec City, Canada (September 2013), SYRCLE organised a workshop and special meeting, with the aim to discuss the value of systematic reviews of preclinical animal studies and their potential relevance to Cochrane. There was overall endorsement for starting an official methods group on systematic reviews of preclinical animal studies in an international collaborative network. The proposed methods group would focus on developing methods for the systematic review of preclinical animal studies, improving the quality of primary preclinical animal studies, and improving the translation from preclinical and clinical trials.
With the establishment of a preclinical animal study methods group in close co-operation with Cochrane, we hope to further advocate the need for and promote the preparation of systematic reviews of preclinical animal studies to aid making better well-informed decisions about health care. Anyone interested in this methods group may contact the first author.
Merel Ritskes-Hoitinga1, Marlies Leenaars2, Marc T Avey3, Maroeska Rovers4, Rob Scholten5
1Merel Ritskes-Hoitinga (firstname.lastname@example.org), SYRCLE, Radboud University Medical Center, Nijmegen, the Netherlands; 2Marlies Leenaars, SYRCLE, Radboud University Medical Center, Nijmegen, the Netherlands; 3Marc T Avey, Post-doctoral Research Fellow, Ottawa Hospital Research Institute, Canada; 4Maroeska Rovers, Professor of Evidence-based Surgery, Department for Health Evidence, Radboud University Medical Center, Nijmegen, the Netherlands; 5Rob Scholten, Director, Dutch Cochrane Centre, University Medical Center Utrecht, the Netherlands.
How to cite: Ritskes-Hoitinga M, Leenaars M, Avey MT, Rovers M, Scholten R. Systematic reviews of preclinical studies can make significant contributions to health care and more transparent translational medicine [editorial]. Cochrane Database of Systematic Reviews 2014;(3):ED000078.
1. Hooijmans CR, Ritskes-Hoitinga M. Progress in using systematic reviews of animal studies to improve translational research. PLOS Medicine 2013;10(7):e1001482. dx.doi.org/10.1371/journal.pmed.1001482
2. Horn J, de Haan RJ, Vermeulen M, Luiten PGM, Limburg M. Nimodipine in animal model experiments of focal cerebral ischemia: a systematic review. Stroke 2001;32(10):2433-8. dx.doi.org/10.1161/hs1001.096009
5. Sena ES, Currie GL, McCann SK, Macleod MR, Howells DR. Systematic reviews and meta-analysis of preclinical studies: why perform them and how to appraise them critically. Journal of Cerebral Blood Flow and Metabolism 2014;Feb 19. dx.doi.org/10.1038/jcbfm.2014.28
7. Korevaar DA, Hooft L, ter Riet G. Systematic reviews and meta-analyses of preclinical studies: publication bias in laboratory animal experiments. Laboratory Animals 2011;45(4):225-30. dx.doi.org/10.1258/la.2011.010121
8. Hooijmans CR, de Vries RBM, Rovers MM, Gooszen HG, Ritskes-Hoitinga M. The effects of probiotic supplementation on experimental acute pancreatitis: a systematic review and meta-analysis. PLOS One 2012;7(11):e48811. dx.doi.org/10.1371/journal.pone.0048811
9. Kilkenny C, Parsons N, Kadyszewski E, Festing MFW, Cuthill IC, Fry D, et al. Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLOS One 2009;4(11):e7824. dx.doi.org/10.1371/journal.pone.0007824
10. Macleod MR, Fisher M, O'Collins V, Sena ES, Dirnagl U, Bath PM, et al. Good laboratory practice: preventing introduction of bias at the bench. Stroke 2009;40(3):e50-2. dx.doi.org/10.1161/STROKEAHA.108.525386
11. Ioannidis JP, Greenland S, Hlatky MA, Khoury MJ, Macleod MR, Moher D, et al. Increasing value and reducing waste in research design, conduct, and analysis. Lancet 2014;383(9912):166-75. dx.doi.org/z10.1016/S0140-6736(13)62227-8
12. Hooijmans, CR, Leenaars M, Ritskes-Hoitinga M. A gold standard publication checklist to improve the quality of animal studies, to fully integrate the three Rs, and to make systematic reviews more feasible. Alternatives to Laboratory Animals 2010;38(2):167-82.
13. Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLOS Biology 2010;8(6):e1000412. dx.doi.org/10.1371/journal.pbio.1000412
14. Sena ES, van der Worp HB, Bath PM, Howells DM, Macleod MR. Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLOS Biology 2010;8(3):e1000344. dx.doi.org/10.1371/journal.pbio.1000344
15. Wever KE, Menting TP, Rovers M, van der Vliet JA, Rongen GA, Masereeuw R, et al. Ischemic preconditioning in the animal kidney, a systematic review and meta-analysis. PLOS One 2012;7(2):e32296. dx.doi.org/10.1371/journal.pone.0032296
16. Vesterinen HM, Sena ES, Egan KJ, Hirst TC, Churolov L, Currie GL, et al. Meta-analysis of data from animal studies: a practical guide. Journal of Neuroscience Methods 2014;221:92-102. dx.doi.org/10.1016/j.jneumeth.2013.09.010
18. Perel P, Roberts I, Sena E, Wheble P, Briscoe C, Sandercock P, et al. Comparison of treatment effects between animal experiments and clinical trials: systematic review. BMJ 2007;334(7586):197. dx.doi.org/10.1136/bmj.39048.407928.BE
19. Leenaars M, Hooijmans CR, van Veggel N, ter Riet G, Leeflang M, Hooft L, et al. A step-by-step guide to systematically identify all relevant animal studies. Laboratory Animals 2012;46(1):24-31. dx.doi.org/10.1258/la.2011.011087
20. de Vries RBM, Hooijmans CR, Tillema A, Leenaars M, Ritskes-Hoitinga M. A search filter for increasing the retrieval of animal studies in Embase. Laboratory Animals 2011;45(4):268-70. dx.doi.org/10.1258/la.2011.011056
21. de Vries RBM, Hooijmans CR, Tillema A, Leenaars M, Ritskes-Hoitinga M. Updated version of the Embase search filter for animal studies. Laboratory Animals 2014;48(1):88. dx.doi.org/10.1177/0023677213494374
22. Hooijmans CR, Tillema A, Leenaars M, Ritskes-Hoitinga M. Enhancing search efficiency by means of a search filter for finding all studies on animal experimentation in PubMed. Laboratory Animals 2010:44(3):170-5. dx.doi.org/10.1258/la.2010.009117
23. Peters JL, Sutton AJ, Jones DR, Rushton L, Abrams KR. A systematic review of systematic reviews and meta-analyses of animal experiments with guidelines for reporting. Journal of Environmental Science and Health Part B 2006;41(7):1245-58. dx.doi.org/10.1080/10934520600655507
24. Hooijmans CR, Rovers MM, de Vries RBM, Leenaars M, Ritskes-Hoitinga M, Langendam MW. SYRCLE's risk of bias tool for animal studies. BMC Medical Research Methodology 2014;14:43. dx.doi.org/10.1186/1471-2288-14-43
Competing interests: The authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request).
Image credit: Jason Snyder, via Wikimedia Commons CC-BY-2.0
Feedback: Please contact David Tovey, Cochrane Library Editor in Chief, with feedback on this editorial and proposals for future editorials.