Growing pain: striving for quality, relevance and applicability in Cochrane Reviews

  • By: Christopher Eccleston, Phil Wiffen, Sheena Derry & Andrew Moore
  • On: May 31, 2013, 09:00
thumbnail image: Growing pain: striving for quality, relevance and applicability in Cochrane Reviews

As The Cochrane Collaboration celebrates its 20th anniversary, we recall the Cochrane Pain, Palliative and Supportive Care (PaPaS) Review Group's first published review, which looked at treating any acute or chronic pain with any antiepileptic drug, at any dose. Already an update of a 1995 review published in the BMJ, this 1998 Cochrane Review was itself updated in 2005,[1] and subsequently withdrawn in 2010. Although in the earlier versions we attempted to differentiate neuropathic pain conditions such as painful diabetic neuropathy or postherpetic neuralgia, there were too few studies with too few patients treated with any one drug in any one condition to afford any subtle insights. There were only 20 trials involving 736 patients in 1998, and 23 trials and 1074 patients in 2005.

One result of the reviews was to give confidence to the pharmaceutical industry to test antiepileptics in chronic neuropathic pain conditions, so that in 2013 there are now nine Cochrane Reviews of individual drugs, with 88 trials and over 16,300 patients (Table 1).

Details of trials of chronic pain

Table 1: Numbers of trials and participants in Cochrane Reviews of antiepileptic drugs for chronic pain

This is not just a numbers game; the latest reviews comply with much more stringent standards on evidence than previously applied. PaPaS editors, working with the International Association for the Study of Pain special interest group on systematic reviews in pain relief, and building on a body of individual patient analyses, proposed a series of criteria for examining evidence in pain.[2] Even more important was the realisation that the conventional 'last observation carried forward’ (LOCF) method of imputing data when patients withdraw from treatment leads to serious overestimation of treatment effects when adverse event withdrawals are high.[3]

The most important development related to outcomes of pain trials. Outcomes had become overly statistical, concentrating on mean differences between an active agent and placebo in the reduction of pain intensity. That sounds reasonable, but being told that an intervention produces, on average, a 10 mm difference from placebo on a 100 mm pain intensity scale is underwhelming. Furthermore, responses to interventions are not distributed normally around the mean: some patients achieve very good pain relief, while the remainder very little pain relief: 'average' pain relief is experienced by only a few.

Our more recent knowledge buttresses the early adoption from clinical experience that only a large pain intensity reduction should count as a successful outcome. The original cut-off of 50% was chosen only because it fell conveniently between 0% and 100% pain reduction. Subsequent research justified the choice of at least 50% pain reduction as a successful outcome. It is the minimum outcome that patients want,[4] and it is associated with the restoration of function, work, and quality of life lost with chronic pain.[5]

Improved quality of review standards has been paralleled by increased quantity of available data, and not just for antiepileptics. About 40 single-dose analgesic reviews in acute pain cover just about all analgesics used orally, involving around 50,000 patients. In migraine, well over 100,000 sufferers have participated in trials covered by about 20 reviews. The combination of high quality and large numbers makes these obvious targets for overviews of reviews; some have been completed, others are in progress. Chronic neuropathic pain offers particular challenges for overviews because of the complexity: historic interest in class effects of drugs conflicts with contemporary insights into the differences between the many different conditions. More than one type of overview may be needed.

Improving quality brings its own challenges. For example, as PaPaS matured we decided that we needed to provide leadership and be clear about what we mean by 'supportive care' as it is a third of our focus. We have now provided this.[6] Examination of palliative care reviews indicated that while the reviews were of good quality, evidence on which to base judgements was lacking,[7] and a search for database or observational studies was unfruitful.[8] Palliative care questions remain among the most challenging.[9]

The most inconvenient truth of improving quality emerges when new standards challenge accepted belief confirmed by existing Cochrane Reviews. This is likely to be a particular concern with opioids for chronic non-cancer pain, where we find average pain changes (based on LOCF imputation where adverse event withdrawal rates are high). Reviews completed before the bias was recognised may be misleading and need revision. Such are the challenges of continuous quality improvement.

Perhaps the biggest challenge of all comes from a different direction – asking consumers of reviews what they want from Cochrane. In 2011 PaPaS asked a panel of international professional consumers what titles they wanted to see in the field of neuropathic pain. We expected a few. We received over 50 sensible title suggestions, dwarfing the number already on The Cochrane Library. Around 10 of these are already underway, with recruiting and training in the planning stage to fill the gaps. It is humbling to think that despite three overviews, 178 reviews, almost 50 protocols, and accelerating productivity, so much remains to be done. And that is without asking questions of professionals and public about their needs in managing acute pain in hospital and primary care, in migraine, in cancer, and in other chronic pain conditions.
Similarly, our attentions are focussing more on the impact of our reviews and on the uptake of evidence. For example, we recently added a review to the library on psychological interventions for children with chronic pain. The evidence of effect on reducing pain experience is strong, but these are treatments rarely offered.[10] As PaPaS grows in confidence we will seek to increase the impact of the evidence we produce. The challenge will be to make a difference to people's lives.

Pain of at least moderate intensity lasting six months or more affects 1 adult in 5, and it has a larger negative effect on quality of life than any other chronic condition for people living in the community.[5] Pain ruins people’s lives. Sufferers want that pain to go away, now, and to not come back.[4] Most, even on treatment, have moderate or severe pain that fractures lives. Patients' experience is at the heart of what we do, and we are never far from that influence.

The Cochrane Collaboration is a unique partnership of the very best of minds, pursuing answers to the most important of questions. We rarely spend time looking back. This quick glance at history has reinforced how important it is to leave our growing pains behind and focus on growing the evidence base for pain. There are important societal questions that require evidence. Headache, migraine, and musculoskeletal disorders remain major problems for all societies.[11] Neuropathic pain is a growing problem associated with ageing and the increase in diabetes. Complex problems of iatrogenisis are emerging, not least from opioid prescribing which is a particular problem that will require our serious attention in the future. Our first 15 years of publishing have been no better than adequate. Consumers of our reviews want us to do more. Patients encourage us to be more courageous. We want to do more, and we can do more. We have plans for growth and we look forward to growing together with our colleagues from across the Collaboration.

Christopher Eccleston1, Phil Wiffen2, Sheena Derry3, Andrew Moore4

1Christopher Eccleston (, Co-ordinating Editor, Cochrane Pain, Palliative and Supportive Care Group, and Director, Centre for Pain Research, University of Bath, UK. 2Phil Wiffen (, Director, Oxford Systematic Review Services, Oxford, UK; 3Sheena Derry (, Senior Research Officer, Pain Research Unit and Nuffield Division of Anaesthetics, University of Oxford, UK; 4Andrew Moore (, Research Director, Pain Research Unit and Nuffield Division of Anaesthetics, University of Oxford, UK.

How to cite: Eccleston C, Wiffen P, Derry S, Moore A. Growing pain [editorial]. Cochrane Database of Systematic Reviews 2013;5:ED000059.


1. Wiffen P, Collins S, McQuay H, Carroll D, Jadad A, Moore A. Anticonvulsant drugs for acute and chronic pain. Cochrane Database of Systematic Reviews 2005;3:CD001133.

2. Moore RA, Eccleston C, Derry S, Wiffen P, Bell RF, Straube S, et al. "Evidence" in chronic pain–establishing best practice in the reporting of systematic reviews. Pain 2010;150(3):386-9.

3. Moore RA, Straube S, Eccleston C, Derry S, Aldington D, Wiffen P, et al. Estimate at your peril: imputation methods for patient withdrawal can bias efficacy outcomes in chronic pain trials using responder analyses. Pain 2012;153(2):265-8.

4. Moore RA, Straube S, Aldington D. Pain measures and cut-offs - no worse than mild pain as a simple, universal outcome. Anaesthesia 2013;68(4):400-12.

5. Moore RA, Derry S, Taylor RS, Straube S, Phillips CJ. The costs and consequences of adequately managed chronic non-cancer pain and chronic neuropathic pain. Pain Practice 2013 Mar 6.

6. Cramp F, Bennett MI. Development of a generic working definition of 'supportive care'. BMJ Supportive & Palliative Care 2013;3:53-60.

7. Wee B, Hadley G, Derry S. How useful are systematic reviews for informing palliative care practice? Survey of 25 Cochrane systematic reviews. BMC Palliative Care 2008;7:13.

8. Hadley G, Derry S, Moore RA, Wee B. Can observational studies provide a realistic alternative to randomized controlled trials in palliative care? Journal of Pain and Palliative Care Pharmacotherapy 2009;23(2):106-13.

9. Chan R, Webster J. End-of-life care pathways for improving outcomes in caring for the dying. Cochrane Database of Systematic Reviews 2010;1:CD008006.

10. Eccleston C, Palermo TM, de C Williams AC, Lewandowski A, Morley S, Fisher E, et al. Psychological therapies for the management of chronic and recurrent pain in children and adolescents. Cochrane Database of Systematic Reviews 2012;12:CD003968.

11. Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990-2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet 2012;380(9859):2163-96.

Competing interests: The authors have completed the Unified Competing Interest form at (available upon request) and declare (1) no receipt of payment or support in kind for any aspect of the article; (2) that PW has received payment for consultancy from the Cochrane Pain and Palliative Care Group, payment for lectures/educational presentations from the Middle East Pain Forum and from Kleijnan Systematic Reviews, royalties from the Oxford University Press and owns a consultancy business - Oxford Systematic Review Services, and that AM has received fees for consultancy and/or lectures from Horizon Pharmaceuticals, Eli Lilly, Menarini, Reckitt Benckiser, Merck, MSD, Ferring, Asahi Pharma, Pfizer, and a grant from Grünenthal, but there are no other financial relationships with any entities that have an interest related to the submitted work; and (3) that there are no other relationships or activities that could be perceived as having influenced, or giving the appearance of potentially influencing, what was written in the submitted work.

Image credit: Claus Lunau/Science Photo Library, C014/2040

Contact the Editor in Chief, Dr David Tovey ( Feedback on this editorial and proposals for future editorials are welcome.


The Cochrane Library - Independent high-quality evidence for health care decision making

Copyright © 2014 John Wiley & Sons, Ltd. All rights reserved