When people making decisions about health care look for guidance from research, the outcomes reported by that research are key; however, there is a general lack of consensus regarding the choice of outcomes in particular clinical settings, which affects trial design, conduct, analysis, and reporting. Measuring outcomes that will not change healthcare decisions leads to a waste of resources and a failure to capitalise on the potential power of research to improve health care and health. It makes the task for systematic reviewers unnecessarily arduous since, regardless of how studies differ in other ways, if the outcomes they have measured and reported are not sufficiently similar it becomes difficult, if not impossible, to combine, compare and contrast their findings. This has been evident to the authors of Cochrane Reviews for some time. In 2009, for example, the five most accessed Cochrane Reviews, as well as the most cited review, all described the difficulties caused by inconsistencies in the outcomes reported in eligible trials. If researchers want their work to resolve uncertainty and improve decision making in health care, they need to ensure that the outcomes they include are relevant to health service users (consumers) and other people making choices about health care.
The choice of the outcomes assessed in healthcare research is not the only issue causing problems for users of health research. An additional challenge arises from selective reporting of outcomes in published reports of studies. A systematic review has shown the scale of the problem of outcome reporting bias (i.e. results-based selection for publication of a subset of the original outcome variables) in randomised trials, and an assessment of Cochrane Reviews concluded that outcome reporting bias “is an under-recognised problem that affects the conclusions in a substantial proportion of Cochrane reviews”. Of course, this bias affects evidence synthesis more widely, and, indeed, also affects the interpretation of individual studies when these are considered on their own.
One way to address these issues is the development and application of agreed standardised sets of outcomes, known as ‘core outcome sets’ (COS), and the COMET (Core Outcomes Measures in Effectiveness Trials) Initiative is seeking to facilitate this, after two successful international meetings in 2010  and 2011  and the development of a database, which we describe below. A COS should be measured and reported, as a minimum, in all clinical trials undertaken in a specific health condition., As an example, one COS developed for rheumatoid arthritis included seven outcomes: pain, patient global score, swollen joint count, tender joint count, acute phase reactants, physician global score and functional status. Consensus is also needed on how selected outcomes should be defined and measured. Within any single area of health care, a variety of instruments might measure the same outcome, leading to great variability in how outcomes are measured, scored and reported, as well as in the quality (reliability and validity) of measures used. Adopting a COS does not imply that trials should be restricted to only those outcomes included within it. Rather, it provides a set of outcomes that should always be collected and reported, leaving researchers free to continue to explore other outcomes as well. If the outcomes in the COS are reported then users of the research, be they systematic reviewers or people looking at studies in isolation, will always have access to the same, basic set of key data.
There is synergy between the development of COS for trials and work to select a small number of patient-important outcomes for Summary of findings (Sof) tables in Cochrane Reviews. Developed by the GRADE (Grading of Recommendations Assessment, Development and Evaluation) working group (www.gradeworkinggroup.org) for use in reviews generally, many Cochrane Reviews have featured these tables since 2008, and they play a dominant role in presenting research in guidelines, such as those produced by the WHO.
COMET is working with the GRADE working group as well as with people across The Cochrane Collaboration to achieve three strategic goals: to increase the number of COS developed using evidence-based methods; to increase their impact on the quality of research by raising awareness and increasing their use; and to establish methods for the development of COS. COMET is working closely with key stakeholder groups: participants at the first two conferences included trialists, systematic reviewers, methodologists, health service users (consumers), clinical teams, journal editors, trial funders, policy makers, people responsible for trials registries and regulators, and contributions and input from representatives of the US Food and Drug Administration, European Medicines Agency, UK National Institute for Health and Clinical Excellence, and consumer organisations.
COMET aims to collate and stimulate the development of relevant resources. Data on relevant individual studies, both published and ongoing, are being included in a free, publically available internet-based database (www.comet-initiative.org). This database will be a unique resource, updated periodically, which should serve to minimise duplication of effort that might arise if someone embarks on developing a COS without knowing that one exists. Over 120 published or ongoing studies related to COS in specific conditions have been entered into the repository as of April 2012.
Engagement between COMET and The Cochrane Collaboration is a key step in the development of a strategy to link the development of COS for trials with outcome specification in the protocols for Cochrane Reviews. We are already working with Cochrane Review Groups on this and Cochrane Centres are another important part of the work, given their responsibilities to provide training and support for the conduct of Cochrane Reviews. Researchers who prepare and maintain systematic reviews are likely both to develop COS for use in their reviews and to identify health areas which would benefit from the development of a COS. We would welcome information on both of these topics, for inclusion in the collation of evidence that is being undertaken by COMET. We are also working with research funders to facilitate the development and use of COS and the Health Technology Assessment (HTA) programme, the largest funder of clinical trials in the UK, have recently amended their application form to require applicants to explain their choice of outcomes, referring to the COMET website as a source of information. We want to work with The Cochrane Collaboration to encourage similar consideration of the choice of outcomes when Cochrane Reviews are being planned and their protocols are being written. We look forward to strengthening these partnerships and to tackling the challenges ahead, not least those that arise when trying to influence practice. The alternative, of allowing the current situation to continue, surely cannot be justified.
1Paula Williamson; 2Mike Clarke
1Paula Williamson (email@example.com), Director, MRC North West Hub for Trials Methodology Research, University of Liverpool, United Kingdom; 2Mike Clarke (firstname.lastname@example.org), Director, MRC All-Ireland Hub for Trials Methodology Research, Centre for Public Health, Queen’s University Belfast, Northern Ireland.
How to cite: Williamson PR, Clarke M. The COMET (Core Outcome Measures in Effectiveness Trials) Initiative: its role in improving Cochrane Reviews [editorial]. Cochrane Database Syst Rev. 2012 Apr13;5:ED000041. http://www.thecochranelibrary.com/details/editorial/1797057/The-COMET-Core-Outcome-Measures-in-Effectiveness-Trials-Initiative-its-role-in-i.html
1. Smyth RM, Kirkham JJ, Jacoby A, Altman DG, Gamble C, Williamson PR. Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists. BMJ 2011;342:c7153.
2. Clarke L, Clarke M, Clarke T. How useful are Cochrane reviews in identifying research needs? Journal of health services research & policy 2007;12(2):101-3.
3. Tovey D. Impact of Cochrane Reviews [editorial]. The Cochrane Library 2010 (7 July). http://www.thecochranelibrary.com/details/editorial/756937/The-Impact-of-Cochrane-Reviews-by-Dr-David-Tovey.html (accessed 2 April 2012).
4. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, et al. Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS one 2008;3(8):e3081.4.
5. Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al. The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ 2010;340:c365.
6. Juliet Walker. The COMET initiative. http://blogs.bmj.com/bmj/2010/03/08/the-comet-initiative/ (accessed 4 February 2012).
7. Emma Veitch. The science of outcomes: how far have we come? http://blogs.plos.org/speakingofmedicine/2011/07/15/the-science-of-outcomes-how-far-have-we-come/ (accessed 04 February 2012).
8. Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials 2007;8:39.
9. Williamson P, Altman D, Blazeby J, Clarke M, Gargon E. Driving up the quality and relevance of research through the use of agreed core outcomes. Journal of health services research & policy 2012;17(1):1-2.
10. Tugwell P, Boers M, Brooks P, et al. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials 2007;8:38.
11. Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. Journal of clinical epidemiology 2010;63(6):620-6.
Competing interests: The authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request) and declare (1) that PW and MC have received grants and support for travel to meetings from the Medical Research Council; (2) that the authors have no financial relationships with any entities that have an interest related to the submitted work; (3) that PW and MC have various academic grants pending, and that part of PW's and MC's salaries support work on initiatives such as COMET, and otherwise the authors/spouses/partners/children have no financial relationships with entities that have an interest in the content of the article; and (4) that there are no other relationships or activities that could be perceived as having influenced, or giving the appearance of potentially influencing, what was written in the submitted work.
Image credit: Reproduced with permission from the Management Group for the COMET Initiative.
Contact the Editor in Chief, Dr David Tovey (email@example.com): Feedback on this editorial and proposals for future editorials are welcome.