From observation to evidence of effectiveness: the haphazard route to finding out if a new intervention works

  • By: Helen Handoll & Nigel Hanchard
  • On: May 01, 2014, 09:00
thumbnail image: From observation to evidence of effectiveness: the haphazard route to finding out if a new intervention works

Most, if not all, Cochrane Reviews have several stories within. A prime set of stories for effectiveness reviews revolve round the rationale for the interventions and the critical evaluation of the evidence, with some consideration of the plausibility of the results. The ultimate emphasis is on whether something actually works in clinical practice, with consideration of the theoretical mechanisms taking second place. A recently updated Cochrane Review provides an instructive example of the haphazard and lengthy route from observation and innovation to getting evidence on clinical effectiveness for a physical intervention.[1]

The review looked at the non-surgical management of anterior shoulder dislocation following closed reduction, and the intervention of interest is immobilisation in external rotation. Acute anterior shoulder dislocation, where the top end of the upper arm bone is pushed out of the joint socket in a forward direction, is typically treated by closed reduction, to put the joint back, followed by a short period (3 to 6 weeks) of immobilisation and then physical therapy. The overall aim of treatment is to restore a fully functioning, painless, and stable shoulder. The importance of achieving shoulder stability is underpinned by the high rates of re-dislocation, especially in active young adults.

The main rationale for immobilisation is to allow healing to take place. While the extent and nature of the soft-tissue damage vary, there are common patterns of injury. Often involved is a Bankart lesion, where the fibrocartilage rim of the shoulder socket tears away from the underlying bone at the front of the joint. Increasing the contact pressure (coaptation) between the torn surfaces might facilitate healing and, ultimately, reduce recurrence. Cadaveric studies undertaken to test coaptation in different shoulder positions have had contradictory results. Using specimens whose muscles were removed, Itoi and colleagues found no difference from full internal (inwards) rotation to 30 degrees of external (outwards) rotation with the elbow at the side.[2] However, they argued that their experiment did not account for the tensioning effect of tendons at the front of the joint, nor the possibility that such tensioning might limit haematoma (blood clot) formation in the living subject. Two cadaveric studies evaluated coaptation with tendons in situ. Miller and colleagues reported increased coaptation at 45 degrees of external rotation,[3] but others were unable to replicate this in any degree of rotation, whether in intact shoulders, shoulders with surgically created Bankart lesions, or dislocated shoulders.[4] In the clinical context, magnetic resonance imaging (MRI) studies have shown better coaptation of Bankart lesions in external rotation.[5,6] An observational arthroscopic study of patients with Bankart lesions found that external rotation improved the position of the fibrocartilage rim in most cases.[7] Considering this somewhat equivocal background, there has been considerable and sustained interest in the possibility that external rotation immobilisation (where the arm is orientated outwards with the forearm away from the chest) may improve healing, and consequently outcomes, in comparison with the traditional approach of internal rotation immobilisation (in a sling).

External rotation is an unnatural position. It inevitably places the person at greater hazard when moving about, and it has to be maintained by a more rigid construction. Initially, the splints were custom-made, but commercially manufactured splints are now available. The commonplace observation that the development and distribution for clinical use of such devices is unhampered by the strict regulations required for pharmacological interventions merits repeating here. Indeed it is to the credit of the Japanese group, led by Itoi, and subsequently members of the orthopaedic community that they set about getting the evidence for effectiveness via randomised (or at least quasi-randomised) trials. What especially marked this topic out at the time was that it was not surgery (many of the investigators are orthopaedic surgeons) and that there was a spate of prospective trial registrations. This shows some of the excitement for this topic, which we shared, and advances in a field that has shown some resistance to randomised trials.

Unfortunately, much of this research effort has been unproductive, often due to substandard trial methods or failure to complete or publish studies. The first version of the Cochrane Review, published in 2006, included evidence from a preliminary report of one "flawed" quasi-randomised trial that included 40 adults.[8] This was the first 'sounding' of a small series of trials by the Japanese investigators. The updated review includes three randomised trials and one quasi-randomised trial, with a total of 470 participants. All four trials compared external rotation with internal rotation. The biggest trial (198 participants) is a multicentre trial, judged at high risk of bias for all domains except sequence generation, that was led and reported by the Japanese investigators. Some uncertainty of the recruitment boundaries of the trials by these investigators has meant that the originally included preliminary report and another report have been excluded in the current version of the review, to avoid double counting. This illustrates what we feel should become a cardinal rule: the need to separate the clinical evaluation of innovations from their innovators, who irrespective of any of their endeavours to be ‘neutral’ have a substantial investment, whether emotional, perhaps financial, or in terms of professional or international status, in the successful implementation of their idea. It is noteworthy, but perhaps incidental, that the finding of the Japanese trial in favour of external immobilisation is in contrast to a lack of differences between external and internal immobilisation found by the other two randomised trials, both of which were at high risk of bias only from lack of blinding.

The review concludes that the evidence is insufficient to demonstrate "whether immobilisation in external rotation confers any benefit over immobilisation in internal rotation".[1] It also notes the absence of evidence from six unpublished trials and two ongoing trials and recommends that the focus of effort for this comparison should be on making this evidence available. Attempts to contact the investigators of the six unpublished trials were alas discouraging on the whole.

Here we have illustrated the poor yield of so much research activity that perhaps could have yielded the definitive answer. Meanwhile, external immobilisers now feature in the often copious lists of costly shoulder braces and immobilisers of manufacturing companies, and trials have started up, one by the Japanese team, to test variants of external immobilisation. Such trials are premature, but the bigger picture is whether all this research effort was on the right question or has it been a distraction? There may be more important questions on this topic, including whether rigid immobilisation in any position is needed at all for older people. The messages from a series of recent articles on the imperative to avoid research waste ring loud and clear.[9] On a final note, Liavaag and colleagues performed magnetic resonance arthography (MRA) on a subset of patients in their multicentre randomised controlled trial comparing immobilisation in external versus internal rotation and reported better healing of Bankart lesions in the external rotation group.[10,11] That the trial itself demonstrated no functional benefits clearly demonstrates the perils of over-reliance on surrogate outcomes.

Helen Handoll1, Nigel Hanchard2

1Helen HG Handoll (h.handoll@tees.ac.uk), Senior Research Fellow, Health and Social Care Institute, Teesside University, Middlesbrough, UK, and Co-ordinating Editor, Cochrane Bone, Joint, and Muscle Trauma Group; 2Nigel CA Hanchard (n.hanchard@tees.ac.uk), Senior Research Fellow, Health and Social Care Institute, Teesside University, Middlesbrough, UK, and Editor, Cochrane Bone, Joint, and Muscle Trauma Group

How to cite: Handoll HHG, Hanchard NCA. From observation to evidence of effectiveness: the haphazard route to finding out if a new intervention works [editorial]. Cochrane Database of Systematic Reviews 2014;(5):ED000081

References

1. Hanchard NCA, Goodchild LM, Kottam L. Conservative management following closed reduction of traumatic anterior dislocation of the shoulder. Cochrane Database of Systematic Reviews 2014;(4):CD004962. dx.doi.org/10.1002/14651858.CD004962.pub3

2. Itoi E, Hatakeyama Y, Urayama M, Pradhan RL, Kido T, Sato K. Position of immobilization after dislocation of the shoulder: a cadaveric study. Journal of Bone and Joint Surgery: American Volume 1999; 81(3):385-90.

3. Miller BS, Sonnabend DH, Hatrick C, O'Leary S, Goldberg J, Harper W, et al. Should acute anterior dislocations of the shoulder be immobilized in external rotation? A cadaveric study. Journal of Shoulder and Elbow Surgery 2004;13(6):589-92. dx.doi.org/10.1016/j.jse.2004.03.006

4. Limpisvasti O, Yang BY, Hosseinzadeh P, Leba T, Tibone JE, Lee TQ. The effect of glenohumeral position on the shoulder after traumatic anterior dislocation. American Journal of Sports Medicine 2008;36(4):775-80. dx.doi.org/10.1177/0363546507312163

5. Itoi E, Sashi R, Minagawa H, Shinizu T, Wakabayashi I, Sato K. Position of immobilization after dislocation of the glenohumeral joint: a study with use of magnetic resonance imaging. Journal of Bone and Joint Surgery: American Volume 2001;83(5):661-7.

6. Seybold D, Schliemann B, Heyer CM, Muhr G, Gekle C. Which labral lesion can be best reduced with external rotation of the shoulder after a first-time traumatic anterior shoulder dislocation? Archives of Orthopaedic and Trauma Surgery 2009;129(3):299-304. dx.doi.org/10.1007/s00402-008-0618-6

7. Hart WJ, Kelly CP. Arthroscopic observation of capsulolabral reduction after shoulder dislocation. Journal of Shoulder and Elbow Surgery 2005;14(2):134–7. dx.doi.org/10.1016/j.jse.2004.07.002

8. Handoll HHG, Hanchard NCA, Goodchild LM, Feary J. Conservative management following closed reduction of traumatic anterior dislocation of the shoulder. Cochrane Database of Systematic Reviews 2006;(1):CD004962. dx.doi.org/10.1002/14651858.CD004962.pub2

9. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet 2014;383(9912):156-65. dx.doi.org/10.1016/S0140-6736(13)62229-1

10. Liavaag S, Stiris MG, Lindland ES, Enger M, Svenningsen S, Brox JI. Do Bankart lesions heal better in shoulders immobilized in external rotation? A randomized single-blind study of 55 patients examined with MRI. Acta Orthopaedica 2009;80(5):579-84. dx.doi.org/10.3109/17453670903278266

11. Liavaag S, Brox JI, Pripp AH, Enger M, Soldal LA, Svenningsen S. Immobilization in external rotation after primary shoulder dislocation did not reduce the risk of recurrence: a randomized controlled trial. Journal of Bone & Joint Surgery: American Volume 2011;93(10):897-904. dx.doi.org/10.2106/JBJS.J.00416

Competing interests: The authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available upon request) and declare no conflicts of interest.

Image credit: Photostock-Israel/Science Photo Library

Feedback: Please contact David Tovey, Cochrane Library Editor in Chief, with feedback on this editorial and proposals for future editorials.

Keywords:

The Cochrane Library - Independent high-quality evidence for health care decision making

Copyright © 2014 John Wiley & Sons, Ltd. All rights reserved