Explanations for Cochrane Summary of Findings (SoF) tables

Examples from table Explanation

Outcomes

The tables provide the findings for the most important outcomes for someone making a decision. These include potential benefits and harms, whether the included studies provide data for these outcomes or not. Additional findings may be reported elsewhere in the review.

Assumed control group risk

Assumed control group risks can be based either on control group risks reported in the included studies or on epidemiological data from elsewhere. When only one control group risk is provided, it is normally the median control group risk across the studies that provided data for that outcome.

Risk is the probability of an outcome occurring. The control group risk is the risk of an outcome occurring in the comparison group (without the intervention).

In this example, the risk of 10 events occurring in every 1000 people indicates what would happen in a typical control group population. When relevant the table will provide information for more than one population, for instance differentiating between people at low and high risk when there are potentially important differences.

Corresponding intervention group risk

Risk is the probability of an outcome occurring. The intervention group risk is the risk of an outcome occurring in the group receiving the intervention.

In this example, the assumed risk in the control group was 10 events in every 1000 persons. Implementing the intervention in this population would result in a corresponding intervention group risk of 1 occurrence in every 1000 people, given the average risk ratio across studies. If the table provides more than one predicted control group risk for an outcome, for instance differentiating between people at low and high risk, then a corresponding intervention group risk is provided for each population.

Relative Effect or RR (Risk Ratio)

 Relative effects are ratios. Here the relative effect is expressed as a risk ratio.

Risk is the probability of an outcome occurring. A risk ratio is the ratio between the risk in the intervention group and the risk in the control group. If the risk in the intervention group is 1% (10 per 1000) and the risk in the control group is 10% (100 per 1000), the relative effect is 10/100 or 0.10.

If the RR is exactly 1.0, this means that there is no difference between the occurrence of the outcome in the intervention and the control group. It is unusual for the RR to be exactly 1.0, and understanding what it means if it is above or below this value depends on whether the outcome being counted is judged to be good or bad.

If the RR is greater than 1.0, the intervention increases the risk of the outcome. If it is a good outcome (for example, the birth of a healthy baby), a RR greater than 1.0 indicates a desirable effect for the intervention. Whereas, if the outcome is bad (for example, death) a RR greater than 1.0 would indicate an undesirable effect.

If the RR is less than 1.0, the intervention decreases the risk of the outcome. This indicates a desirable effect, if it is a bad outcome (for example, death) and an undesirable effect if it is a good outcome (for example, birth of a healthy baby).

What is the difference between absolute and relative effects?

The effect of an intervention can be described by comparing the risk of the intervention group with the risk of the control group. Such a comparison can be made in different ways.

One way to compare two risks is to calculate the difference between the risks.This is the absolute effect.

Consider the risk for blindness in a patient with diabetes over a 5-year period. If the risk for blindness is found to be 20 in 1000 (2%) in a group of patients treated conventionally and 10 in 1000 (1%) in patients treated with a new drug, the absolute effect is derived by subtracting the intervention group risk from the control group risk: 2% - 1% = 1%. Expressed in this way, it can be said that the new drug reduces the 5-year risk for blindness by 1% (absolute effect is 10 fewer per 1000).

Another way to compare risks is to calculate the ratio of the two risks. Given the data above, the relative effect is derived by dividing the two risks, with the intervention risk being divided by the control risk: 1% ÷ 2% = ½ (0.50). Expressed in this way, as the “relative effect”, the 5-year risk for blindness with the new drug is 1/2 the risk with the conventional drug.

Here the table presents risks as x per 1000 (or 100, etc.) instead of %, as this tends to be easier to understand. Whenever possible, the table presents the relative effect as the risk ratio (RR).

Usually the absolute effect is different for groups that are at high and low risk, whereas the relative effect often is the same. Therefore, when it is relevant, we have reported indicative risks for groups at different levels of risk. Two or three indicative control group risks and the corresponding intervention group risks are presented when there are important differences across different populations.

Mean difference
The mean difference is the average difference between the intervention group and the control group across studies. Here a weighted mean difference is used, which means the results of some of the studies make a greater contribution to the average than others. Studies with more precise estimates for their results (narrower confidence intervals) are given more weight.

This way of measuring effect is used when combining or comparing data for continuous outcomes, such as weight, blood pressure or pain measured on a scale. When different scales are used to measure the same outcome, for example different pain scales, a standardized mean difference (SMD) may be provided. This is a weighted mean difference standardized across studies giving the average difference in standard deviations for the measures of that outcome.

Confidence Interval

A confidence interval is a range around an estimate that conveys how precise the estimate is; in this example the result is the estimate of the intervention group risk. The confidence interval is a guide to how sure we can be about the quantity we are interested in (here the true absolute effect). The narrower the range between the two numbers, the more confident we can be about what the true value is; the wider the range, the less sure we can be. The width of the confidence interval reflects the extent to which chance may be responsible for the observed estimate (with a wider interval reflecting more chance).

95% Confidence Interval (CI)

As explained above, the confidence interval indicates the extent to which chance may be responsible for the observed numbers. In the simplest terms, a 95% CI means that we can be 95 percent confident that the true size of effect is between the lower and upper confidence limit (for example 0 and 3 in this example). Conversely, there is a 5 percent chance that the true effect is outside of this range.

Not statistically significant

Statistically significant means that a result is unlikely to have occurred by chance. The usual threshold for this judgement is that the results, or more extreme results, would occur by chance with a probability of less than 0.05 if the null hypothesis (no effect) was true. When results are not statistically significant, as in this example, this is stated and a smaller font is used for the absolute effect to alert users to the possibility that the results may have occurred by chance.

2637

(9 studies)

No. of participants (studies)

The table provides the total number (No.) of participants across studies (2637 in this example) and the number of studies (9) that provided data for that outcome. This indicates how much evidence there is for the outcome.

Quality of the evidence

The quality of the evidence is a judgement about the extent to which we can be confident that the estimates of effect are correct. These judgements are made using the GRADE system, and are provided for each outcome. The judgements are based on the type of study design (randomised trials versus observational studies), the risk of bias, the consistency of the results across studies, and the precision of the overall estimate across studies. For each outcome, the quality of the evidence is rated as high, moderate, low or very low using the following definitions:


Further research is very unlikely to change our confidence in the estimate of effect.
Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.
Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.
We are very uncertain about the estimate.

A blank space indicates that the information is not relevant.

The Cochrane Library - Independent high-quality evidence for health care decision making

Copyright © 2013 John Wiley & Sons, Ltd. All rights reserved