Meta-Analysis of Continuous Outcomes Using SPSS28

This worksheet focuses on a meta-analysis of continuous data which includes outcomes measured as scale data such as weight, calories, or anything where the outcomes are reported using means and standard deviations.

Example Data

We use the results of four studies (research papers) that examined a method called “SMI” (Suboccipital Muscle Inhibition) versus “Other” methods to improve the flexibility of the knee joint in adults. The outcome of interest was the increase in the angle the knee can be moved (Popliteal Knee Angle) in degrees, after using either “SMI” or an “Other” technique. Hence the data are measured in degrees (for angles) and so are continuous data. The studies looked at the change in knee angle, but note that this is measured as Pre-treatment minus Post-treatment (not post-pre) since a reduction is a good thing so a reduction would be recorded as a positive change.

The data below show the mean change for each group (pre-post). It also shows the standard deviation of the change which we need and the sample size in each group. The data were extracted from the information given in the papers. Note that some of our data is missing.

		Experimental Group			Control Group
Study ID	Mean Change	SD	n	Mean Change	SD	n
Kuan 2019	7.41	7.12	27	5.37	6.78	27
Aparicio 2009	4.14		34	0.85		34
Cho 2015	5.5	6.6	25	2.3	5.0	25
Joshi 2018	9.5		20	9

For help on how to extract the above data from the published papers, please see the accompanying resource that looks at extracting data for meta-analyses.

It was not possible to obtain the standard deviation for the change in knee angle for two of the studies due to a lack of information in the papers. There are various options open to you to “estimate” these which are discussed in Chapter 10 of the Cochrane Handbook. However, for simplicity we will estimate the missing standard deviations using the information we do have. For example the estimate for the missing SD for the SMI group, could be based on the mean of the values of 7.12 and 6.6 we do have from other two studies. A simple way of doing this is (7.12+6.6)/2 = 6.86. There are perhaps better ways of doing this but the aim here is to keep things simple. Hence for Aparicio (2009) and Joshi (2018) we use 6.86 as the estimate for the Experimental group.

Similarly for the Control group we estimate the missing SDs as (6.78+5.0)/2 = 5.89. Hence for Aparicio (2009) and Joshi (2018) we use 5.89 as the estimated SD for the Control group.

Entering the data into SPSS

Using SPSS start typing the data into the data sheet so that your screen looks as shown:

We can add the column (variable) names by clicking on the “Variable View” tab in the bottom left corner of the SPSS Data Sheet. Then when you see the screen overleaf type in names for the columns. We have used Exp for the Experimental or treatment group and Ctr for the Control group.

Running the Meta Analysis

From the Analyze menu select Meta-Analysis then Continuous Outcomes then Raw Data.

Move Exp_n to the Study Size box for Treatment Group and the rest of the variables to the appropriate boxes as shown, plus the paper Author(s) to the Study ID box:

Leave the Effect Size and Model options as the defaults as shown above unless you know why you need to change these. For more information on this choice consult Borenstein (2009) and/or Tufanaru et al. (2015).

Next click on the Plot button and in the next dialogue box that comes up below, on the Forest Plot tab select all the options ticked:

Click on Continue and then OK.

The results will appear in a new window (the Viewer window) that should open up on your PC.

Understanding the Forest Plot

The numbers on the right-hand side include the estimated effect size using Cohen’s d from each study. For example, Kuan has a Cohen’s d value of 0.29. Using the following benchmark for Cohen’s d (see Cohen, 1988): small (d = 0.2), medium (d = 0.5), and large (d = 0.8), a value of 0.29 is therefore a small effect. The other numbers on the right-hand side then give a 95% confidence interval for the true effect size. Kuan has a lower value of -0.24 and an upper value of +0.83, which includes zero suggesting the true effect size could be positive or negative. Hence, from this study there is no evidence in favour of SMI over other methods. These results are visualised on the Forest Plot on the right-hand side. The square boxes indicate the effect size (Cohen’s d) reported in each study and the lines then the confidence intervals. The top box shows the Cohen’s d value of 0.29 for Kuan and the line indicates the confidence interval from -0.24 to +0.83.

The vertical line at zero indicates the location of the null effect (i.e. no difference). The diamond at the bottom indicates the overall effect and the 95% confidence interval for the overall effect from the meta-analysis. A larger box indicates the meta-analysis gave that study a larger weight and hence made a larger contribution to the overall results. Studies with greater weight have lower variation (i.e. greater accuracy) and a narrower confidence interval. This will often, but not always, be the larger studies.

Reporting Results

All four studies are consistent with a greater increase in knee angle with SMI compared to other methods since the estimated effect sizes (boxes) are all above zero. However, only Aparicio has a 95% confidence interval that does not include zero and hence shows evidence of an effect. Kuan and Joshi both have 95% confidence intervals that do include zero and hence do not show evidence of an effect. Cho has a 95% confidence interval that does just include zero and hence does not quite show evidence of an effect. The reported differences were therefore only statistically significant in Aparicio, but not quite significant in Cho and not at all significant in the other two studies.

The output also includes this table that shows the overall effect (Cohen’s d) to be d=0.377. This table also shows the p-value for the overall effect with p=0.007. The overall meta-analysis therefore shows that there is evidence of a statistically significant effect (p=0.007) with a small to medium effect (d = 0.377) indicating an improvement in knee angle with SMI compared to other methods. It is worth noting that the 95% Confidence Interval for the overall effect is quite wide with a lower value of 0.104 suggesting a small effect is possibly the true effect.

Checking Heterogeneity

The Cochrane Handbook (section 9.5) suggest the follow interpretation of the \(I^2\) values.

I²	Interpretation
0% to 40%	Might not be important
30% to 60%	May represent moderate heterogeneity
50% to 90%	May represent substantial heterogeneity
75% to 100%	Considerable heterogeneity

From the output we get this table:

In our case, \(I^2\) = 0.00 or 0% so there is little or no heterogeneity evident. This supports the reliability of our results.

References

Borenstein, M., Hedges, L., Higgins, J. and Rothstein, H. (2009). Introduction to Meta-Analysis. John Wiley & Sons.

Tufanaru, C., Munn, Z., Stephenson, M. and Aromataris, E. Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness. International Journal of Evidence-Based Healthcare 13(3):p 196-207, September 2015. DOI: 10.1097/XEB.0000000000000065

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.

For more resources, see sigma.coventry.ac.uk Adapted from material developed by Coventry University Creative Commons License