Meta-Analysis of Categorical Outcomes Using SPSS28

This worksheet focuses on a meta-analysis of categorical outcomes. This includes binary outcomes such as increased or not, success or failure, presence or absence of something etc.

Example Data

Our example uses the results of three studies that examined (amongst other things) the presence/absence of the TT genotype* in two groups of participants. The treatment or experimental group of interest were those with hypertension (raised blood pressure) and the control group were those with no history of hypertension. We want to know if the TT genotype is somehow linked with having hypertension (raised blood pressure).

Imagine your friend is looking at a book and you ask them if the second word in the first sentence is the word “was”. We are asking if at a specific point in the book the word is “was” (yes or no). We are sort of asking the same thing here but this time in relation to the genome and asking if at a particular point in the genome the genotype is “TT” (yes or no).

The data are shown in the table below. For the first study by Say et al. (2005), the Hypertension (Treatment) group, the number of patients with the TT genotype was 22 and the number without the TT genotype was 79. For the Control group (No Hypertension) the number with the TT genotype was 10 and the number without was 77.

Study ID	Hypertension Group TT	Hypertension Group Non TT	Hypertension Group Control Group TT	Hypertension Group Non TT
Say et al 2005	22	79	10	77
Rodriguez-Perez et al 2001	87	212	60	255
Cheng et al 2012	165	135	69	81

For help on how to extract the above data please see the resource that looks at extracting data for meta-analyses.

Entering the data into SPSS

Using SPSS start typing the data into the data sheet so that your screen looks as shown:

We can add the column (variable) names by clicking on the Variable View tab in the bottom left corner of the SPSS Data Sheet. Then when you see the screen below type in names for the columns. You can also amend the number of decimal places displayed as zero since we have whole numbers only here.

Click back on the Data View tab in the bottom left and your screen should look as follows:

Running the Meta Analysis

From the Analyze menu, click Meta Analysis, then Binary Outcomes, then Raw Data.

Move Hyper_TT to the Success box for Treatment Group and the rest of the variables to the appropriate boxes as shown, plus the paper name to the Study ID box:

Leave the Effect Size and Model options as the defaults as shown above unless you know why you need to change these.

For more information on this choice either consult Borenstein (2009) and/or Tufanaru et al. (2015).

Next click on the Plot button and in the next dialogue box that comes up below, on the Forest Plot tab select all the options ticked:

Click Continue and then back at the main dialogue box click OK. The results will appear in a new window (the Viewer window) that should open up on your PC. The results include a Forest plot on the right hand side.

Understanding the Forest Plot

The numbers on the left-hand side include the estimated effect size using the Odds Ratio from each study. For example, Say et al. has a Log Odds ratio of 2.14. The ‘Lower’ and ‘Upper’ values then give a 95% confidence interval for the true Odds Ratio. Say et al. has a lower value of 0.95 and an upper value of 4.82, suggesting the true Odds Ratio could be greater than one or less than one. These results are visualised on the Forest Plot. The square boxes indicate the Odds Ratio reported in each study and the lines then show the confidence intervals. The top box shows the Odds Ratio of 2.14 for Say et al. and the line either side of that top box indicates the confidence interval going from 0.95 to 4.82.

The vertical line at one indicates the location of the null or no effect. An Odds Ratio of one means that the incidence of the TT genotype could be the same in both groups. The diamond at the bottom indicates the estimated Odds Ratio from the overall meta-analysis results, and the line either side of the diamond indicates the 95% confidence interval for this. A larger box indicates the meta-analysis gave that study a larger weight and hence made a larger contribution to the overall results. Studies with greater weight have lower variation (i.e. greater accuracy) and a narrower confidence interval. This will often, but not always, be the larger studies.

Reporting Results

All three studies are consistent with the incidence of the TT genotype being higher with the Hypertension group compared to the Control group, since the odds ratios are all above 1.

However, only the Rodriguez-Perez study displayed a statistically significant effect, indicated by the p value of 0.00 being clearly below 0.05. Note also the confidence interval does not include 1. Note also the p-value of 0.00 is not zero but zero when rounded to two decimal places. It could have been, say 0.0004, which would be reported to as 0.00 when rounded to 2 d.p. The other two studies were not quite significant since their p-values are both 0.07 and hence just above 0.05. Note these last two studies also have 95% confidence intervals that DO include 1 (just!).

The overall meta-analysis however, does show that there is a statistically significant effect due to the p-value (reported as 0.00 when rounded to 2 d.p.). The Odds Ratio of 1.64 (see the ‘OR’ column in the results above) is greater than 1 which indicates that there is a higher incidence of the TT genotype with the Hypertension group compared to the Control group (if the Odds Ratio had been less than 1 then we would have concluded there was a lower incidence with the experimental group).

The Odds Ratio value of 1.64 indicates the odds of a hypertensive person having the TT genotype are 1.64 times those of the odds for someone without hypertension. For more help with odds ratios see Borenstein (2009). The 95% Confidence Interval for the odds ratio indicates that the true ratio could be between 1.27 and 2.12.

Checking Heterogeneity

The Cochrane Handbook (section 9.5) suggest the follow interpretation of the \(I^2\) values.

I²	Interpretation
0% to 40%	Might not be important
30% to 60%	May represent moderate heterogeneity
50% to 90%	May represent substantial heterogeneity
75% to 100%	Considerable heterogeneity

In our case, \(I^2\) = 0.00 or 0% (see the Residual Heterogeneity Estimates table below) so there is little or no heterogeneity evident. This supports the reliability of our results.

We can also test H0: No Heterogeneity versus H1: Heterogeneity is present, with p=0.62 and so we do not reject H0 and have no evidence of the presence of heterogeneity.

References

Borenstein, M., Hedges, L., Higgins, J. and Rothstein, H. (2009). Introduction to Meta-Analysis. John Wiley & Sons.

Chapter 9 of the Cochrane Handbook. https://handbook-5-1.cochrane.org/chapter_9/9_analysing_data_and_undertaking_meta_analyses.htm

Tufanaru, C., Munn, Z., Stephenson, M. and Aromataris, E. Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness. International Journal of Evidence-Based Healthcare 13(3):p 196-207, September 2015. DOI: 10.1097/XEB.0000000000000065

For more resources, see sigma.coventry.ac.uk Adapted from material developed by Coventry University Creative Commons License