Meta-Analysis of Categorical Outcomes Using JASP

This worksheet focuses on a meta-analysis of categorical outcomes. This includes binary outcomes such as increased or not, success or failure, presence or absence of something etc.

Example Data

Our example uses the results of three studies that examined (amongst other things) the presence/absence of the TT genotype* in two groups of participants. The treatment or experimental group of interest were those with hypertension (raised blood pressure) and the control group were those with no history of hypertension. We want to know if the TT genotype is somehow linked with having hypertension (raised blood pressure).

Imagine your friend is looking at a book and you ask them if the second word in the first sentence is the word “was”. We are asking if at a specific point in the book the word is “was” (yes or no). We are sort of asking the same thing here but this time in relation to the genome and asking if at a particular point in the genome the genotype is “TT” (yes or no).

The data are shown in the table below. For the first study by Say et al. (2005), the Hypertension (Treatment) group, the number of patients with the TT genotype was 22 and the number without the TT genotype was 79. For the Control group (No Hypertension) the number with the TT genotype was 10 and the number without was 77.

Study ID	Hypertension Group TT	Hypertension Group Non TT	Hypertension Group Control Group TT	Hypertension Group Non TT
Say et al 2005	22	79	10	77
Rodriguez-Perez et al 2001	87	212	60	255
Cheng et al 2012	165	135	69	81

For help on how to extract the above data please see the resource that looks at extracting data for meta-analyses.

Entering the data into JASP

The data can be found in the csv file called Genotype.csv. It contains the data shown in the previous table above, but also includes the following extra variables that have been calculated from that data: Odds Ratio, Log Odds Ratio and Standard Error of the Log Odds ratio.

Note: When undertaking a meta-analysis on your own data, there is an equivalent Excel spreadsheet called Meta-analysis_categorical_data.xlsx which you can use to automatically calculate the extra variables shown above, when your initial data for your two groups are inputted into that same Excel file (you do not need to calculate these yourself). You can then copy and paste that data into JASP to have a data set that looks like that above.

Running the Meta Analysis

From the Main menu click the plus found in the top right corner. Tick Meta-Analysis.

From the Meta-Analysis menu, select Meta-Analysis again under Classical.

Move Log Odds Ratio to the Effect Size box, Std Error to the Effect Size Standard Error box and Paper to the Study Labels box.

Open the Statistics tab and tick Forest plot.

The results include a Forest plot in the results area.

Understanding the Forest Plot

The numbers on the right-hand side include the estimated effect size using the Log Odds Ratio from each study. For example, Say et al. has a Log Odds ratio of 0.76. The other numbers on the right-hand side then give a 95% confidence interval for the true Log Odds Ratio. Say et al. has a lower value of –0.05 and an upper value of 1.57, suggesting the true Log Odds Ratio could be greater than zero or less than zero. These results are visualised on the Forest Plot. The square boxes indicate the Log Odds Ratio reported in each study and the lines then show the confidence intervals. The top box shows the Log Odds Ratio of 0.76 for Say et al. and the line either side of that top box indicates the confidence interval going from –0.05 to 1.57.

The vertical line at zero indicates the location of the null or no effect. A Log Odds Ratio of zero means that the incidence of the TT genotype could be the same in both groups. The diamond at the bottom indicates the estimated Log Odds Ratio from the overall meta-analysis results, and the line either side of the diamond indicates the 95% confidence interval for this. A larger box indicates the meta-analysis gave that study a larger weight and hence made a larger contribution to the overall results. Studies with greater weight have lower variation (i.e. greater accuracy) and a narrower confidence interval. This will often, but not always, be the larger studies.

Reporting Results

All three studies are consistent with the incidence of the TT genotype being higher with the Hypertension group compared to the Control group, since the estimated Log Odds Ratios are all above zero. If the Log Odds Ratios had been less than zero then we would have concluded there was a lower incidence with the Hypertension group.

However, only the Rodriguez-Perez study displayed a statistically significant effect, indicated by the fact that the 95% confidence interval for the Log Odds Ratio here does not include zero and only covers values above zero. This shows that this paper found evidence that the incidence of the TT genotype is higher with the Hypertension group compared to the Control group. The other two studies were not quite significant since both have 95% confidence intervals that do include zero indicating that the true Log Odds Ratios could be zero, meaning no difference.

The output also includes this table that shows the overall Log Odds Ratio to be 0.493. This table also shows the p-value for the overall effect which is less than 0.001. The overall meta-analysis therefore shows that there is a statistically significant effect. The Log Odds Ratio of 0.493 is greater than zero which indicates that there is a higher incidence of the TT genotype with the Hypertension group compared to the Control group. The 95% Confidence Interval for the Log Odds Ratio indicates that the true ratio could be between 0.24 and 0.75 (see Forest Plot above). For more help with odds and Odds Ratios see Borenstein (2009).

Checking Heterogeneity

The Cochrane Handbook (section 9.5) suggest the follow interpretation of the \(I^2\) values.

I²	Interpretation
0% to 40%	Might not be important
30% to 60%	May represent moderate heterogeneity
50% to 90%	May represent substantial heterogeneity
75% to 100%	Considerable heterogeneity

In our case, \(I^2\) = 0.00 or 0% (see the Residual Heterogeneity Estimates table below) so there is little or no heterogeneity evident. This supports the reliability of our results.

References

Borenstein, M., Hedges, L., Higgins, J. and Rothstein, H. (2009). Introduction to Meta-Analysis. John Wiley & Sons.

Chapter 9 of the Cochrane Handbook. https://handbook-5-1.cochrane.org/chapter_9/9_analysing_data_and_undertaking_meta_analyses.htm

Tufanaru, C., Munn, Z., Stephenson, M. and Aromataris, E. Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness. International Journal of Evidence-Based Healthcare 13(3):p 196-207, September 2015. DOI: 10.1097/XEB.0000000000000065

For more resources, see sigma.coventry.ac.uk Adapted from material developed by Coventry University Creative Commons License