Meta-Analysis of Categorical Outcomes Using Jamovi

This worksheet focuses on a meta-analysis of categorical outcomes. This includes binary outcomes such as increased or not, success or failure, presence or absence of something etc.

Example Data

Our example uses the results of three studies that examined (amongst other things) the presence/absence of the TT genotype* in two groups of participants. The treatment or experimental group of interest were those with hypertension (raised blood pressure) and the control group were those with no history of hypertension. We want to know if the TT genotype is somehow linked with having hypertension (raised blood pressure).

Imagine your friend is looking at a book and you ask them if the second word in the first sentence is the word “was”. We are asking if at a specific point in the book the word is “was” (yes or no). We are sort of asking the same thing here but this time in relation to the genome and asking if at a particular point in the genome the genotype is “TT” (yes or no).

The data are shown in the table below. For the first study by Say et al. (2005), the Hypertension (Treatment) group, the number of patients with the TT genotype was 22 and the number without the TT genotype was 79. For the Control group (No Hypertension) the number with the TT genotype was 10 and the number without was 77.

Study ID	Hypertension Group TT	Hypertension Group Non TT	Hypertension Group Control Group TT	Hypertension Group Non TT
Say et al 2005	22	79	10	77
Rodriguez-Perez et al 2001	87	212	60	255
Cheng et al 2012	165	135	69	81

For help on how to extract the above data please see the resource that looks at extracting data for meta-analyses.

Entering the data into Jamovi

The data can be found in the CSV file Genotype.csv. It contains the above data, alongside Odds Ratio, Log Odds Ratio and standard error. The latter variables are automatically calculated when hypertension and control group TT and NON TT are inputted into the CSV file (you do not need to calculate these yourself).

Running the Meta Analysis

Select the Analyses tab at the top of the screen and click the Modules plus icon in the top right corner. Then select jamovi library.

Scroll down until you find the module titled MAJOR – Meta Analysis for JAMOVI. Click INSTALL.

This will add the MAJOR module to your Analyses tab at the top. From the MAJOR menu, click on Effect Sizes and (Sampling Variances or Standard Errors).

Move Log Odds Ratio to the Effect Size box, Std Error to the Variance or SE (correspondent observed ES) box and Paper to the Study Label box.

Under Model Options, tick Standard Errors after Selected (correspondent observed ES).

The results include a Forest plot in the results area.

Understanding the Forest Plot

The numbers on the right-hand side include the estimated effect size using the Log Odds Ratio from each study. For example, Say et al. has a Log Odds ratio of 0.76. The other numbers on the right-hand side then give a 95% confidence interval for the true Log Odds Ratio. Say et al. has a lower value of –0.05 and an upper value of 1.57, suggesting the true Log Odds Ratio could be greater than zero or less than zero. These results are visualised on the Forest Plot. The square boxes indicate the Log Odds Ratio reported in each study and the lines then show the confidence intervals. The top box shows the Log Odds Ratio of 0.76 for Say et al. and the line either side of that top box indicates the confidence interval going from –0.05 to 1.57.

The vertical line at zero indicates the location of the null or no effect. A Log Odds Ratio of zero means that the incidence of the TT genotype could be the same in both groups. The diamond at the bottom indicates the estimated Log Odds Ratio from the overall meta-analysis results, and the line either side of the diamond indicates the 95% confidence interval for this. A larger box indicates the meta-analysis gave that study a larger weight and hence made a larger contribution to the overall results. Studies with greater weight have lower variation (i.e. greater accuracy) and a narrower confidence interval. This will often, but not always, be the larger studies.

Reporting Results

All three studies are consistent with the incidence of the TT genotype being higher with the Hypertension group compared to the Control group, since the estimated Log Odds Ratios are all above zero. If the Log Odds Ratios had been less than zero then we would have concluded there was a lower incidence with the Hypertension group.

However, only the Rodriguez-Perez study displayed a statistically significant effect, indicated by the fact that the 95% confidence interval for the Log Odds Ratio here does not include zero and only covers values above zero. This shows that this paper found evidence that the incidence of the TT genotype is higher with the Hypertension group compared to the Control group. The other two studies were not quite significant since both have 95% confidence intervals that do include zero indicating that the true Log Odds Ratios could be zero, meaning no difference.

The output also includes this table that shows the overall Log Odds Ratio to be 0.493. This table also shows the p-value for the overall effect which is less than 0.001. The overall meta-analysis therefore shows that there is a statistically significant effect. The Log Odds Ratio of 0.493 is greater than zero which indicates that there is a higher incidence of the TT genotype with the Hypertension group compared to the Control group. The 95% Confidence Interval for the Log Odds Ratio indicates that the true ratio could be between 0.24 and 0.75 (see Forest Plot above). For more help with odds and Odds Ratios see Borenstein (2009).

Checking Heterogeneity

The Cochrane Handbook (section 9.5) suggest the follow interpretation of the \(I^2\) values.

I²	Interpretation
0% to 40%	Might not be important
30% to 60%	May represent moderate heterogeneity
50% to 90%	May represent substantial heterogeneity
75% to 100%	Considerable heterogeneity

In our case, \(I^2\) = 0.00 or 0% (see the Residual Heterogeneity Estimates table below) so there is little or no heterogeneity evident. This supports the reliability of our results.

References

Borenstein, M., Hedges, L., Higgins, J. and Rothstein, H. (2009). Introduction to Meta-Analysis. John Wiley & Sons.

Chapter 9 of the Cochrane Handbook. https://handbook-5-1.cochrane.org/chapter_9/9_analysing_data_and_undertaking_meta_analyses.htm

Tufanaru, C., Munn, Z., Stephenson, M. and Aromataris, E. Fixed or random effects meta-analysis? Common methodological issues in systematic reviews of effectiveness. International Journal of Evidence-Based Healthcare 13(3):p 196-207, September 2015. DOI: 10.1097/XEB.0000000000000065

For more resources, see sigma.coventry.ac.uk Adapted from material developed by Coventry University Creative Commons License