One–Way analysis of variance (ANOVA) Using Jamovi

When to use a One-Way ANOVA

A one-way analysis of variance (ANOVA) is a statistical test that can be used to assess whether the mean value of some outcome variable is different between three or more groups.

The observations/measurements on the outcome variable must be Scale data, such as weight, where you might want to compare the mean weight of the population of several different countries. The groups must be independent of each other, for example each person’s weight measurement is included in only one country.

If your measurements are Ordinal, then consider a Kruskal Wallis test instead.

If you wish to compare just two groups, then you can use ANOVA, but a better simpler alternative is to use an independent samples t-test.

For more help on “What test do I need” go to the sigma website statistical worksheets resources page.

Example

A paper bag manufacturer investigated the impact of varying hardwood fibre concentrations (5%, 10%, 15%, and 20%) on the tensile strength of their products, measured in pounds per square Inch (psi). Six test specimens were produced at each concentration level, resulting in 24 bags being tested. The data shown below can be downloaded in a CSV file called hardwood.csv:

Source: Applied Statistics and Probability for Engineers - Montgomery and Runger

The research question aimed to determine if there existed a significant difference in mean tensile strengths among the four hardwood concentration levels. To address this, a one-way ANOVA was conducted, testing the following hypotheses:

H0: There is no difference in mean tensile strength between the four hardwood concentrations.

H1: There is a difference in mean tensile strength between at least two of the hardwood concentrations.

The data need to be entered in Jamovi as two columns. The first column, Conc, represents the independent or grouping variable, where 1 means 5% hardwood concentration, 2 means 10% hardwood concentration, and so on. The second column, Strength, contains all the tensile strength measurements.

Using Jamovi

To perform the one-way ANOVA in Jamovi, from the Analyses Menu, click ANOVA then ANOVA:

Move Strength (the outcome variable) into the Dependent Variable box. Next move Conc to the Fixed Factors box.

Open the Estimated Marginal Means tab. Move Conc to the Term 1 box. Tick Marginal means plots and Marginal means tables.

Examining the Output

Using the main results produced by Jamovi you can see the following:

The Estimated Marginal Means - Conc table provides descriptive statistics for tensile strength for the different wood concentrations. The Marginal Means Plot show that in our samples, the higher the concentration, the higher the mean tensile strength, ranging from 10.0 psi at 5% to 21.2 psi at 20%. although we note there appears to be less difference in strength between the 10% concentration (15.7 psi) and the 15% concentration (17.0 psi).

The table titled ANOVA - Strength provides the main results of our test. The column labelled p provides you with your p-value which is reported as <.001. This is less than 0.05 so there is evidence in favour of H1 that there truly is a difference in mean tensile strength somewhere among the four hardwood fibre concentration levels (5%, 10%, 15%, and 20%). The table also shows that the F statistic was 19.6 on 20 degrees of freedom (df) within groups (Conc) and 3 df between groups (Residuals), which we often include when reporting our results.

However, the ANOVA test only tells us that there is a difference somewhere, not where specifically, which is why we need to undergo post hoc tests to identify where the differences lie.

Post-hoc tests

The nature of these differences can be explored further by performing post hoc tests to drill down into the results, but only if the ANOVA test has been found to be statistically significant, as in our case. Post-hoc tests identify any significant differences in mean scores between all possible pairs of the factor levels, in our case pairs of concentration levels. In effect they are variations on t-tests comparing each pair of means.

To obtain the post hoc tests, under the Post Hoc Tests tab, move ‘Conc’ to the right-hand side box and ensure Tukey under Correction is ticked:

Because our factor has 4 levels, there are a total of 6 possible pairings:

Careful examination of the Jamovi output indicates a statistically significant difference in mean strength between:

5% and 10%, 5% and 15%, 5% and 20% (p=0.005, p<0.001 and p<0.001, respectively), 10% and 20% (p=0.007), and 15% and 20% (p=0.047). However, there was no statistically significant difference between 10% and 15% (p=0.802).

Reporting Results

We could write the results as:

“ANOVA was used to compare the mean tensile strengths of paper bag products containing hardwood at four different concentration levels. There was evidence of a significant difference (F(20,3)=19.6, p<0.001). Tukey’s post hoc test revealed statistically significant differences between 5% and all other concentrations at 10% (p=0.005), 15% (p=0.001) and 20% (p<0.001), as well as statistically significant differences between 10% and 20% (p=0.007), and between 15% and 20% (p=0.047). However, there was no statistically significant difference between 10% and 15% (p=0.802). On average, the 20% hardwood concentration group (M = 21.2) scored higher than the 15% (M = 17.0), 10% (M = 15.7), and 5% (M = 10.0) groups. Overall, these findings suggest that the concentration of hardwood significantly impacts the tensile strength of the paper, except between the 15% and 10% groups.”

Further Work

To further enhance our results, we need to compute eta-squared for practical significance and assess normality plus homogeneity of variances. This will ensure that our one-way ANOVA results are robust enough.

Normality

For the test to be valid it should be reasonable to assume that the measurements in each group are approximately normally distributed. If we have 30 or more measurements in each group, then we can safely make that assumption and need not check this any further.

Since we only had 6 measurements in each group, we need to do some further assessments. However, when we only have very small samples, it can sometimes be quite a challenge to determine whether the data in each group come from a normal distribution or not.

Hence, the best way to assess normality for one-way ANOVA is to repeat the main analysis but save what are called the residuals. The residuals are the differences between each observed value and the mean value for that group. We then need to check if this single column of residuals can be assumed to be normally distributed.

To save the residuals in Jamovi, open the Save tab in the ANOVA analysis and tick Residuals.

This will create a new variable called Residuals that can be found in the data and can be viewed by clicking on the Data Menu tab.

Then from the Analyses menu, click Exploration, then Descriptives. Move ‘Residuals’ to the Variables box:

Under the Plots tab, tick Histogram to obtain a histogram.

Normality could be judged by examining the shape of the histogram to see if it makes a roughly symmetric bell-shaped curve. However, small sample sizes tend to make the histogram look ‘jagged’, making it harder to discern normality by eye. It is probably better to assess normality using the Shapiro-Wilk test, which appears in the Jamovi output under Assumption Checks.

Return to the ANOVA analysis. Under the Assumption Checks tab, tick Normality test.

You need a non-significant result, i.e. the p-value needs to be greater than 0.05 to be able to assume normality. In our example, p = 0.576 and we can assume our sampled differences data is normally distributed. If this were not the case, we would need to use the non-parametric equivalent of one-way ANOVA, called the Kruskal Wallis test.

Equal variances

The ANOVA test we undertook also assumes homogeneity (equality) of variance. This essentially means, can we assume the amount by which the tensile strength varies amongst bags with 10% concentration hardwood, is the same amount the strength of bags with 15% varies and so on?

Levene’s test is used in Jamovi to evaluate the homogeneity of variance assumption. In the ANOVA analysis, open the Assumption Checks tab and tick Homogeneity test.

In the output we obtain the following table:

We examine the p-value. If this is above 0.05 then we can assume equality of variances. In our case p=0.583 so this assumption is fine in our case.

For more resources, see sigma.coventry.ac.uk Adapted from material developed by Coventry University Creative Commons License