Independent Samples t-test Using Jamovi

When to use an Independent Samples t-test

The independent samples t-test can be used to assess whether the mean value of some outcome variable is different between two groups.

The observations/measurements on the outcome variable must be Scale data, such as weight, where you might want to compare the mean weight of the population of two different countries.

If your measurements are:

Ordinal then consider a Mann-Whitney test instead
Nominal then consider a Chi-squared test.

If you wish to compare more than two groups then consider using ANOVA (or a Kruskal Wallis test for Ordinal data).

For more help on “What test do I need” go to the sigma website statistical worksheets resources page.

Example

Consumer Reports, a US magazine, conducted a study to compare the calorie content of beef and poultry hotdogs. They used an independent samples t-test to determine if there was evidence of a difference in the mean calorie content between the two types of hotdogs.

The data shown below can be downloaded in a CSV file called hotdog.csv. The data should be set up as two columns. One column contains all the measurements/observations for calories. The other column then indicates which group the measurements came from.

*Data is taken from Moore DS & McCabe GP (2002) Introduction to the Practice of Statistics WH Freeman & Co. USA

Using Jamovi

To perform the independent samples t-test, from the Analyses Menu, click the T-Tests menu then select Independent Samples T-test.

Move calories (the outcome variable) to the Dependent Variables box. Next, move hotdog to the Grouping Variable box. Under Tests, tick Student’s and Welch’s. Tick Mean difference and the associated Confidence interval. Tick Descriptives.

Examining the Output

The table Group Descriptives is useful because it provides descriptive statistics for the calorie content of beef and poultry hotdogs. In our case, the mean calorie content for beef hotdogs is 157, which is higher than the mean calorie content for poultry hotdogs of 122.

But do the results provide evidence that this reflects a true difference in the mean calorie content? Or could we have observed this difference just by chance?

The Research Question is: Is there a difference in the mean calorie content in beef and poultry hotdogs?

The independent samples t-test answers this by testing the hypotheses:

H0: There is no difference in the mean calorie content

H1: There is a difference in the mean calorie content

The table titled Independent Samples T-Test provides the main results of our test. The column labelled p provides you with your p-value which is reported as <.001. This is less than 0.05 so there is evidence in favour of H1 that there is a difference in the mean calorie content of beef hotdogs compared to poultry hotdogs.

The table also shows that the t statistic was 4.35 on 35 degrees of freedom (df), which we often include when reporting our results. Note that we are extracting these results from the first row of the table labelled Student’s t. We will come back to the choice of row later when we consider assumptions in the test.

Reporting Results

We could write the results as:

“An independent samples t-test was used to compare the mean calorie content of beef and poultry hotdogs. Beef hotdogs had a higher mean calorie content (M=157, SD=22.6) compared to poultry hotdogs (M=122, SD=25.5), and this difference was statistically significant, t(35) = 4.35, p<0.001.”

Further Work

We can tick a few additional options in Jamovi, to obtain effect sizes and check assumptions.

You can obtain Cohen’s d by making sure the Effect Size option is ticked, and also tick Confidence interval. Also, under Assumption Checks tick Homogeneity test and Normality test.

Effect Sizes Cohen’s d

Look again at the Independent Samples T-Test table. The value for Cohen’s d most often reported is the one in the Effect Size Column.

A commonly used interpretation of this value is based on benchmarks suggested by Cohen (1988). Here effect sizes are classified as follows: A value of Cohen’s d around 0.2 indicates a small effect, a value around 0.5 is a medium effect and a value around 0.8 is a large effect. In our case Cohen’s d was 1.43, so we have a very large effect.

Equality of Variances

We also need to assess the assumption of homogeneity (equality) of variance. This essentially means can we assume the amount by which the calorie content varies in beef hotdogs is about the same as the amount calories vary in poultry hotdogs.

Levene’s test is used in Jamovi to evaluate the homogeneity of variance assumption. The following table can be found in the Jamovi output under Assumptions:

The p-value from Levene’s test is shown in the fourth column of the table. You need the p-value to be greater than 0.05 to be able to assume homogeneity of variances. Here p=0.314 and so we can assume equal variances. Hence, we use the results for the t-test in the first row of the Independent Samples T-Test table labelled Student’s t.

If Levene’s test had given us a p-value below 0.05, then the results for the t-test would come from the second row labelled Welch’s t.

Normality

For the test to be valid it should be reasonable to assume that the calorie measurements are approximately normally distributed. If we have 30 or more measurements in each group, then we can safely make that assumption and need not check this any further. Since we only had 20 measurements for beef and 17 for poultry, we need to do some further assessments:

From the Analyses Menu, click the Exploration menu then select Descriptives.

Then move ‘calories’ to the Variables box and ‘hotdog’ to the Split by box.

Open the Plots tab and tick Histogram.

Normality could be judged by examining the histograms of your calorie data -see graphs below- to see if both beef and poultry display a roughly symmetric bell-shaped curve.

With small sample sizes, the differences in scores can make the histogram appear jagged, making it difficult to determine normality. It is probably better to assess normality using the Shapiro-Wilk test, which appears in the Jamovi output previously obtained from the Independent Samples T-Test under Assumptions.

You need a non-significant result, i.e. the p-value needs to be greater than 0.05 to be able to assume normality. In our example, p = 0.288, so we can assume the data is normally distributed. If this were not the case, we would need to use the non-parametric equivalent of the independent samples t-test, called the Mann-Whitney test.

For more resources, see sigma.coventry.ac.uk Adapted from material developed by Coventry University Creative Commons License