Independent Samples t-test Using JASP

When to use an Independent Samples t-test

The independent samples t-test can be used to assess whether the mean value of some outcome variable is different between two groups.

The observations/measurements on the outcome variable must be Scale data, such as weight, where you might want to compare the mean weight of the population of two different countries.

If your measurements are:

Ordinal then consider a Mann-Whitney test instead
Nominal then consider a Chi-squared test.

If you wish to compare more than two groups then consider using ANOVA (or a Kruskal Wallis test for Ordinal data).

For more help on “What test do I need” go to the sigma website statistical worksheets resources page.

Example

Consumer Reports, a US magazine, conducted a study to compare the calorie content of beef and poultry hotdogs. They used an independent samples t-test to determine if there was evidence of a difference in the mean calorie content between the two types of hotdogs.

The data shown below can be downloaded in a CSV file called hotdog.csv. The data should be set up as two columns. One column contains all the measurements/observations for calories. The other column then indicates which group the measurements came from.

*Data is taken from Moore DS & McCabe GP (2002) Introduction to the Practice of Statistics WH Freeman & Co. USA

Using JASP

To perform the independent samples t-test, from the Main Menu, click the T-Tests menu then select Independent Samples T-test under Classical:

Move calories (the outcome variable) to the Dependent Variables box. Next, move hotdog to the Grouping Variable box. Under Tests, tick Student and Welch. Tick Location parameter and the associated Confidence interval. Tick Descriptives.

Examining the Output

The table Group Descriptives is useful because it provides descriptive statistics for the calorie content of beef and poultry hotdogs. In our case, the mean calorie content for beef hotdogs is 156.85, which is higher than the mean calorie content for poultry hotdogs of 122.47.

But do the results provide evidence that this reflects a true difference in the mean calorie content? Or could we have observed this difference just by chance?

The Research Question is: Is there a difference in the mean calorie content in beef and poultry hotdogs?

The independent samples t-test answers this by testing the hypotheses:

H0: There is no difference in the mean calorie content

H1: There is a difference in the mean calorie content

The table titled Independent Samples T-Test provides the main results of our test. The column labelled p provides you with your p-value which is reported as <.001. This is less than 0.05 so there is evidence in favour of H1 that there is a difference in the mean calorie content of beef hotdogs compared to poultry hotdogs.

The table also shows that the t statistic was 4.346 on 35 degrees of freedom (df), which we often include when reporting our results. Note that we are extracting these results from the first row of the table labelled Test - Student. We will come back to the choice of row later when we consider assumptions in the test.

Reporting Results

We could write the results as:

“An independent samples t-test was used to compare the mean calorie content of beef and poultry hotdogs. Beef hotdogs had a higher mean calorie content (M=156.85, SD=22.6242) compared to poultry hotdogs (M=122.47, SD=25.483), and this difference was statistically significant, t(35) = 4.346, p<0.001.”

Further Work

We can tick a few additional options in JASP to obtain effect sized and check assumptions.

You can obtain Cohen’s d via the main dialogue by making sure the Effect Size (Cohen’s d) option is ticked. Also, under Assumption Checks tick Normality and Equality of variances – Levene’s.

Effect Sizes Cohen’s d

Look again at the Independent Samples T-Test table. The value for Cohen’s d most often reported is the one in the Cohen’s d Column.

A commonly used interpretation of this value is based on benchmarks suggested by Cohen (1988). Here effect sizes are classified as follows: A value of Cohen’s d around 0.2 indicates a small effect, a value around 0.5 is a medium effect and a value around 0.8 is a large effect. In our case Cohen’s d was 1.434, so we have a very large effect.

Equality of Variance

We also need to assess the assumption of homogeneity (equality) of variance. This essentially means can we assume the amount by which the calorie content varies in beef hotdogs is about the same as the amount calories vary in poultry hotdogs.

Levene’s test is used in JASP to evaluate the homogeneity of variance assumption. The following table can be found in the JASP output under Assumption Checks:

The p-value from Levene’s test is shown in the fourth column of the table. You need the p-value to be greater than 0.05 to be able to assume homogeneity of variances. Here p=0.314 and so we can assume equal variances. Hence, we use the results for the t-test in the first row of the Independent Samples T-Test table labelled Test - Student.

If Levene’s test had given us a p-value below 0.05 then we the results for the t-test would come from the second row labelled Test - Welch.

Normality

For the test to be valid it should be reasonable to assume that the calorie measurements are approximately normally distributed. If we have 30 or more measurements in each group, then we can safely make that assumption and need not check this any further. Since we only had 20 measurements for beef and 17 for poultry, we need to do some further assessments:

From the JASP main menu click Descriptives then move ‘calories’ to the Variables box and ‘hotdog’ to the Split box:

Open the Basic plots tab and tick Distribution plots.

Normality could be judged by examining the histograms of your calorie data -see graphs below- to see if both beef and poultry display a roughly symmetric bell-shaped curve.

With small sample sizes, the differences in scores can make the histogram appear jagged, making it difficult to determine normality. It is probably better to assess normality using the Shapiro-Wilk test, which appears in the JASP output previously obtained from the Independent Samples T-Test under Assumption Checks.

You need a non-significant result in both groups, i.e. the sig values (p-values) both need to be greater than 0.05 to be able to assume normality. In our example, p = 0.283 for beef and p=0.277 for poultry, so we can assume the data in both groups is normally distributed. If this were not the case, we would need to use the non-parametric equivalent of the independent samples t-test, called the Mann-Whitney test.

For more resources, see sigma.coventry.ac.uk Adapted from material developed by Coventry University Creative Commons License