library(dplyr)
library(ggplot2)
library(tibble)
zinc <- tibble(
location = 1:10,
bottom = c(
0.430, 0.266, 0.567, 0.531, 0.707,
0.716, 0.651, 0.589, 0.469, 0.723
),
surface = c(
0.415, 0.238, 0.390, 0.410, 0.605,
0.609, 0.632, 0.523, 0.411, 0.612
)
)Lab 8: More Statistical Inference
Paired data, confounding, and comparing more than two groups
Download the student lab template
Today
In the last lab, we compared the means of two independent groups. Today we extend that framework in three directions:
- Paired observations: when two measurements belong together.
- Confounding: when an observed difference may partly reflect another variable.
- Three or more groups: when ANOVA is more appropriate than a collection of separate t-tests.
Most of the lab uses a sample from the American Community Survey (ACS), following the same basic analysis used in the earlier Cornell version of this course.
Part I: A quick paired-data example
Zinc concentrations in water
Suppose environmental researchers measure zinc concentration in both bottom water and surface water at the same 10 locations.
Because the two measurements come from the same location, they are paired.
Create the within-location difference:
zinc <- zinc |>
mutate(diff = bottom - surface)
zincSummarize the differences:
zinc |>
summarise(
mean_difference = mean(diff),
sd_difference = sd(diff),
n = n()
)Run the paired t-test:
t.test(
zinc$bottom,
zinc$surface,
paired = TRUE
)Why are these observations paired rather than independent?
What does a positive value of
diffmean?State the null hypothesis in terms of the mean paired difference.
Based on the test, is there evidence that average zinc concentration differs between bottom and surface water?
A paired t-test is really a one-sample t-test on the within-pair differences.
Part II: American salaries
Since 2005, the American Community Survey has collected information from millions of U.S. households. We will use a random sample from the 2012 ACS.
load(url(
"https://github.com/GarciaRios/govt_3990/raw/gh-pages/Labs/lab6/acs.RData"
))
glimpse(acs)Variables we will use include:
| Variable | Description |
|---|---|
income |
Annual wage and salary income |
employment |
Employment status |
hrs_work |
Weekly hours worked |
race |
Race category |
age |
Age |
gender |
Gender |
citizens |
U.S. citizenship status |
time_to_work |
Travel time to work |
lang |
Language spoken at home |
married |
Marital status |
edu |
Education level |
disability |
Disability status |
First, examine employment status:
acs |>
count(employment) |>
mutate(
percent = 100 * n / sum(n)
)- What percentage of the sample is employed?
For the rest of the earnings analysis, keep only employed respondents:
acs_emp <- acs |>
filter(employment == "employed")Gender and annual income
Start with a visualization:
ggplot(
acs_emp,
aes(x = gender, y = income)
) +
geom_boxplot() +
labs(
x = NULL,
y = "Annual income",
title = "Annual income by gender"
) +
theme_minimal()Calculate group summaries:
acs_emp |>
group_by(gender) |>
summarise(
mean_income = mean(income, na.rm = TRUE),
sd_income = sd(income, na.rm = TRUE),
n = sum(!is.na(income))
)At first glance, how do average incomes differ across the two groups?
What does the boxplot show that the means alone do not?
Now conduct the two-sample t-test:
income_gender_test <- t.test(
income ~ gender,
data = acs_emp,
alternative = "two.sided",
conf.level = 0.95
)
income_gender_testState the null and alternative hypotheses.
Report the p-value and the 95% confidence interval for the difference in means.
What does the analysis tell us about the population difference in average income?
What does this analysis not establish about gender and income?
Part III: Could hours worked matter?
The difference in average income is clear, but the two groups may also differ in other ways.
One possibility is hours worked.
Create a new variable distinguishing full-time and part-time workers:
acs_emp <- acs_emp |>
mutate(
emp_type = case_when(
hrs_work >= 40 ~ "full time",
TRUE ~ "part time"
)
)Examine employment type by gender:
acs_emp |>
count(gender, emp_type) |>
group_by(gender) |>
mutate(
percent = 100 * n / sum(n)
)- Are the two gender groups equally distributed across full-time and part-time work?
Now create separate datasets:
acs_full <- acs_emp |>
filter(emp_type == "full time")
acs_part <- acs_emp |>
filter(emp_type == "part time")Compare average income by gender among full-time workers:
acs_full |>
group_by(gender) |>
summarise(
mean_income = mean(income, na.rm = TRUE),
sd_income = sd(income, na.rm = TRUE),
n = sum(!is.na(income))
)t.test(
income ~ gender,
data = acs_full
)Now repeat for part-time workers:
acs_part |>
group_by(gender) |>
summarise(
mean_income = mean(income, na.rm = TRUE),
sd_income = sd(income, na.rm = TRUE),
n = sum(!is.na(income))
)t.test(
income ~ gender,
data = acs_part
)Does the income difference remain among full-time workers?
Does it remain among part-time workers?
What do these analyses suggest about whether work hours may help explain part of the original income difference?
Stratifying by one variable does not make this a causal analysis. Other differences between groups may still matter.
Part IV: More than two groups
A two-sample t-test works when we are comparing two population means.
Suppose instead we want to ask:
Does average annual income differ across education levels?
First inspect the education categories:
acs_emp |>
count(edu)Summarize income across education levels:
acs_emp |>
group_by(edu) |>
summarise(
mean_income = mean(income, na.rm = TRUE),
sd_income = sd(income, na.rm = TRUE),
n = sum(!is.na(income))
)Visualize the distributions:
ggplot(
acs_emp,
aes(x = edu, y = income)
) +
geom_boxplot() +
labs(
x = "Education",
y = "Annual income",
title = "Annual income across education levels"
) +
theme_minimal()How many population means are we comparing?
Why would a series of separate t-tests be a poor first approach?
For a one-way ANOVA:
[ H_0:_1=_2==_k ]
The alternative is that at least two population means differ.
Fit the ANOVA model:
income_edu_model <- aov(
income ~ edu,
data = acs_emp
)
summary(income_edu_model)What is the p-value for the overall ANOVA?
At (), do you reject or fail to reject the null hypothesis?
What can you conclude from a statistically significant ANOVA?
What can you not yet conclude?
Which groups differ?
If the overall ANOVA indicates that at least some means differ, we can follow it with adjusted pairwise comparisons.
TukeyHSD(income_edu_model)You can also visualize the pairwise confidence intervals:
First, look at the default Tukey plot:
plot(
TukeyHSD(income_edu_model)
)Now recreate the same information with ggplot2:
res <- TukeyHSD(income_edu_model)
df_tukey <- res$edu |>
as.data.frame() |>
tibble::as_tibble(rownames = "comparison")
ggplot(
df_tukey,
aes(
x = reorder(comparison, diff),
y = diff,
ymin = lwr,
ymax = upr
)
) +
geom_pointrange(linewidth = 1) +
geom_hline(
yintercept = 0,
linetype = "dashed"
) +
coord_flip() +
labs(
x = NULL,
y = "Difference in mean income ($)",
title = "Pairwise differences in mean income",
subtitle = "Tukey-adjusted 95% confidence intervals"
) +
theme_minimal()Which education-level comparisons show evidence of different population mean incomes?
How can you tell from both the adjusted p-values and the confidence intervals?
Why do we adjust these comparisons rather than simply run many ordinary t-tests?
Group Policy Analysis
Each group will receive one ACS question. Your job is to decide which comparison method is appropriate, run it, and give a two-minute briefing.
Group A: Citizenship and income
Does average annual income differ between citizens and non-citizens?
- outcome:
income - group:
citizens
Group B: Language and commute time
Does average commute time differ between people who speak English at home and those who speak another language?
- outcome:
time_to_work - group:
lang
Group C: Race and income
Does average annual income differ across the race categories in this sample?
- outcome:
income - group:
race
Treat this as a disparity analysis, not evidence that race itself causes income differences.
Group D: Education and commute time
Does average commute time differ across education levels?
- outcome:
time_to_work - group:
edu
Group E: Education and weekly hours worked
Does the average number of weekly work hours differ across education levels?
- outcome:
hrs_work - group:
edu
What each group should do
- Identify whether the comparison involves two groups or three or more groups.
- State the population parameter(s) and hypotheses.
- Create one useful visualization.
- Calculate group means, standard deviations, and sample sizes.
- Run the appropriate test:
- two groups →
t.test() - three or more groups →
aov()
- two groups →
- If the ANOVA is significant, use
TukeyHSD(). - Give a substantive conclusion and one important limitation.
Two-minute briefing
Be ready to report:
Our question was…
The appropriate method was… because…
We found…
For policy, this may matter because…
One important limitation is…
On Your Own
Choose a numerical outcome and a categorical grouping variable from the ACS data that your group did not analyze.
Write a policy-relevant research question.
Identify the appropriate method: independent two-sample t-test or one-way ANOVA. Explain why.
Create one visualization and calculate descriptive statistics.
State the null and alternative hypotheses.
Conduct the appropriate hypothesis test.
- If you used ANOVA and rejected the null hypothesis, conduct adjusted pairwise comparisons.
- Write a short policy interpretation that reports:
- the observed pattern;
- the inferential conclusion;
- the size/direction of the relevant difference(s);
- one limitation of the analysis.
The method changes with the design, but the inferential logic does not:
question → parameter → estimate → uncertainty → test → interpretation