library(tidyverse)Lab 4: From Respondents to Poll Estimates
A practical introduction to survey weights
Purpose
In Lab 3, we treated each respondent as if every person contributed equally to the estimate.
Real surveys often use weights. A weight changes how much each respondent contributes to the final poll estimate.
This lab is intentionally practical. We will use R commands you already know and add only one important new function:
weighted.mean()The main question is:
How does weighting change what we report from a poll?
There is no need to memorize a new set of survey commands. Focus on the workflow and the interpretation.
Learning objectives
By the end of this lab, you should be able to:
- explain in plain language what a survey weight does;
- identify the correct weight for the statewide and Latino samples;
- calculate an unweighted estimate;
- calculate a weighted estimate;
- compare weighted and unweighted results;
- inspect whether some respondents receive more or less weight;
- produce simple weighted subgroup estimates; and
- explain what weighting can and cannot fix.
For the October Texas poll:
weight_baseis the weight for the statewide Texas voter sample.weight_hispis the weight for the Latino sample.
The weight you use depends on the population you want to describe.
Part I: Warm-up — use what you already know
1. Load the data
Use the same October Texas poll files from Lab 3:
- Download the respondent-level CSV
- Open the codebook
- Open the statewide crosstabs
- Open the Latino crosstabs
Download the CSV to your Lab 4 folder. Keep the codebook and crosstabs open for reference.
Check your working directory:
getwd()If needed, set it to the folder containing the poll file.
setwd("C:/Users/YourName/Documents/Lab4")Read the data:
texas <- read_csv("texas_poll.csv")Check the dimensions:
dim(texas)2. Create the Hinojosa indicator
In govvote26_full:
1= Gina Hinojosa
Create the same 0/1 indicator used in Lab 3.
texas <- texas |>
mutate(
hinojosa = if_else(
govvote26_full == 1,
1,
0,
missing = NA_real_
)
)Check it:
count(texas, hinojosa)This is all familiar code.
Part II: Build a simple weight ourselves
Before using the weights supplied with the Texas poll, we will build a very simple weight from scratch.
The goal is to understand the logic.
3. Imagine a sample that does not match the population
Suppose we have a small sample with:
- 40% men
- 60% women
But suppose the population we want to describe is:
- 50% men
- 50% women
The sample does not match the population.
Men are underrepresented.
Women are overrepresented.
A simple post-stratification weight can be written as:
\[ \text{weight} = \frac{\text{population proportion}} {\text{sample proportion}} \]
For men:
\[ \frac{0.50}{0.40} = 1.25 \]
For women:
\[ \frac{0.50}{0.60} \approx 0.83 \]
So:
- each man receives a weight of about
1.25; - each woman receives a weight of about
0.83.
The logic is intuitive:
- an underrepresented group receives a weight greater than 1;
- an overrepresented group receives a weight less than 1.
4. Create the toy dataset
toy <- tibble(
gender = c(
"Man", "Man", "Man", "Man",
"Woman", "Woman", "Woman",
"Woman", "Woman", "Woman"
),
support = c(
1, 1, 0, 0,
1, 0, 0, 0, 0, 0
)
)Look at the data:
toyCount the respondents:
toy |>
count(gender)We have:
- 4 men = 40% of the sample;
- 6 women = 60% of the sample.
5. Calculate the weights
The population target is 50% men and 50% women.
For men:
0.50 / 0.40For women:
0.50 / 0.60Now add those weights to the data.
toy <- toy |>
mutate(
weight = case_when(
gender == "Man" ~ 1.25,
gender == "Woman" ~ 0.83
)
)Look at the result:
toy6. Compare the raw and weighted estimates
First calculate the ordinary sample mean:
mean(
toy$support
)Now calculate the weighted mean:
weighted.mean(
toy$support,
toy$weight
)The two results differ because the weighted estimate gives relatively more influence to the underrepresented group and relatively less influence to the overrepresented group.
A weight does not change what a respondent answered.
It changes how much that respondent contributes to the final estimate.
Checkpoint 1
- Why did men receive a weight greater than 1?
- Why did women receive a weight less than 1?
- Why can the weighted estimate differ from the raw sample mean?
Your response:
7. What happens in a real poll?
Real survey weights are usually more complicated than this example.
A professional poll may adjust for several characteristics at the same time, such as:
- age;
- gender;
- race and ethnicity;
- education;
- geography;
- language; and
- features of the sampling or recruitment design.
Methods such as post-stratification, raking, or calibration can combine information from several variables.
We do not need to program those procedures ourselves in this lab.
The important idea is the same:
Respondents who are underrepresented relative to the target population generally receive more influence, while respondents who are overrepresented generally receive less.
Now we will use the weights already constructed for the October Texas poll.
Part III: Statewide Texas estimate
8. Create the statewide sample
Keep respondents with a statewide weight:
statewide <- texas |>
filter(!is.na(weight_base))Check the number of respondents:
nrow(statewide)9. Calculate the unweighted estimate
unweighted_statewide <- mean(
statewide$hinojosa,
na.rm = TRUE
)
unweighted_statewideThis gives every respondent equal influence.
10. Calculate the weighted estimate
Now use weight_base:
weighted_statewide <- weighted.mean(
statewide$hinojosa,
statewide$weight_base,
na.rm = TRUE
)
weighted_statewideCompare them:
unweighted_statewide
weighted_statewideCalculate the difference in percentage points:
(
weighted_statewide -
unweighted_statewide
) * 100Checkpoint 2
- Did weighting increase or decrease Hinojosa support?
- By about how many percentage points?
- Is the change large enough that it would change how you describe the result?
Your response:
Weighting does not have to produce a large change.
If the weighted and unweighted estimates are similar, that is also an informative result.
Part IV: The Latino sample
11. Create the Latino sample
Use weight_hisp:
latino <- texas |>
filter(!is.na(weight_hisp))Check the sample size:
nrow(latino)12. Compare unweighted and weighted Latino estimates
Unweighted:
unweighted_latino <- mean(
latino$hinojosa,
na.rm = TRUE
)
unweighted_latinoWeighted:
weighted_latino <- weighted.mean(
latino$hinojosa,
latino$weight_hisp,
na.rm = TRUE
)
weighted_latinoCompare:
unweighted_latino
weighted_latinoCheckpoint 3: How much does weighting change the Latino estimate?
Your response:
Part V: What do the weights look like?
13. Inspect the weights
You already know summary().
summary(
latino$weight_hisp
)Look especially at:
- minimum;
- median;
- mean; and
- maximum.
You can also look at a few individual values:
head(
latino$weight_hisp
)A respondent with a larger weight contributes more to the weighted estimate.
A respondent with a smaller weight contributes less.
Checkpoint 4
Suppose one respondent has a weight of 2.0 and another has a weight of 0.5.
Which respondent contributes more to the estimate?
Your response:
14. Why would weights differ?
Possible reasons include:
- some groups were oversampled;
- some groups were underrepresented among respondents;
- the sample does not perfectly match known population characteristics;
- the survey design intentionally gave some people different chances of entering the sample.
We do not need to reconstruct every step of the weighting algorithm to use the supplied poll weights appropriately.
Part VI: Weighted subgroup comparisons
15. Recreate the gender variable
As in Lab 3:
1= men2= women
latino <- latino |>
mutate(
gender_group = case_when(
gender == 1 ~ "Men",
gender == 2 ~ "Women",
TRUE ~ NA_character_
)
)Check it:
count(
latino,
gender_group
)16. Create separate datasets
Men:
men <- latino |>
filter(gender_group == "Men")Women:
women <- latino |>
filter(gender_group == "Women")17. Weighted estimate for men
weighted.mean(
men$hinojosa,
men$weight_hisp,
na.rm = TRUE
)18. Weighted estimate for women
weighted.mean(
women$hinojosa,
women$weight_hisp,
na.rm = TRUE
)Checkpoint 5
- Which group has higher weighted Hinojosa support?
- About how large is the difference?
- Is the weighted difference similar to what you found in Lab 3 using the raw respondents?
Your response:
Part VII: What weighting can and cannot do
19. What weighting can help with
Weighting can help adjust for known differences between the sample and the population.
For example, weights may account for differences in:
- age;
- gender;
- race or ethnicity;
- education;
- region; or
- features of the sampling design.
20. What weighting cannot guarantee
Weighting does not automatically fix:
- a badly worded question;
- respondents who misunderstood the question;
- people who were completely excluded from the recruitment process;
- bogus or low-quality interviews;
- all forms of nonresponse bias; or
- an important difference that was never measured or used in weighting.
A weighted estimate is not automatically an unbiased estimate.
Weighting is an adjustment tool, not a guarantee of perfect representation.
21. A note about precision
When weights differ substantially, some respondents contribute much more than others.
That means a weighted sample can contain less statistical information than a simple sample of the same raw size.
This is the idea behind effective sample size and design effects.
We will discuss those concepts in lecture.
For this lab, you do not need to calculate them by hand.
Part VIII: From output to a poll statement
Suppose your output gives:
- an unweighted estimate;
- a weighted estimate; and
- a published poll topline.
Ask:
- Which estimate should be used to describe the target population?
- How different is it from the raw sample?
- Does weighting change the substantive story?
- What limitations still remain?
Checkpoint 6: Write one or two sentences describing the weighted Hinojosa estimate. Do not simply report the number; explain what population it is intended to describe.
Your response:
On Your Own
Now repeat the same workflow for James Talarico in the U.S. Senate vote among Latino respondents.
In senvote26_full:
1= James Talarico
Question 1: Create the variable
Create a 0/1 variable called talarico.
# Your codeCheck it with count().
Question 2: Compare raw and weighted estimates
Calculate:
- the unweighted Latino estimate;
- the weighted Latino estimate using
weight_hisp; and - the difference between them in percentage points.
# Your codeWrite two or three sentences explaining what changed after weighting.
Question 3: Compare Latino men and women
Using the men and women datasets, calculate weighted Talarico support for each group.
# Your codeAnswer:
- Which group has higher weighted support?
- About how large is the difference?
- Does the result tell the same substantive story as the unweighted comparison?
Question 4: Inspect the weights
Use:
summary(latino$weight_hisp)In a short paragraph, explain what the range of weights tells you about how respondents contribute to the final estimate.
Question 5: Interpretation
In one short paragraph:
- explain why the weighted estimate is more appropriate than the raw sample mean for reporting the poll result;
- identify one problem weighting can help address; and
- identify one survey-quality problem weighting cannot automatically solve.
What to remember
The practical workflow is:
# Raw estimate
mean(outcome, na.rm = TRUE)
# Weighted estimate
weighted.mean(
outcome,
weight,
na.rm = TRUE
)The important questions are:
Which population am I describing?
Which weight belongs to that population?
How much does weighting change the result?
What problems can weighting not solve?
Submission
Submit the completed QMD source file and rendered HTML file.
Before submitting:
- restart R;
- set your working directory;
- render the document;
- make sure every code chunk runs;
- check that your written interpretations match the output; and
- make sure you used
weight_hispfor the Latino analysis.