Abstract: This investigation measured the efficacy of completely handwritten and partially handwritten pro-choice postcards sent to modeled pro-choice Arizona voters ahead of the 2022 primary. A randomly selected subset of 60,000 voters who met criteria (Democratic-leaning, mid-low turnout propensity, modeled pro-choice, Arizona registered voters) were randomly assigned to received a completely handwritten postcard detailing the 15 week abortion ban passed by the GOP-controlled state government, a partially handwritten postcard with the same message, or no communication. The analysis did not reveal statistically significant effects, likely due to being statistically underpowered, but trends indicate that the completely handwritten postcards performed better in terms of voter turnout than both partially handwritten postcards (by 0.52%) and no communication (by 0.39%). At the same time, trends indicated that the partially handwritten postcard was less effective than no communication in this sample. Further, there was a marginally statistically significant interaction between the two postcard conditions and pro-choice issue scores in the treatment-on-treated analysis,  such that at higher levels of issue support (closer to 100), the partially handwritten postcard was more effective, and at lower levels of issue support (closer to 70), the completely handwritten postcard was more effective. While these results are not statistically significant, the trends suggest that the fully handwritten postcards were associated with a small, positive boost to voter turnout, as expected, but that partially written postcards with the same message did not share this positive effect. Initial indications suggest that completely handwriting postcards is key to their efficacy. 

Objective: This study investigated the difference in efficacy between completely and partially handwritten postcards to voters. It also attempted to target voters who care about abortion rights with a message about abortion rights, in line with our values-matched messaging research.

Background: This study was a randomized controlled trial (RCT) that investigated the effects of sending completely and partially handwritten informational GOTV postcards to voters ahead of the 2022 primary election in Arizona on primary election voter turnout. To our knowledge, this is the first head to head study that compares completely handwritten and partially handwritten postcards to voters. 

Specifics: 

This study was a randomized controlled trial (RCT) designed by SDAN. The study targeted a randomly chosen subset of 60,000 registered Arizona voters who met the inclusion criteria: 1) ‘Biden surge’ voters (people who voted in the 2020 general election but did not vote in the 2016 general election, and may or may not have voted in the 2018 general election); 2) modeled support scores of  70+ on pro-choice; 3) 80+ scores on partisanship; 4) old enough to have been eligible to vote in 2016. Of the voters who met criteria, 30,000 voters were enrolled in the control condition and 15,000 voters were enrolled in each of the two postcard treatment conditions.

In the control condition, targets received no communication. In the postcard conditions, targets received a postcard with the same image, script, and highlighting/marks of emphasis. In the completely handwritten condition, all elements of the postcard were handwritten. In the partially handwritten condition, only the address, the greeting, and the volunteer name were handwritten, but the main message was printed. Postcards were delivered to homes from approximately July 20-22, 2022. After the election, the study data was matched back to the TargetSmart voter file to determine if targets voted in the August 2, 2022 primary election.

Basic Takeaways:

• People in the handwritten postcard condition voted at the highest rate (19.54%), followed by the control condition (19.15%), and partially handwritten postcards had the lowest turnout rate (19.02%). These differences were not statistically signficant in the regression model (ps = 0.170 and 0.734, respectively).
• The efficacy of both types of postcards was unaffected by voter turnout propensity, an indicator of how likely voters were to vote already (ps > 0.44).
• The interaction between condition and modeled pro-choice  issue support score, how likely someone is to support legislation to uphold abortion rights, was marginally statistically significant (p =0.104). The interaction indicated that completely handwritten postcards were more effective for voters with lower pro-choice  scores (closer to 70), and partially handwritten postcards were more effective for voters with higher pro-choice scores (closer to 100). 
• This study was underpowered, which likely contributed to non-statistically significant results. However, trends in the sample indicate that the handwritten aspect of handwritten postcards does matter, as the completely handwritten postcards condition outperformed the partially handwritten postcards condition by 0.52%.
• The group of people enrolled in this study reflect a fairly narrow group: registered Arizona Biden surge voters who are left-leaning, care about abortion rights, and have mid-low voter turnout propensity. This means the results cannot be widely generalized.

Key findings: 

People who received completely handwritten postcards voted at a slightly higher rate than people who did not receive postcards, and people who received partially handwritten postcards voted at a slightly lower rate than people who did not receive postcards.

• Voter turnout in the handwritten postcard condition was 0.39% higher than turnout in the control condition. Voter turnout in the partially handwritten condition was 0.13% lower than voter turnout in the completely handwritten condition. These differences were not statistically significant in the regression model (ps = 0.170 and 0.734, respectively).

The efficacy of the postcards was not affected by the targets’ predicted likelihood of voting in the general election.

• There were no statistically significant interaction effects of turnout x condition, nor did they trend towards marginal significance (ps > 0.44), indicating no significant moderating effect of predicted turnout on the efficacy of condition.

Targets who were predicted to be more pro-choice were more likely to respond well to the partially written postcards and targets who were predicted to be lower on the pro-choice score range (70-100) were more likely to respond well to the completely handwritten postcards, but results were marginally significant.

• Completely handwritten postcards appear to have worked better on people who had lower pro-choice support scores and partially handwritten postcards appear to have worked better on people at the top of the pro-choice scores (p = 0.104). 

Caveats and Considerations

• Generalizability is limited. The study also focuses on a very specific group of people: Biden surge voters who are pro-choice, Democratic-leaning, and live in Arizona. This indicates that these results should not be widely generalized, but rather serve as an initial data point. 
• We are underpowered to detect some effects. This study was statistically underpowered at its current sample size and may have affected the statistical significance of results. A larger sample would provide more certainty about these initial results. 
• Primary elections are quieter than general elections. Primaries are quieter election contests than general elections, with voters receiving fewer election messages and materials. However, the statewide contests for Governor and US Senate did not have competitive primaries, which may have attenuated turnout in this election for Arizona voters.

Contributions and Future Directions:

The conclusions in this study are hard to parse since the study is underpowered. However, the results suggest that, as expected, completely handwritten postcards boosted turnout more than doing nothing. But partially handwritten postcards did not perform similarly to completely handwritten postcards, with this condition having the lowest turnout in the sample. This initial suggestive result indicates that there is something important about the handwriting on volunteer-written postcards to voters, and that partial handwriting may not carry the same quality for recipients. Interestingly, there was a marginally significant interaction between issues score and the treatment condition such that the partially handwritten postcards performed better with people at the top of the issues support score band in the study (70-100), but completely handwritten postcards performed better with people at the lower end of this score band. 

Future studies that focus on a comparison between completely and partially handwritten postcards should consider using larger ns in more competitive elections that elicit higher voter turnout. They may also explore different issues with more generalizable populations. 

A longer report can be found here if you are interested in learning more about this study.

SDAN’s commitment: It is SDAN’s intention to provide as much context as possible to allow for the nuanced interpretation of our data. SDAN’s convention is to contextualize effects by reporting p values, confidence intervals, and effect sizes for all models tested (these items may be in the longer report linked in the blog). Additionally, SDAN always differentiates between planned and exploratory analyses and a priori and post hoc tests, and reports the results of all planned analyses regardless of statistical significance. If you have questions about these findings, please email Mallory.