Testing triggers for generosity

Do you think you’d be more likely to donate when a nonprofit gives you five project choices or just features one? And how close to the deadline of a matching offer would you be most compelled to give? That’s what we sought to find out with a recent email test…

In a 2012 project partner survey, we found that our nonprofit partners want more reliable funding from GlobalGiving. They wanted to be better able to plan and budget in advance; more monthly recurring donations from GlobalGiving donors would help them do so. In 2012 and 2013 we experimented with new ways to boost recurring donations, and one of our most successful techniques was a matching offer for new recurring donations that we emailed to our e-newsletter list.

Looking for ways to optimize returns on this tried and true email campaign this year,  we tested two different hypotheses in our September 2014 send:

  • First, would offering people a choice of five projects to support outperform an offer presenting them with just one choice?
  • Second, would it be better to send the appeal on the day before the deadline or the day of the deadline? We measured success by tracking the dollars raised per one thousand emails sent.

Cell 2 The metric of success in this experiment was dollars-per-thousand-emails-sent, or $/1K sent. This is a metric that is helpful at comparing performance of appeals across lists of varying sizes and levels of engagement, and is one of the key metrics featured in M+R’s annual benchmark study on the online activity of nonprofits. We also regularly track open, click-through rates, and conversion rates to learn about engagement with our emails.

We created four test cells to generate emails containing donor history-based personalization. We created four equally-sized randomized segments of our audience for this promotion. The subject line was kept constant across all cells. The four test cells were:

  • Cell 1: One project + sent the day before the deadline
  • Cell 2: Five projects + sent the day before the deadline
  • Cell 3: One project + sent the day of the deadline
  • Cell 4: Five project + send the day of the deadline

Here’s what happened:

The people who had five projects to choose from were more likely to give than those who just had one, and the emails sent the day before the deadline raised more funds than those sent the day of the deadline.

Four-cell test results

Here’s what we learned, and what we’ll do in the future:

1. Five choices are better than one. Although people clicked through the email at slightly lower rates when presented with five projects, (1.1% click-through rate for five projects versus 1.5% click-through rate for one project) readers of the five-choice emails were 27% more likely to give than those presented with just one project – and that’s a statistically significant finding. Going forward, we’ll now feature more than one project option, and we’ll hone our findings by testing five options versus four, three, or two.

Significance Test

2. Give ‘em some notice. The average $/k emails sent for the day-before emails was $135, and the average for the day-of-deadline emails was $118, making the day-before emails the winner. There was no statistically significant difference in the open rates, click-through rates, and conversion rates of these emails; the only difference that stood out was donation amount: the people receiving the appeal the day before the deadline made donations that were 12% larger than those receiving the appeal the day of the deadline. This wasn’t an expected outcome, so we’ll see if this trend continues during future promotions of the same kind. Until we figure out more, our plan will be to send an appeal the day before the deadline with a follow-up the day of the deadline.

 

 

 

1 Comment

Comments Closed