Marie Curie: how to show your advertising caused the result

Sales going up while your campaign runs doesn't show that the campaign did it. To show that, you have to rule out everything else that could have caused the rise, one thing at a time. Marie Curie's paper on its 2010 campaign does it better than almost any I've read. It won Gold at the IPA Effectiveness Awards in 2011.

I've spent this week reading award papers. This one is a lesson in rigour, and the discipline in it is open to any team.

Marie Curie gives free nursing care to people who are terminally ill. Every March it runs the Great Daffodil Appeal, and just over a quarter of the money comes from volunteers with collecting tins. Most charity advertising asks people to give money. In January and February 2010 Marie Curie advertised, two thirds of it on radio, to ask for something else: an hour of your time, to collect. The whole campaign cost £184,151.

On the face of it, it worked. The charity had 13,774 collectors the year before and over 20,000 that year. That is correlation: two things that happened together. Causation is the harder claim, that one made the other happen. The paper sets out to show causation, and it starts by saying what it didn't have: "Due to the low investment in the campaign, we do not have any tracking research at our disposal."

So it did three things.

First, it looked at the route people came in by. Collectors normally sign up through their local branch. The campaign advertised a national phone number, a web address and a Facebook page, and the paper says every new recruit came in through one of those three. It shows the split as a chart with no numbers on it. Most came through the website. The paper calls only the phone line dedicated, and doesn't say whether the web address was a page of its own.

Second, it set the results against where the ads ran. The radio was heavier in some regions than in others. The paper puts the radio weight in each of six regions beside the new recruits there. The more radio a region had, the more new recruits it had. One region is out of line: Wales and the West had more radio than Scotland or the North and fewer recruits.

Third, it wrote down everything else that could explain the rise, and answered each one. There are six on its list. Here are two.

• The main appeal advertising ran in March, and collectors had to register by the end of February. So the bigger campaign couldn't have recruited them.
• Compared with the March before, March 2010 was colder, less sunny and equally wet. So a fine month didn't bring the collectors out.

That last answer came from the Met Office, and it was free.

The paper is just as careful with what the campaign paid back. Collector numbers were on a rising trend anyway, though they didn't rise every year. So it works the return out three ways, each against a different base, and uses the lowest of the three from there on: £2.45 for every £1 spent, after the cost of the campaign is taken off.

Read that figure with one thing in mind. It covers two years, and the second year is an assumption: the charity believes 46% of new collectors come back the next year. On the first year alone, by my own sum from the paper's figures, the return is about £1.39 for every £1.

One limit. This is one paper, and I can't tell you which part of it the judges liked. It also won the prize for Best New Learning, and the paper's own section on what it learned is about asking people to collect, not about the proof. So the idea may have counted for as much as the rigour.

The paper holds a good deal more than this: the other four causes, the media plan, the charts and the full working of the return. It is worth reading in full. You can read the IPA's summary, and buy the paper, on the IPA's site.

Here is what to take from it before your next campaign.

1. Give the campaign its own way in: a phone number, a web address or a code that nothing else uses. Then count what comes through it. Marie Curie's paper can't say how many came by each route, because it prints no count.
2. If you can, don't run the same weight everywhere, and keep a note of where it ran.
3. Write the list of everything else that could explain a rise. Find one fact for each. Start with the weather.

Sam