John Geer

Marketing Attribution for Fun and Profit

15 Minute Read

Summary

To help more companies avoid the rocky shoals of misallocated marketing, here is an overview of some of the most popular ways to measure the impact of marketing (sometimes generally called “marketing attribution”).

The most effective marketing often comes from using a combination of attribution methods. To help keep everyone honest, many of those combinations should include at least occasional experiments.

Why Do Marketing Attribution?

Failory’s surveys estimate that over half of startups fail because of marketing mistakes. If this is the case, your understanding of marketing attribution may matter more to your company’s future than your legal, financial, or team dynamics understanding.

However, measuring the impact of marketing is challenging. To make things even harder, many of the experts and organizations you will meet on your journey benefit from more marketing spend. As a result, they would prefer that you use attribution methods that overestimate the impact of your marketing.

This review of attribution methods will help you understand the questions to ask and when one method may be more accurate than another.

This is a longer article, so if you want the highlights, jump to the experiments section. :)

The Default: Last-Touch Attribution

What It Is

A visitor’s individual activity is tracked, particularly whether they are shown your ads and if they make a purchase. If they do buy something, credit for that purchase is assigned to the ad they saw just before that purchase.

This is the default attribution method of several online ad platforms, so it is the first tool you should learn about in your quest to protect the company.

Assumptions

  1. Accurate Spending & Sales Data
  2. Impact Stability
  3. Continuous Tracking
  4. Only Marketing Causes Sales
  5. Single Touch

In order for last-touch attribution to accurately express the revenue caused by your campaigns, we have to make a few assumptions.

We have to assume we have accurate spending, sales, and tracking data (assumptions 1 and 3 above). In order for this approach to work, we have to be able to track most individuals.

If someone sees your ad and makes a purchase, we are also assuming the purchase is caused by the ad. Not because they heard about your brand from a friend or because they previously made a purchase and liked the product.

In order for this last-touch story to work, we also have to assume only one ad is enough to cause a purchase (assumption 5). This is more likely to be true for smaller, more impulsive, purchases.

When It Works

If these assumptions are true, it is a great attribution method!

The tracking assumption is most accurate on ad platforms where people have to sign in. Think of ads and transactions inside Facebook, Google, or Amazon. It also helps if purchases are made relatively quickly after seeing an ad, so there is less time for browser caches to be cleared or someone to switch devices.

The assumption of “Only Marketing Causes Sales” is most accurate for brands just starting out. If you don’t have much of a reputation yet, this assumption is solid. However, if you are well known, this gets hard to defend with a straight face.

When It Likely Goes Wrong

Sales With Non-Marketing Causes

This doesn’t measure whether ads cause sales. It simply assumes that they do.

If you have a strong brand with a particular audience, there is a good chance many purchases are caused by your reputation instead of your ads. In this case, last-touch attribution may notably overestimate your marketing’s influence.

Visitors About to Purchase

This attribution method focuses on customers who are very close to purchasing. But, often there is earlier interaction with those customers. Some marketing may focus on introducing potential customers to your product rather than trying to trigger a sale. Even if those campaigns are profitable, last-touch attribution will undercount them and make them seem unprofitable.

Encouraging Cheating

Because this approach doesn’t measure causation and works on short time scales, it can be relatively easy to cheat.

If someone can predict who is more likely to purchase (perhaps by focusing on the groups that are already customers), they can show them low quality and cheap ads then claim that purchases made are because of those ads. If you are an ad platform or a marketing agency that wants to justify more spending (or fees), this is a dishonest way to do that.

The Refinement: Multi-Touch Attribution (Static)

What It Is

Like with last-touch attribution, a visitor’s activity is tracked. If they purchase, this approach also assigns credit to ads they have seen. The main difference is that this method may assign partial credit to many ads.

There are many variants of this approach that distribute credit a little differently. Some assign credit evenly to all ads (linear), just the first (first-touch), primarily the first and last (u-shaped), more towards the beginning (time decay), etc.. You get the idea.

Last touch can be seen as a special case of static multi-touch, where all credit is assigned to the last ad.

Assumptions

  1. Accurate Spending & Sales Data
  2. Impact Stability
  3. Continuous Tracking
  4. Only Marketing Causes Sales
  5. Single Touch

This makes many of the same assumptions as last-touch attribution. But it doesn’t have to assume only a single interaction matters (though it can).

When It Works

This can work well for companies that are doing a good amount of marketing on different channels, but that still don’t have much reputation just yet.

Even though these approaches can acknowledge that more than a single ad is needed, it tends to work best when the purchasing process is relatively quick. If someone considers a purchase for too long, the tracking is more likely to break.

When It Likely Goes Wrong

Sales With Non-Marketing Causes

Multi-touch attribution also doesn’t measure how many purchases a campaign causes.

If you have an audience that knows and likes you, this attribution method will claim more credit for your ads than they deserve.

The Hopeful: Discount Codes

What It Is

Unique discount codes (or links) are put on different ads. When used, the codes are matched back to the ads they were shown on to assign credit for sales.

Notably, not all coupon codes are designed to provide attribution information. Sometimes, they are more of a pricing strategy (“price discrimination”). The highest price is offered by default. But, to also reach a more price-sensitive audience, companies can offer discounts. Some companies even have policies of offering discounts to those who call in and ask.

Assumptions

  1. Accurate Spending & Sales Data
  2. Impact Stability
  3. Continuous Tracking
  4. Only Marketing Causes Sales
  5. Single Touch

This has many of the same assumptions as last-touch attribution. However, we are trading out confidence in continually tracking visitors for the hope that the impact of an ad is relatively stable over the time period we are considering.

When It Works

If you are new to an area and start with direct mail or some other less sharable marketing medium, this can give you a sense of campaign performance.

When it Likely Goes Wrong

Sales With Non-Marketing Causes

Discount codes also don’t estimate how many purchases a campaign causes. It simply assumes all purchases that use a discount, are caused by that discount.

The assumption that only marketing causes sales is suspect (and sometimes self-serving) in the best of times. Here your customers have a financial incentive to play along.

The Showman: Multi-Touch Attribution (Algorithmic)

What It Is

Visitors are tracked and credit for sales are assigned to ads they saw.

Like static multi-touch attribution, this can assign partial credit for a purchase to many marketing channels. The difference here is the distribution of credit is based on what is most predictive of purchases in past data.

Assumptions

  1. Accurate Spending & Sales Data
  2. Impact Stability
  3. Continuous Tracking
  4. Only Marketing Causes Sales
  5. Single Touch

This is pretty similar to static multi-touch attribution. But, we trade one assumption for another. We let go of the assumption that we know which ads had the most impact (first, last, all of them, etc.) and derive that from the data. However, to do that, we have to make another assumption: that the impact of an ad is pretty much the same now as it was in the data we used to estimate the distribution (2. Impact Stability).

In order for this to work, we also have to have some historical data to build the models with. So, it is usually not a first attribution method, but something you can switch to after a little while.

When It Works

This works well in many of the situations that static multi-touch attribution works.

If the impact of a channel is pretty stable over time (the market isn’t changing too quickly), this can be a good adjustment from a static multi-touch attribution. When possible, data-informed estimates do tend to be better than guesses.

When it Likely Goes Wrong

Sales With Non-Marketing Causes

Yup, it has the same problem of all the attribution methods mentioned so far: it doesn’t estimate whether an ad causes a sale. I mention this so much because it is so important to remember.

When considering an algorithmic method, it is worth asking whether it will assign credit for a sale even if none of the channels are particularly predictive of a purchase. If it does, this can make marketing look much better than it is and encourage companies to spend more in the wrong places.

Data Isn’t Collected Correctly

With this approach, we are starting to analyze the data like an experiment. But, we often aren’t collecting the data like an experiment.

In order to tell the difference between two campaigns, we need them to focus on different audiences or at different times. In order to find the right spending level, we need to try a campaign at different spending rates.

If a given algorithmic approach doesn’t suggest coordinating your campaigns in experimental ways, it is likely missing a big opportunity.

The Scientist: Visitor-Randomized Experiments

What It Is

Individual visitors are randomized into experimental categories and shown different (or no) ads. The differences in purchases between those groups is then used to estimate the impact of the ad changes.

Assumptions

  1. Accurate Spending & Sales Data
  2. Impact Stability
  3. Continuous Tracking
  4. Only Marketing Causes Sales
  5. Single Touch

This still requires tracking and accurate sales and spending data, but we are able to directly investigate one of the biggest assumptions of the other attribution methods: that purchases are caused by the ads shown.

When It Works

Now we are getting serious about not crashing the company! With experiments, we can take a principled approach to estimating how many sales are caused by an ad campaign.

This approach works well if we have great tracking of users, like on the ad platforms themselves. For example, on Facebook’s site they can reliably track whether an account has been shown an ad or not. However, when a visitor leaves Facebook to make a purchase, the tracking quality goes downhill.

When it Likely Goes Wrong

Tracking Breaks

Science can be a magical thing. It is our best method for figuring out our complex world. But, it often requires high quality observational data (or a very good understanding of exactly how the data is biased, so we can compensate).

Some studies estimate that 30% of internet users are using ad blockers. Apple’s Safari browser (the standard on iPhones) blocks all third-party cookies by default (which are essential for tracking users once they leave ad platforms). For many, the ability to track individuals across sites may already be broken.

When individual tracking breaks (assumption 3), many of the attribution methods above also fall.

The Economist: Group-Randomized Experiments

What It Is

These are experiments in which groups of people (such as cities or neighborhoods) are randomized into different experimental categories.

For example, some cities may be shown an ad while others aren’t. The change in purchases after showing the ad is used to estimate its impact.

The analysis of these experiments can be complex. Usually there are differences between the groups that need to be compensated for in order to get an unbiased view of a campaign’s impact.

Assumptions

  1. Accurate Spending & Sales Data
  2. Impact Stability
  3. Continuous Tracking
  4. Only Marketing Causes Sales
  5. Single Touch

This is like the Visitor-Randomized Experiments, but we trade the assumption of being able to continuously track individuals for the assumption that the impact of a campaign is relatively steady between cities (or across time).

When It Works

Like above, these experiments help us directly estimate how many sales were caused by an ad campaign! This is a huge step towards honestly understanding our marketing. Understanding this is especially important for companies with name recognition among a certain audience.

Group-based experiments can be perfect for measuring big campaigns in which tracking individuals isn’t an option, such as radio or TV campaigns.

When it Likely Goes Wrong

Because we can only randomize a few big units, we get a much less precise estimate of the impact than if we could randomize lots of small units (people). As a result, it can be pretty easy to miss small changes among all the noise. This approach isn’t very effective at comparing changes like different headlines or images.

The Satellite: Marketing (or Media) Mix Modeling (MMM)

What It Is

This approach associates changes in marketing spending with changes in revenue, often based on timing and location. This works especially well if the spending was structured like an experiment. Then the change in revenue (or lack of a change) is easier to identify.

MMM provides a way to combine the results from many experiments (or quasi-experiments) into an overall understanding of what is influencing revenue.

Assumptions

  1. Accurate Spending & Sales Data
  2. Impact Stability
  3. Continuous Tracking
  4. Only Marketing Causes Sales
  5. Single Touch

This has some of the fewest overall assumptions.

We need accurate spending and sales data, but almost every attribution approach needs that too. We do also have to assume that the impact of a channel is relatively stable over some period of time. How long that time period needs to be depends on what data we have.

In order to understand the association between spending and sales faster, some assumptions about the nature of their relationship are often made. For example, it is often assumed that there is a delay (lag) between spending and revenue lift. It is also assumed that as we spend more we’ll eventually see diminishing returns as we reach all the interested folks in a particular audience.

When It Works

Think of this as a unifying structure across spending experiments. If those experiments are well set up, this can tell us exactly what we want: the optimal spending for each channel and how much revenue those cause.

However, we usually aren’t able to run perfect experiments, or the impact of our marketing is changing. If both of these challenges aren’t too extreme, MMM can often provide some reasonable guesses and suggest what we should try next to learn the most.

When it Likely Goes Wrong

Change

Unfortunately, the impact of a channel only slowly comes into focus. If that impact is changing faster than we are able to focus on it, we can end up with just a big confusing blur.

These models are sometimes built infrequently, which can force one to assume a good bit of stability.

When considering this approach, it is worth asking how often it will be re-fit and evaluated (faster is usually better). Also, ask about the frequency of the data it uses (the standard is weekly, but it doesn’t have to be so) and how long it assumes a channel behaves the same.

Detail

Because the profitability of a channel is often discovered gradually, it’s worth mentioning that this doesn’t provide a great deal of content-level detail. MMM provides an overall view of spending profitability. However, it usually can’t provide enough detail for tactical decisions like which headline or image to use in an ad.

Data

Unfortunately, spending and sales data aren’t always available. Without a system to accurately manage sales data or collect marketing data from many different systems, this approach isn’t really possible.

Complexity

How this model is built depends on how the data was collected and what data is available. For example, sometimes companies are able to use holdout regions to estimate macroeconomic impacts (which can help a lot). However, some marketing isn’t easy to limit geographically, which means large market changes have to be (carefully) estimated some other way.

This complexity means an accurate model can take time to build and requires a good understanding of its structure to interpret and communicate the results well.

Conclusion

As I’m sure you see, none of these approaches are perfect. They are all attempts to understand the complex and noisy relationship between spending and revenue. To do that, the best approaches often combine insights from different methods to inform different types of decisions (budgeting, ad content refinement, bidding strategy, etc.).

While no method is ideal, experiments should be part of almost every company’s approach. They provide one of the best views of how much revenue a campaign causes. It is hard to understate the importance of this information. Without it, the complexity of the world can trick us all, leading us to spend all our money in the wrong place.

Experiments also provide a way for the market to give meaningful feedback. It can sometimes be like playing 20 questions with a defiant child, but it still gives essential clues about what your customers want and how to adapt.

As challenging as attribution is to do well, it is often worth it. Marketing is such a large portion of many budgets. Mistakes here can put almost any company out of business. But, getting marketing right, speaking to your audience in the ways they want to hear you, can do great things for you and your customer.

Further Reading

If you like my work, consider connecting to me on LinkedIn.