Biz Tips: The ROAS might look like 0%, but is your display campaign actually working?

Biz Tips: The ROAS might look like 0%, but is your display campaign actually working?

GROWTH:

The ROAS might look like 0%, but is your display campaign actually working?

Is your display campaign working? Use the data to go beyond the KPIs and find out

Understanding ad performance with parallel slopes regression

Photo by Samuel Zeller on Unsplash

A few months ago, I chanced across a digital marketing blog post about switching off a display campaign. They’d looked at a client’s account, seen that the display campaign wasn’t converting particularly well and decided that, by switching it off and putting the budget behind the better converting campaign, the result would be more revenue.

That was not what happened. What happened was that the performance of the other campaign dropped. It would seem that the visibility of the display campaign was working by promoting awareness and familiarity, which ended up driving more conversions through other campaigns.

Should the display campaigns just have been switched off? With the data you have at your disposal as a modern digital marketer, perhaps you could have uncovered the effect of the display campaign — or designed an experiment to test it — before just switching it off.

TL;DR: Marketing activity can be a complicated, inter-connected spider’s web of channels and campaigns. If your campaign report says that one campaign isn’t performing, that doesn’t necessarily mean that it’s not working: it could be creating an environment that allows other campaigns and activity to perform better. So, before you just pause it and re-allocate it’s budget, spend some time with the data.

Follow Chris on Twitter for updates about the space where marketing and data meet.

In this article, we’ll use an approach called parallel slopes regression to look at a synthetic PPC dataset to attempt to uncover whether our display campaign is having any effect. I’ll be using R, but you can apply similar ideas in Excel or your software of choice. Seriously though, if you can write a formula in Excel, you‘re already coding!

The campaigns and the data

Imagine our account has two campaigns: one display and one search. The numbers for our display campaign are quite poor: no transactions and no revenue for the spend. We’ll not mention what that spend was, we’ll come back to that at the end.

Our search campaign is doing alright in terms of performance, let’s take a look at how much revenue it’s generating for its spend:

# Investigate the effect of display advertising using parallel slopes regression
# import data as display_data
library(readr)
display_data <- read_csv("display_data.csv")
glimpse(display_data)
Observations: 40
Variables: 8
$ spend 22.61, 37.28, 55.57, 45.42, 50.22
$ clicks 165, 228, 291, 247, 290, 172, 68
$ impressions 8672, 11875, 14631, 11709, 14768
$ display 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
$ transactions 2, 2, 3, 2, 3, 2, 1, 1, 3, 3, 4, 5
$ revenue 235.52, 179.68, 566.24, 839.04
$ ctr 1.90, 1.92, 1.99, 2.11, 1.96, 1.98
$ con_rate 1.21, 0.88, 1.03, 0.81, 1.03, 1.16
# summarise performance of campaign
library(dplyr)
library(knitr)
display_data %>%
summarise(cost = sum(spend),
clicks = sum(clicks),
impressions = sum(impressions),
transactions = sum(transactions),
revenue = sum(revenue)) %>%
kable()
|    cost| clicks| impressions| transactions| revenue|
|-------:|------:|-----------:|------------:|-------:|
| 1925.83| 11369| 472893| 143| 42968.8|

Right, we’ve spent £1926 and made £42,969 for a return on advertising spend of 22.3. We know that the margins are client works with are good, so this ROAS works for us and, more importantly, the client isn’t making a loss on each sale.

Given the ROAS on our display campaign is 0, should we switch it off and start making £22.30 from each £ of budget we can add to this campaign? Let’s take a look at our campaign in a bit more detail.

Throughout the course of this campaign, we’ve been moving the daily budget about, so that we can run just the sort of analysis we’re about to run: a linear regression. For the purposes of this analysis, we just want to get a quick ‘eyeball’ of any trend, we don’t need to build a predictive model, we’ll just plot the graph and have a quick look at the correlation coefficient. All we really want to know is if our revenue goes up when we spend more on our digital campaign. We can plot that very quickly in ggplot:

# Plot revenue against spend scatter plot
library(ggplot2)
ggplot(display_data, aes(x = spend, y = revenue)) +
geom_point() +
geom_smooth(method = "lm", se = FALSE) +
labs(title = "Daily Revenue Against Campaign Spend",
x = "Daily spend (£)",
y = "Daily revenue (£)") +
theme_minimal()

Just eyeballing the graph, we can see that the trend looks quite strong: the more we spend, the more we make. Time to make a note: how far can you push that spend to the right before it starts to flatten out…?

# Determine R-squared correlation coefficient
lm(revenue ~ spend, data = display_data) %>%
summary()
Call:
lm(formula = revenue ~ spend, data = display_data)
Residuals:
Min 1Q Median 3Q Max
-702.56 -224.90 5.79 260.94 843.95
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 62.82 140.69 0.447 0.658
spend 21.01 2.65 7.928 1.42e-09 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 375.3 on 38 degrees of freedom
Multiple R-squared: 0.6232, Adjusted R-squared: 0.6133
F-statistic: 62.86 on 1 and 38 DF, p-value: 1.415e-09

Looking at the R-squared correlation coefficient above, I think that looks okay, it suggests that about 61% of the variability in our revenue can be explained by what we spend on our ads. If only every campaign could be like that!

The display factor

If we go back and take a quick look at the dataset we have available, you can see that there is a variable called display.

# Look at top and bottom of dataset
head(display_data, n = 5)
# A tibble: 5 x 8
spend clicks impressions display transactions revenue ctr con_rate

1 22.6 165 8672 0 2 236. 1.9 1.21
2 37.3 228 11875 0 2 180. 1.92 0.88
3 55.6 291 14631 0 3 566. 1.99 1.03
4 45.4 247 11709 0 2 839. 2.11 0.81
5 50.2 290 14768 0 3 791. 1.96 1.03
tail(display_data, n = 5)
# A tibble: 5 x 8
spend clicks impressions display transactions revenue ctr con_rate

1 65.3 412 13534 1 6 1651. 3.04 1.46
2 50.2 286 9797 1 4 1646. 2.92 1.4
3 85.1 479 15500 1 7 1644. 3.09 1.46
4 91.5 592 19433 1 9 2829. 3.05 1.52
5 65.3 362 11727 1 5 1269. 3.09 1.38

The display variable takes a value of 0 or 1. When it is 0, the display campaign was off, when it was 1, it was running. What does our scatterplot look like if we colour the points using this extra feature? First, we need to change the variable into a factor, as in this context, 1 and 0 represent groups, not numbers.

# Change display variable to a factor
display_data$display <- factor(display_data$display)
class(display_data$display)
[1] "factor"
# Re-draw scatterplot to show display on or off
ggplot(display_data, aes(x = spend, y = revenue, colour = display)) +
geom_point() +
labs(title = "Daily Revenue Against Campaign Spend",
x = "Daily spend (£)",
y = "Daily revenue (£)") +
theme_minimal()

That looks like it could be interesting, the points where our display campaign is on look as though they’re generally higher on the plot than the points without. Is that telling us that, when we have the display campaign running, we drive more revenue for the same spend on the search campaign than when it’s not running? Let’s investigate that further…

Multiple regression and parallel slopes

By adding the display campaign variable to the regression model we produced above, we can look at the modelled outcome of including the effect of our display campaign on our search campaign performance. As we’re now using multiple explanatory variable, this is called multiple regression.

As one of the variables is categorical — display on or off — this is an example of parallel slopes regression, for reasons which will become clear! We’ll being by building the model and looking at the output.

# Build multiple regression model
mult_mod <- lm(revenue ~ spend + display, data = display_data)
summary(mult_mod)
Call:
lm(formula = revenue ~ spend + display, data = display_data)
Residuals:
Min 1Q Median 3Q Max
-757.68 -182.11 22.26 219.77 617.36
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -165.751 129.116 -1.284 0.207214
spend 21.422 2.209 9.698 1.05e-11 ***
display1 417.151 98.941 4.216 0.000154 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 312.6 on 37 degrees of freedom
Multiple R-squared: 0.7455, Adjusted R-squared: 0.7317
F-statistic: 54.19 on 2 and 37 DF, p-value: 1.012e-11

That’s more like it. Looking at the R-squared correlation co-efficient, we now see that our model is representing 73% of the variation in the data; quite an improvement.

The other two interesting numbers to look at in the above model summary are the estimates for the spend and display variables. The estimate of the spend variable tells us the gradient of the line. The value of 21.42 tells us that as the spend increases by £1, the revenue increases by £21.42 — very similar to the ROAS we calculated earlier.

The estimate for the display variable is where things get a bit “wow”. That figure of 417.15 suggests that, when we have the display campaign running, we make an extra £417.15 for the same search campaign spend than when we don’t. That’s quite a big increase in revenue. We’ll see what that model looks like by plotting our ‘parallel slopes’ regression plot. To do this, we’ll use the broom package to calculate the predicted values of revenue and use those to plot the regression lines on our scatterplot.

# Calculate model values and plot parallel slopes regression lines
# Augment model
library(broom)
aug_mod <- augment(mult_mod)
ggplot(aug_mod, aes(x = spend, y = revenue, colour = display)) +
geom_point() +
labs(title = "Daily Revenue Against Campaign Spend",
x = "Daily spend (£)",
y = "Daily revenue (£)") +
geom_line(aes(y = .fitted)) +
theme_minimal()

And that’s why this is called parallel slopes regression. You can see from the plot that the increase in revenue is driven by the increase in search campaign spend, but that having the display campaign running just helps it work better, putting that line higher up on the plot. There’s one more thing left to consider though.

Summing up the display campaign performance

Earlier in the post, I said we wouldn’t bother discussing the cost of the display campaign. It’s time to do that now. We didn’t need to know how much we were spending earlier, as we were only considering it as a categorical variable of being ‘on’ or ‘off’, rather than considering it as a numerical variable and using its daily spend as we did with the search campaign.

Using this method, we’ve concluded that, even though the display campaign has driven no conversions directly, when it’s there it improves the performance of the search campaign. Now we have a good number for the amount by which that performance is improved, it’s time to include that in the ROAS calculations. If you’re spending £5k a day on the display campaign, it’s costing more than it is generating in improved search campaign revenue; if it’s only costing £5, it’s probably worth leaving on.

Once you have an understanding of how the display campaign is working to augment your search campaign performance, you can include it in your campaign performance calculations. For example, you could calculate your weekly ROAS by considering these two campaigns together, adding the costs of both.

Once you get a feel for the numbers, you can make more informed, data-driven decisions about which campaigns are effective, and which aren’t. You are in a better position to decide if you should leave a campaign on, or if you will drive more conversions by switching it off and moving the budget elsewhere.

This was a synthetic dataset, built in order to illustrate the concepts of the technique. In your real PPC account data, things aren’t always that clear. You might not have as large a spread of daily spends and, with the majority of SME accounts, the conversion data could be sporadic, noisy and you might have to use much longer time periods in order to spot trends.

And, in some cases, you might not always be able to design an experiment where you switch things off and on, or adjust budgets across a wide scale — particularly with a client’s account. In those cases, you might have to be more creative, perhaps breaking things down to the city level to give you more variation to work with.

I think the take home message is that the digital advertising space — and modern paths to conversion — can be quite long and complex (a look at the top conversion paths in Analytics’ multi-channel funnels can tell you that), and channels and campaigns often don’t work in isolation. And if your campaigns work together, maybe their performance should be measured together.

Of course, this technique doesn’t just have to be used for display campaigns, what about applying it to when you have a print or radio ad running? Who says you can’t measure the effect of offline marketing activities…?

The code above and the synthetic dataset is available on GitHub if you were wanting to perform this analysis yourself. There is a lot we haven’t considered here (such as checking the assumptions of our model), so if you’re interested, I encourage you to read more about linear regression, it’s a relatively simple, but very powerful tool.

Follow Chris on Twitter

Thanks for reading The Marketing & Growth Hacking Publication

Follow us on Twitter. Join our Facebook Group. Contact us for a sponsored post. Get early access to our job board, GrowthJob.

If you enjoyed this story, please recommend 👏 and share to help others find it!


The ROAS might look like 0%, but is your display campaign actually working? was originally published in Marketing And Growth Hacking on Medium, where people are continuing the conversation by highlighting and responding to this story.

Join The Rockstar Entrepreneur Community Now: Start Rockin Now

Similar Posts:

Leave a Reply

Your email address will not be published. Required fields are marked *