Biz Tips: Organizing Growth Experimentation

Biz Tips: Organizing Growth Experimentation

GROWTH:

Organizing Growth Experimentation

The Right Way to Conduct Growth Experimentation

https://medium.com/media/4a1c0ca20dae06053ac8c59ebd5cbe96/href

Oooooh, a new shiny object! 💠 Marketing technology continues to evolve at a far faster pace than marketing teams are able to internalize, interconnect and leverage. Added to that fact is that while in theory, more data could mean better signals, in practice, it often means more noise that drowns out the signals.

We love to say we’re conducting “experiments” because it sounds scientific. We love to say we’re only trying something as a “pilot” because it sounds non-committal and thus something that can hopefully avoid bureaucratic processes.

Understandable. Note, however, that while you may want to run a pilot to test a new martech tool, we should not confuse that with an experiment. An experiment tests a falsifiable hypothesis. As marketers, our hypotheses should be about who is using our product, why, in what way, and how much value (and what kind) our product is delivering.

Any learning process based on the scientific method is experimentation-centric. The handy graphic below is a good refresher on what exactly the scientific method is:

Image borrowed from newmr.org

We can and should take a more disciplined approach to marketing experimentation. An ideal approach is: 1) is truly governed by the scientific method, 2) increases institutional knowledge, and by extension, competitive advantage, and 3) results in a better experience for our customers.

I’m going to walk through how we organize our marketing experiments, using our own product, GLIDR. The basic steps are:

  1. List and prioritize your assumptions.
  2. Structure and conduct a scientifically valid experiment.
  3. Organize and analyze the evidence.

If you’re familiar with the Build/Measure/Learn loop, this is essentially the same concept.

Assumptions:

Every idea has some assumptions behind it. For the sake of example, let’s say that I’m interested in testing my hypothesis that our product will resonate better with a particular audience segment. “Resonance” will be measured by website visitor to free trial conversion. That’s a fairly simple hypothesis and at first glance, one might not notice any related assumptions I’ve made. However, there are:

  • I’m assuming that the average reader comprehension for my product messaging is healthy. If it’s not, and nobody understands my product messaging, it doesn’t make a lot of sense to test segment vs. segment performance.
  • If average reader comprehension is healthy, I’m still assuming that we have a good baseline of data for the control group segment.
  • I’m assuming that my value proposition resonates with the test group.
  • I’m assuming that the website visitor to free trial conversion rate is a good quantitative indicator of how well my product resonates.

Listing our assumptions like this can help us evaluate the experiment results in richer context. It can also help us decide which crucial assumptions are underpinning our business and therefore, which should be tested through experiments. In fact, given the listed assumptions above, we should probably move on to a messaging comprehension test soon after we’ve completed testing our audience segmentation hypothesis.

Creating an assumption in GLIDR

Experiments:

To test my hypothesis, I’m going to use Google’s new Optimize/Google Ads (né Google AdWords) integration as an experiment. The test group in this example is UI/UX Designers and User Researchers. The first step is to define the experiment:

Creating an experiment in GLIDR

Then we go into Google Optimize and set up the experiment. As shown below, we did a simple 50/50 traffic split. By the way, this was a real experiment we ran and the custom landing page is here.

Structuring the experiment in Google Optimize

Staying in Google Optimize, we adjust the targeting so that we’re only experimenting with traffic that originates from our UI/UX Designer, User Researcher search campaign:

Tuning targeting in Google Optimize to work with our Google Ads campaign

Analyzing the Evidence

In the case of this simple A/B landing page copy test, Google Optimize will tell us when we’ve collected enough data to determine a winner. We would then bring that evidence back into GLIDR so that we could easily share the results and create a searchable historical record of the experiment, results and consequential decisions we made.

As shown above, it doesn’t have to be difficult. It’s entirely possible to invest no more than an hour setting up an experiment that might yield real growth insights. If you can get in the habit of running experiments like these, you can help experimentation become part of your company’s culture (or at least your team’s culture).

In summary, we marketers would better serve ourselves and our companies if we were to conduct more meaningful experiments, and share the results and analyses across at least our product teams and marketing teams.

Additional Resources:

Some of the best resources on experimentation are coming out of the product management field. It was clear at this week’s “Industry” product management conference in Cleveland that the line between marketing and product management continues to blur. If you aren’t already close with your company’s product managers, find them and take them out for coffee ☕️.

Have your own favorite resources for growth experimentation? Name them in the comments below!

Thanks for reading The Marketing & Growth Hacking Publication

Follow us on Twitter. Join our Facebook Group. Contact us for a sponsored post. Get early access to our job board, GrowthJob.

If you enjoyed this story, please recommend 👏 and share to help others find it!

https://medium.com/media/84161f08cf29a541b8d17683fee648c4/href


Organizing Growth Experimentation was originally published in Marketing And Growth Hacking on Medium, where people are continuing the conversation by highlighting and responding to this story.

Join The Rockstar Entrepreneur Community Now: Start Rockin Now

Similar Posts:

Leave a Reply

Your email address will not be published. Required fields are marked *