Skip to main content

How to Effectively Use a Testing Framework

How To Use

S
Written by Strategy Organization
Updated over 2 years ago

Submit a Strategy request here

Background:

As good stewards of our client’s dollars, it's essential to constantly validate the efficacy of our marketing program . One powerful tool that can help you achieve this is, developing a testing roadmap. By testing different marketing channels and tactics, you can measure the incremental impact of your campaigns on your target audience and identify the most effective ways to reach them. Are we skeptical of a channel’s contribution to our overall program? Are we looking to validate a new way of purchasing media? Are we bullish on a new marketing campaign, channel, or tactic? Marketing experiments, and structuring a learning agenda through testing roadmaps can help!

Overview:

The media planning team can help facilitate and construct a roadmap that determines testing priorities, allocates optimal resources towards experiments, and can put that into a easy to digest framework to activate against.

Expected Outputs (depending on client requirements):

Examples of questions testing roadmaps can help answer:

  • What is media’s impact to the overall business?

  • What is [channel]’s impact to the overall business?

  • What is the right mix of channels to maximize ROI?

  • What is [tactic]’s impact to the overall business?

  • Which creative concept is more impactful to sales?

Key Considerations when Determining Testing Priorities:

  • The more granular the test, the harder it is to measure

  • The number of concurrent tests is limited by viable markets

  • We will proactively bring ideas or questions

  • Prioritize highest investment or most “suspicious” elements

  • Comparison tests (e.g. this vs that, rather than on vs off) are harder to execute + measure

General Testing Process:

  1. Identity Questions: What are we trying to solve or answer? What matters most to the business?

  2. Assess Risk: What is the limit of lost sales? More risk equates to more/faster insights.

  3. Identify Marketplaces Which (and how many) markets can we run tests that will be stat sig?

  4. Implement Testing Implement the test in the market(s) selected and approved by the client.

  5. Measure Results Measure the impact of the experiment vs the BAU program or compare mkt.

  6. Optimize Program Incorporate insights from the test into the program (reallocate budget, etc).

Pre-Final Testing Roadmap Process:

  • Planning team will collaborate with stakeholders to create a roadmap based on the testing priorities that have been identified

  • Testing roadmap will have an overarching framework for the learning agenda, including markets, budgets, flighting, and optimization/measurement methodologies

  • Once the roadmap is delivered to Client Services and Activation for internal review:

    • Collaborate on what’s possible to spend in a given testing priorities

    • Align upon the specific buying models and allocated services and other tactics

    • If something is impossible (e.g. too much budget in brand search) please flag

    • If a tactic is incapable because of restrictions (no ability to ad sequence) flag

  • Once the planner, CS, and Activation teams are aligned, move forward

Client Approved Plan Process:

  • Finalized roadmap gets presented to client and adjusted until final approval

  • Have a round of discussion with the planner and all teams to align on next steps

  • Structure experiments in accordance with the testing roadmap

  • Once activated, collaborate with the Data Intelligence team to package and share experiment results

  • Optimize program with the learnings

Once Testing is Complete:

After we have executed on the Testing Roadmap, it's crucial to optimize your campaign based on the data collected. This involves analyzing the results and identifying the most effective, or the least effective, marketing channels and tactics. You may need to adjust your marketing strategy by reallocating your budget to the most effective channels, testing new tactics or channels, or refining your messaging and creative to reflect results.

Experiments should be an iterative process. The macro marketplace changes, audience’s perceptions shift, creative messages deteriorate, and we need to be prepared to re-test, re-learn, and re-apply learnings.

Resources:

Did this answer your question?