MMTs – Paid Media Training Deck
Highlights:
What are our testing options?
IMPORTANT: see how much the recommended markets are spending for whichever tactic is being tested to ensure it’s large enough to be stat. sig.
Before you begin, make a copy of this MMT QA Checklist Template and read on!
Google Ads Testing Guide
Overview - Matched market testing for a select few Google Ads tactics presents a unique challenge. This document is meant to be a guide to conduct effective matched market testing based on feedback and data from real tests.
Summary - MMT’s rely on the ability to add or remove spend from a given channel or tactic in the test market. Let’s say there are four tactics we are running from Google Ads - Brand, Non-Brand, Shopping, and Performance Max. If we want to run an MMT for Performance Max, we can run a holdout test which will reduce Performance Max budget in the test market. In theory this should lead to a clean test, but in practice what we have found is that Google Ads will see this decline in investment, and allocate additional funds from the Brand, Non-Brand, and Shopping campaigns to fill the gap. Decreasing Performance Max spend and increasing spend from all other channels invalidates the test, meaning we have to find some other solution to isolate Performance Max’s incrementality.
Recommended Tests - There are two approaches that lead to accurate tests:
Full Channel Test
There will always be the option to do a full Google Ads holdout or growth test, this is a great option for any client with a low amount of spend or revenue.
Baseline > Change Strategy > Run another Full Channel Test
Run the initial full channel test laid out above in #1 to find a baseline iROAS.
Make changes to the strategy (excluding brand from PMAX, shifting budget higher in the funnel, etc.) that we think will increase incrementality.
After 2-4 weeks of the new campaign strategy being live run another full channel tests, then compare the initial test to the second test to verify changes to strategy increased incrementality.
Tactic Holdout — Use if PMax is not live
Typical holdout test of excluding the chosen markets from the tactic you are testing for incrementality
Riskier Tests - These tests get more specific into the tactic level, and are only recommended for clients that spend more than $30K per month on Google Ads.
The “Go Dark” Method - Holdout
Identify a market pair that meets all of the normal criteria for market selection
Turn off all paid search in the test & control markets other than whatever tactic your test is focused on (if testing PMAX exclude both markets from brand search, and NB search, if testing Brand search exclude both markets from PMAX).
Implement a normal holdout test for that campaign tactic
The “Go Dark” Method - Growth
The drawback of this method is that it requires a lot of set up, but will allow individual tactic results for the Search tactics.
Identify a market pair that meets all of the normal criteria for market selection
Turn off all paid search in the test & control markets (Brand, Non-Brand, Shopping, and Performance Max), then wait a month for them to settle to a new level
Run a growth test, the test market receives the budget for the specific tactic being tested (choose one from the following list: Brand, Non-Brand, Shopping, and Performance Max) and the control market does not receive any search budget.
Go through the normal reporting process and discover the incrementality of the individual tactic.
YouTube Ads - MMT Toolkit
Best Practices, Methodology & Important Things To Know
Resources To Get Started:
YouTube MMT Overview:
Holdout Test: Exclude test markets from existing live YouTube campaigns
Growth Test: Net new YouTube campaigns, only targeting the test markets
Heavy Up: Increasing investment beyond the standard existing spend levels
Ensure that the targeting in the platform is done on the DMA level, and aligns with recommended markets from analytics team
Campaign should be optimizing towards frequency
Not conversions / click throughs / etc.
Minimum frequency target: 3 impressions per week
Once the test is done/reaches significance the exclusions can be removed/the growth campaigns can be paused.
Analytics team to provide first look at stat sig information at 3-4 weeks which can help inform more accurate test flight length
Apply learnings to channel/tactics once MMT is finished.
Whether the result is incremental or non-incremental the team can shift strategy to help clients meet their business goals.
Goal(s):
Determine impact of YouTube ads on overall revenue (iROAS)
Determine impact of YouTube ads on customer acquisition costs (iCAC)
Flight Length: 6-8 weeks or when statistical significance is reached
Budget Allocation & Requirements:
Growth test: Planning team will utilize GWI to recommend a budget and help set frequency goal
Frequency goal per user should inform budgetary decisions by funnel stage
Typically we want to focus primarily on ToF for biggest impact while leveraging RT to help hit the frequency per user target
Holdout test: N/A as we are holding out spend in the test market
Market Selection: Provided by analytics team via the MMT request board
Campaign Settings:
YT Campaign Launch Checklist - make copy and run through checklist before launch
Goal Selection
Must select either brand awareness & reach or product & brand consideration as campaign goal to unlock reach and frequency ad types
Ad Types
Non skippable, skippable in stream, bumper ads, ad sequence
Bid Strategies
Max CPV
Target CPM
Target Frequency
Exclusions
Use standard inventory
Create master exclusion list for placement scrubs
Exclude sensitive topics (politics, etc.), apps and low quality placements
Exclude content suitable for families to help avoid childrens channels
YT Campaign Goal Selection
Campaign Structure:
Example MMT Setup
Example Successful Campaign Build Strategies - WIP
Stately Men Growth Test:
Prospecting:
Launched a YT campaign with "target frequency" subtype with a goal of 4 views per week
Budget Allocation: started 60/40 budget split in favor of prospecting, switching to 50/50 split after 2-3 weeks
Audience Targeting:
tFREQ campaign type only allows for one ad group, so we created one persona based on employment in specific fields
based on insights from their team, we were able to identify that men in the finance, education or real estate sector as our target market.
Excluded all women, and men ages 18-24 and 65+ (based on GWI)
Creative was a mix of YT Shorts, standard landscape videos and TV Commercials on (total of 7 different videos)
Longer duration videos (20 - 60s)
Remarketing:
Audience Targeting: product page viewers, video ad viewers from the prospecting campaign for audiences to get frequency up as much as possible
Creative: mainly leveraged TV commercial videos that were shorter in length (15 - 30s) and tested 8 variations
No audience exclusions in place
SpoonfulOne Growth Test:
Prospecting:
Launched 30s skippable instream ad
Audience Targeting:
Parents - detailed demographics
In market for baby products
Custom Intent Audiences
tCPM bidding
Remarketing:
Ad Sequencing campaign which was sequential of a 15s, then a 6s, then a 30s video 2:54
Audience Targeting: Prospecting ad viewers
tCPM bidding
Targeting Best Practices
Leverage provided GWI report to help inform target personas
Keep budget and target frequency per user in mind when determining targeting criteria
Ex: Avoid targeting large audiences when budget won’t allow for it
Work with supervisor to determine the best path forward for targeting
Creative Best Practices:
Optimizations to Make:
Reach / Frequency metrics are located in the columns filters at the campaign level
Optimize MMT campaigns towards frequency:
Frequency goal benchmark: 3 impressions per week
Leverage Target Frequency subgoal / ad type to automate frequency optimizations
Adjusting tCPM and Max CPV targets
Ex: increasing tCPM to influence more reach
Refining targeting to serve smaller audience pool
Ex: Focusing on smaller Custom Intent vs. Affinity
Adjusting frequency / view caps
Reallocating, increasing or decreasing budgets from either specific campaigns or stage of the funnel
Ex: Shifting spend from prospecting to RT video viewers
Push to spend entirety of allocated budget and avoid pulling back spend while test flight is live
Creative Testing & Iteration:
Hook Testing
Highlight brand, showcase USPs, testimonials / social proof, educational, emotional appeal, etc.
CTA testing in end frame
Ad Types & Length tests
Ad Sequencing, non skippable, bumpers, etc.
6 sec, 15 sec, 30+ sec
Measurement:
Results are pulled and compiled by analytics team when test reaches statistical significance
Statistical significance is calculated using a credible interval, which is the likelihood that your variable is above a threshold value.
In most cases we want to ensure the iROAS is greater than 0 for incrementality, and 1 for profitability at a 95% statistical significance. For iCAC, we want to ensure our estimate is below our threshold (ex: Estimated iCAC of $50, a threshold of $100 and a confidence of +95%)