Skip to content

How do I test new ad creative?

By cartscaling Reviewed 5 min read

The short answer

Test different ideas, not small edits. Run a small number of clearly distinct angles against each other, give each enough budget and enough time to produce real orders, and judge on cost per order while using the earlier metrics to explain why. Then log what you learned so the next test starts further ahead.

Test angles against each other, not variations of the same one

The most common waste in creative testing is running four versions of one idea with a different colour or a different first frame. Those differences rarely produce a result big enough to detect, so you spend the budget and learn nothing.

Test ideas that could not be confused with each other. A different reason to buy: saves money, saves time, solves a specific problem, looks good, lasts longer, is made differently. A different person speaking: the founder, a customer, a specialist, nobody at all. A different format: a demonstration, a comparison of your own two options, a review being read out, a plain product on white.

Write each one as a sentence before you shoot it. If two of your test assets have the same sentence, they are the same test.

Three to five distinct ideas at a time is usually right. Fewer and you are not learning fast enough. More and each one gets too little budget to say anything.

Once an angle wins, the second round is where variations earn their place. Take the winning idea and change the opening line, the length, the ending, or the person delivering it. Those small changes are worth testing on a proven angle because the audience has already told you the idea works.

And keep the offer and the landing page identical across the test. Changing two things at once turns a creative test into a guess.

Give each asset enough orders to be judged, and enough days to settle

A creative test that is stopped early tells you which asset got lucky on day one. The fix is not complicated: decide before launch what would make you call a result, then wait for it.

The thing to wait for is orders, not impressions or clicks. An asset with plenty of clicks and two purchases has not been measured. Set a minimum number of conversions per asset that you would be comfortable acting on, work out roughly what that costs at your current cost per order, and fund the test accordingly. If the maths says the test costs more than you want to spend, run fewer assets rather than under funding all of them.

Time matters as well as volume. Automated delivery needs a few days to stop shuffling, purchase behaviour differs across weekdays and weekends, and some products have a consideration gap between the click and the order. A test that spans at least a full week avoids the worst of that.

Give the assets a fair fight. Either separate audiences with equal budgets, or one audience where the platform can distribute spend and you accept that it will favour early leaders. Both are usable, but they answer slightly different questions, so pick one and be consistent.

And resist rescuing a loser. Raising the budget on the asset that is behind, pausing it for a day, or editing it mid test all destroy the comparison. Let it run, take the answer, and put the next idea in.

Read the whole funnel so you know why one won

Cost per order decides which asset wins. The metrics before it tell you what to build next, which is the more valuable output of a test.

Read them in order. How many people watch past the first few seconds tells you whether the opening works. Click through rate tells you whether the promise was interesting enough to act on. Add to cart rate tells you whether the page delivered what the ad implied. Cost per order tells you whether any of it was worth it.

The patterns are informative. Strong attention and weak clicks means the content was entertaining and the offer was invisible. Strong clicks and weak add to cart usually means the ad promised something the product page does not confirm, often a price, a feature or a delivery time. Everything strong until checkout means the problem is not the creative at all.

That last one matters, because it stops you from blaming creative for a checkout leak and burning another month producing videos.

Watch return rates too, on a delay. An asset that overstates a product wins the test and loses on refunds a fortnight later. If you can, look at the return rate of orders by creative before crowning a winner for good.

Finally, be honest about what the test cannot tell you. Small differences in cost per order between two assets on modest volume are noise. Act on the clear results, keep the close ones in rotation, and stop trying to rank things the data cannot separate.

Keep a library and a testing rhythm, because assets wear out

One test in isolation is an anecdote. A routine turns tests into knowledge, and the knowledge is what lowers your costs over a year.

Name assets so the name says what the idea was: the angle, the format, the person, the version. Then tag every result the same way. After a dozen tests you can ask useful questions across all of them, such as whether founder led content beats customer content for this catalogue, or whether demonstrations beat comparisons. Those answers hold longer than any individual video.

Keep the winners in a library with the raw footage, so a proven idea can be recut rather than reshot when it fades.

Then set the rhythm. Plan a fixed number of new assets per month, produced whether or not the current ones are still working, because the time to have a replacement ready is before the decline. The signs of fatigue are a slowly falling click through rate at rising frequency, and a cost per order that drifts up without any change to the audience or the offer.

When fatigue arrives, try a recut before a rebuild: a new opening, a shorter edit, a different thumbnail, a different piece of the same footage. That often buys weeks at a fraction of the cost.

And feed the rest of the business. A creative angle that wins in ads usually deserves to be on the product page, in the email flow, and in the subject line. The test told you what your customers care about, which is worth more than a media result.

Worked example, not a result

Sizing a four angle test

Angles being tested
4
Current cost per order
$25
Orders wanted per angle before deciding
30
Budget needed per angle
$750
Total test budget
$3,000
Planned run length
10 days
Assets kept in rotation afterwards
2
New assets produced next month
4

Worked example, not a result

Related questions

Keep the clear winner plus one or two that were close, so delivery has options and the audience does not see the same thing constantly. Retire anything that has lost twice, and bring the next batch in on a schedule rather than in a panic.

No. Click through rate tells you the promise was interesting, not that it was true. Assets with a strong rate and a weak add to cart rate are usually promising something the product page does not confirm, and they cost more per order than duller ones.

Test fewer things, less often, and accept slower learning. Two angles with enough budget to produce real orders beats six that each get a handful. Below that, judge changes by comparing a fortnight before and after and treat the result as directional.

Keep reading

Want this answered for your store?

Send it. The plan looks at your numbers, not the general case.

Store, name, email and phone. A person replies from a real inbox. Privacy

Get your free growth plan