Most stores test a shipping threshold by moving it, looking at average order value a fortnight later, and declaring a result. That method will give you an answer every time, and the answer will be unreliable, because average order value moves for reasons that have nothing to do with the line.
A threshold change alters two things at once. It changes what a customer has to spend to avoid a charge, and it changes what you give away on every order that crosses it. A test that watches only the first half is watching the part that always looks good.
This piece is about the test design rather than where to put the line. The design matters more than people expect, because a badly run threshold test does not just fail to answer the question, it produces a confident wrong answer that then sits in the store for years. Four decisions make the difference: what you measure, how you separate the groups, how long you run, and what you agree in advance would count as a failure.
Write down the success measure before you touch the setting
The measure is contribution per order, not average order value. Contribution is what is left after product cost, shipping cost and any discount, and it is the only number that notices when a bigger basket cost you more to fulfil than the extra items brought in.
Write the formula down for your own store first: order value, minus cost of goods, minus what you actually pay to ship that order, minus discounts. Then take the average across all orders in the period. That single number is what has to improve for the test to have worked.
Alongside it, keep two supporting numbers. Total orders, because a threshold that raises basket size while quietly deterring small orders can leave you with fewer customers and less profit. And items per order, because it tells you whether people are adding a second product or simply buying a more expensive version of the first.
Writing these down before the change is not bureaucracy. It is the only protection against the very human habit of finding the metric that made the decision look right after the fact.
A threshold test that only reports average order value cannot tell you whether you won.
Read your own order distribution so the test has a hypothesis
A test without a hypothesis is a guess with extra steps. Build the hypothesis from your existing orders.
Export a few months of order totals and put them into bands, in increments that suit your prices. You are looking for the shape: where the bulk of orders sit, how long the tail is, and whether there is already a cluster just below your current line. A cluster right under the threshold tells you the line is doing work. A flat distribution across the line tells you customers are ignoring it.
Also separate first orders from repeat orders, because they usually have different shapes and a threshold affects them differently. And separate by channel, because paid traffic and email traffic often arrive with different basket sizes.
From that, form a specific hypothesis, for example: moving the line from 50 to 65 dollars will pull the cluster of orders currently sitting between 45 and 55 dollars upward, and the additional items will carry enough margin to cover the shipping we give away on the orders above the line.
Now you have something a test can confirm or refuse.
Choose between a split test and a before-and-after, honestly
There are two ways to run this and they suit different stores.
A split test is better if you can do it. Half the visitors see one line, half see the other, at the same time, in the same conditions. Some platforms and apps allow this for shipping rules; many do not, since shipping settings tend to be store-wide. If you can only split by market or region, you can still learn something, but only if the regions behave similarly, which you should check on last year's data before you rely on it.
A before-and-after is what most stores end up running. It is acceptable if you control the calendar. Compare equal-length periods, avoid any period containing a promotion, a product launch, a big press mention or a seasonal peak, and check that traffic mix is roughly the same in both, because a month with more paid traffic will have a different basket profile regardless of the line.
Whichever you pick, change one thing. If you move the threshold and add a progress bar at the same time, you have tested a package and learned nothing about either part.
Give it a full purchase cycle and enough orders to mean something
Two weeks is the default and it is usually too short. The right length is whichever is longer: a full purchase cycle for your product, or long enough to accumulate a number of orders in each group that would not swing on a handful of outliers.
You do not need a statistics degree for the second part. A practical check is to ask what happens to your result if you remove the three largest orders in each group. If the conclusion flips, you do not have enough data yet. Keep going.
For the first part, the reason to cover a full cycle is that thresholds change buying rhythm as well as basket size. A customer who used to place two small orders a month may now place one larger one. Over a fortnight that looks like a win. Over a quarter it may be the same revenue with half the fulfilment cost, which is a real gain, or the same revenue with one fewer opportunity to delight them, which is not.
And resist the urge to peek and stop early on a good week. Stopping the moment the number looks right is the most common way a test produces a false positive.
The ways stores fool themselves with this test
The first is running it through a promotion. A discount code changes basket behaviour entirely and will overwhelm any threshold effect.
The second is ignoring shipping cost changes. If your carrier rates changed mid-test, or you started shipping a heavier product, the giveaway is a different size than it was and contribution moves for reasons unrelated to the line.
The third is comparing a period with a big new-customer push against one without. New customers buy differently from returning ones, so a shift in the mix changes the numbers on its own.
The fourth is judging on revenue rather than contribution, which is the original sin discussed above.
The fifth is forgetting the customers who never got to the cart. A high threshold can deter someone at the product page, where you will never see them as an abandoned cart. Watch product-page-to-cart rate alongside the rest, because that is where this effect hides.
The sixth is testing on the whole catalogue when the problem is one product line whose weight makes free shipping unaffordable at any realistic order size.
What to do with the result, including a null one
If contribution per order improved and order count held, keep the new line and write down the date, the numbers and the reasoning. That record is what stops the decision being relitigated in six months by someone who saw one bad week.
If contribution improved but orders fell, look at the size of each effect. Fewer, better orders can be the right answer for a store with fulfilment constraints, and the wrong answer for one trying to build a customer base for repeat purchases. This is a business decision, not a data one, and it should be made deliberately.
If nothing moved, that is a real result and it is useful. It usually means the threshold is not what limits basket size in your store, and the next thing to try is the offer that sits near it: a genuinely relevant add-on at the cart, a bundle built from what people already buy together, or a multi-pack.
Then leave it alone for a while. Thresholds set customer expectations, and a line that moves every quarter teaches people to wait for a better one.
When we set a threshold in a plan, we set it from the store's own order distribution and margin, and we say plainly what the test needs to show before it stays.