A statistically significant sample is one large and varied enough that luck is an unlikely explanation for the result. You will rarely run a formal significance test as a retail trader, but the mindset behind it - distrust small samples, respect variance - is what keeps you from betting real money on noise.
The core problem: noise
Trading returns are noisy. Any strategy, even a coin flip, produces winning streaks and losing streaks. Over a handful of trades those streaks dominate, so a worthless strategy can post a great win rate purely by chance. Significance asks a simple question: could this result plausibly have happened by luck? If yes, you cannot trust it yet.
Two things drive significance
- Sample size: more trades average out randomness. A result over 300 trades is far harder to explain by luck than the same result over 15.
- Effect size vs variance: a large, consistent edge shows through quickly; a small edge buried in volatile returns needs many more trades to separate from noise.
This is why sample size and significance are two sides of the same coin.
Practical significance checks
- The 100-trade floor: treat results under about 100 trades as provisional.
- The outlier test: remove your single best trade. If the edge disappears, it was concentrated in luck, not spread across a repeatable pattern.
- The sub-period test: split the sample in half. If both halves are positive, the edge is more believable than if one half carries everything.
- The condition test: confirm the edge appears in more than one market regime.
Important: significance is about confidence, not certainty. Even a well-sampled edge can fade if the market changes. The goal is not proof - it is lowering the odds that you are trading pure randomness.
Why this matters before real money
Traders blow up not only from bad strategies but from trusting good-looking noise. A 20-trade backtest that shows +0.5R expectancy feels like a green light and is often just a lucky run. Demanding significance - more trades, stable sub-periods, no single-trade dependence - is what stops you funding an illusion. It connects directly to reading your analytics with appropriate skepticism.
Build significant samples quickly
The obstacle to significance is usually time - live trading produces trades slowly. Backtesting solves it. In a simulator you can replay across different years and conditions to reach a hundred or more trades in a session, then use the report to run the outlier and sub-period checks. That is how you earn statistical confidence without waiting a year to gather it.
Statistical significance FAQ
What is a statistically significant sample in trading?
A sample large and varied enough that luck is an unlikely explanation - in practice usually 100+ trades across different conditions, so a few outliers cannot swing the result.
How do I know if my result is just luck?
Check sample size and single-trade dependence. If removing your best trade collapses the edge, or the result rests on under 30 trades, luck is likely.
Is 100 trades statistically significant?
It is a reasonable working threshold, but a small, volatile edge needs more trades than a large, consistent one to be trusted.