Skip to main content
Run a sweep and the results arrive ranked, winner on top. The natural move is to take the top row — it scored best, after all. This page is about why that natural move is often the wrong one, and how the same sweep, read differently, is your best defense against the mistake.
Optimization and overfitting are not the same thing. Overfitting is what happens when you use a sweep as a leaderboard and blindly take the #1 row. Used as a map — showing you which whole regions of parameter space work and which don’t — the very same sweep is what protects you.

​
What overfitting is

A backtest replays real history, and real history is part signal, part noise — repeatable behavior mixed with one-off accidents. When you test many parameter combinations, some will inevitably line up well with the noise: a stop that happened to dodge one bad afternoon, an EMA length that happened to catch one lucky run. Those combinations score beautifully on the window you tested and have no reason to score well on the next one. That’s overfitting: tuning a strategy so tightly to the past that it has memorized the accidents instead of learning the behavior. And the more combinations a sweep tries, the more chances the noise gets to produce a flattering winner. The top of the ranking is exactly where the luckiest accidents accumulate.
A sweep finds the parameters that fit the past best — which is not the same as the parameters that will work next. Treat the best result as a hypothesis, not a guarantee. Past performance does not guarantee future results.

​
Peaks and plateaus

The defense comes from looking at the shape of the results, not just their order. Plot performance against a swept parameter and two very different pictures can produce the same #1 row:

The fragile peak

One combination scores far above everything around it. Move the stop by a tick or the EMA by one bar and the edge evaporates. That sensitivity is the signature of noise: a real market behavior doesn’t switch off because a parameter moved 5%. This is the overfitted pick — impressive on the ranking, unlikely to survive contact with new data.

The robust plateau

A broad, contiguous region of parameter values that all perform decently. The strategy works with a 300stop∗and∗a300 stop *and* a 350 stop and a $400 stop — which tells you it’s keying on something about how the market actually moves, not on one lucky configuration. This is the pick that has a chance of holding up.
The counterintuitive part: a good value inside the plateau beats the peak, even when the peak’s backtest number is higher. The peak’s extra score is mostly rented from noise, and noise doesn’t repeat. The plateau’s score is lower but owned.

​
How optimization defends against overfitting

This is why a sweep is worth running even though it creates the opportunity to overfit: a single backtest can’t tell you whether its parameters sit on a peak or a plateau. Only testing the neighborhood can. The sweep is a sensitivity map of your strategy, and AskFutures gives you two views built for reading it that way:
  • The Parametric heatmap (for two swept parameters) shades every combination by your metric. Robustness is visible at a glance: look for a contiguous block of good cells, not one bright cell surrounded by losers. A strong row or column also tells you something — the strategy cares about one parameter and is indifferent to the other.
  • The Dot Chart (and the one-parameter scatter) shows every combination as a point. A broad cluster of decent results is the plateau; a single dot floating above scattered losers is the peak.
1

Sweep a real neighborhood

Give each parameter a range wide enough to see the shape — not just three values hugging your current setting. You’re mapping terrain, not confirming a guess.
2

Read the shape before the ranking

Open the Parametric heatmap or the scatter first. Is there a region that works, or just a point? If there’s no plateau anywhere, the honest conclusion is that the edge isn’t robust — no row of the ranking fixes that.
3

Pick from the middle of the plateau

Choose a value comfortably inside the good region — even if it isn’t the top-ranked row. The center of a plateau has the most room to stay profitable when the market drifts; the edge of one is halfway to being a peak.
4

Re-test the pick on data it hasn't seen

Save the chosen combination as a new version and run it on a different window — a more recent stretch you didn’t sweep over. A plateau pick that also survives out-of-sample is as much confidence as a backtest can give you. See version and compare.
A useful habit when two combinations look close: prefer the one whose neighbors score better, not the one whose own score is higher. You’re not choosing a number, you’re choosing a neighborhood.

​
Red flags that you’re overfitting

The top combination is great, but one step away in any direction is break-even or worse. The exact opposite of what a real edge looks like.
An EMA of 23, a stop of $437. When the “best” value isn’t part of any sensible region, it was almost certainly elected by noise.
If the idea only works after sweeping four parameters across huge ranges, the sweep isn’t refining an edge — it’s manufacturing one. More combinations tried means a luckier, less trustworthy winner. Fix the logic in chat instead; see when to optimize.
The swept window looks great, the recent months you held out don’t. That gap is the overfitting, measured directly.
Backtest and optimization results are hypothetical and simulated — no real trades were placed, and simulated results are designed with the benefit of hindsight. Past performance, actual or simulated, does not guarantee future results. Always test before you trade.

​
Next steps

Optimization

How sweeps work: ranges, metrics, and the two-tier engine.

Optimize a strategy

Run a sweep by chatting or on the card, and read the heatmap.

Version & compare

Save the plateau pick as a new version and test it out of sample.

Is the backtest real?

Where the AI stops and the deterministic math begins.