The single-window backtest trap: how 4-fold Walk-Forward and Regime Matrices expose curve-fitting

Splitting historical data into 70% In-Sample and 30% Out-of-Sample (OOS) is a huge step forward from naive backtesting. Yet a single OOS split still has a blind spot: what if the final 30% of your chart happened to be a clean, relentless bull trend where almost any long rule made money?

Short answer: to know whether a strategy is structurally sound, test it across four rolling Walk-Forward windows and inspect its Bull / Bear / Choppy Range Regime Matrix. If it only survives one regime or fails two of the four time folds, the edge is fragile.

How 4-Fold Walk-Forward works in Evident

In Strategy Studio → Robustness, the test window is divided into four sequential chronological folds ($Q_1, Q_2, Q_3, Q_4$). The strategy's fixed rules and risk parameters are evaluated independently on each fold with full fees and slippage:

• Consistency Score: how many of the 4 folds produced positive net return ($3/4$ or $4/4$ is required for high confidence).
• Fold Drawdown Stability: checks whether a single historical quarter hides a catastrophic drawdown.

The Market Regime Performance Matrix

Every bar in the backtest is classified objectively into one of three regimes:

1. Bull Trend: price above rising medium-term trend with directional strength.
2. Bear Trend: price below falling medium-term trend with directional strength.
3. Choppy / Sideways Range: high Choppiness Index / low ADX where directional follow-through fails.

The report displays trade count, win rate, profit factor and net return for each regime side by side. When you see a trend strategy losing 80% of its bull-market gains inside the Choppy Range row, the fix is clear: add a regime gate (chop(14) < 58.5, adx(14) > 18, or hurst(close, 36) > 0.52) and re-verify.

Test a Strategy in Studio

This is research, not investment advice.