How to read a backtest

The question a backtest answers

Only this: had you followed exactly this plan, with exactly these moves on exactly these days, over these years, how would the balance have changed? It knows nothing about why the markets moved, and it cannot say whether the same pattern will hold again.

The numbers, in the order to read them

1. Worst year and biggest drop first. They show what the plan asks you to sit through. A plan you would abandon in its bad year does not have its good years either.
2. Growth per year against the benchmark. The gap between the plan and the C Fund held all year is the whole case for making the moves at all.
3. Beat the benchmark: the share of years the plan won. A plan that wins two years in three is unusual; one that wins every year has almost certainly been picked for that.
4. Moves a year. Each move is a day you must remember and a chance to be wrong about the date. Fewer is better unless the extra moves clearly pay for themselves.
5. Mean, median and standard deviation (Advanced mode) for the shape of the yearly returns. A mean far above the median means a few great years carry the average.

Why every backtest flatters the plan

A plan drawn up by looking at the past was chosen because it fitted the past. If you try a thousand plans, the best of them will look remarkable, and most of what makes it look remarkable is luck that will not repeat. The strategies page ranks saved plans by how they did over the years they were built on, and so it shows this effect at its strongest. Its top rows are an interesting list, not a recommendation.

The years that count: out-of-sample

Every result is split in two. The years before 2018 are the training period, the years from 2018 on are the test. A plan that does well in both has at least survived years it was not tuned to. You can move the split in Advanced mode; moving it later gives the plan more years to be tuned on and fewer to prove itself in.

For a saved strategy there is a stricter test: its return since the day it was saved, next to the benchmark over the same days. Nobody could have looked ahead at those prices. It is short at first and becomes the most telling number on the page as the years pass.

What the backtest leaves out

• Taxes, loans and withdrawals, and the timing of your own contributions. The balance starts as one lump sum and no money is added.
• Your behaviour: the plan assumes you make every move on the planned day. In practice people skip moves in bad years, which is when the plan most depends on them.
• Fund expenses are already inside the share prices, so they are counted; there is nothing else to subtract.
• The I Fund changed its benchmark index in mid-2024. Results that use I before then reflect the old index.
• L Fund histories are shorter than the core funds', and a backtest that uses one starts at that fund's first price and says so.

A sensible way to use it

• Start from the plan you actually hold, or an L Fund, and see what the moves add. Judge the gap, not the headline.
• Prefer plans with few moves and a story you could explain to a colleague. "The market goes up in these months" is not a story.
• Look at the out-of-sample years and the worst year before the growth rate.
• Save a plan and come back to it. The since-saved number is the only one that cannot be argued with.

Terms used here are defined in the glossary; the formulas are on the methodology page.