Learn › Survivorship bias in backtesting

Survivorship Bias in Backtesting: What It Is and How to Avoid It

Survivorship bias in backtesting happens when a test includes only the stocks that still exist, or are still in an index, today. The companies that failed, were taken over or were removed drop out of the test, so a strategy can look better than it would have on the stocks available at the time.

What is survivorship bias?

The usual illustration comes from the Second World War, in the form it is often retold. Engineers studied the bullet holes on bombers that came back and proposed armouring the places with the most holes. The statistician Abraham Wald pointed out that those were the places a plane could be hit and still return; the planes hit elsewhere had not come back to be counted.

A stock backtest has the same problem when it starts from today’s list of companies. Every company on that list survived the period being tested. The ones that went bankrupt, were taken over at a low price or shrank out of the index are not on it, so the test never meets them.

How many stocks leave the S&P 500?

From September 2006 to September 2026, 885 tickers were in the S&P 500 at some point (a ticker is a list entry: a renamed company or a second share class counts separately, so there were fewer companies). 382 of them were no longer in it at the end. Over the same years there were 415 removals, more than 382, because a ticker can leave more than once, including ones that later returned. Some are the same company under a new ticker; the rest were taken over, merged, demoted to a smaller index or failed. A test that uses only the 503 tickers in the latest list leaves all of them out. (Our moving-average and RSI studies use 483 of those 503: the ones with usable data that passed the studies' screens.)

8 ticker entries in the list end in Q, the suffix US markets have given the shares of many companies in bankruptcy proceedings (not every bankrupt company gets one). The list records them under that later ticker: most of these companies had already left the index when they filed. Among them are Lehman Brothers, Washington Mutual and the old General Motors.

Which strategies does it flatter?

• Buying and holding: a test on today’s members excludes the companies that failed or left the index before the present, including some with catastrophic losses.
• Buying dips (mean reversion): a test on today’s members misses the stocks whose falls ended in bankruptcy; our RSI mean reversion backtest names this as its main limitation.
• Trend following sells a falling stock early, so it may be affected differently: the stocks it would have sold before a collapse are also missing from the test, which is what our moving average backtest faces.

It also hides a quieter effect. Today’s index includes companies that joined after years of growth. A test that holds them from the start buys future winners before they were large enough to qualify, a form of look-ahead bias.

How to avoid survivorship bias

1. Use point-in-time membership. Include a stock only while it was a member of the index on each date.
2. Get prices for delisted stocks. Many free feeds drop them or reuse their tickers; check each one against its dates.
3. Decide the delisting rule in advance. A takeover can pay cash or shares in another company; a bankruptcy can leave little or nothing. Use a defensible outcome; never simply drop the stock.
4. When you cannot, say so. State that the test uses today’s members, and which kinds of strategy that favours.

How much does it change a backtest?

Our study that reruns a moving-average backtest and an RSI dip-buying backtest on the S&P 500 as it was at each date, delisted companies included, is in preparation.

Common questions

What is survivorship bias in backtesting?

Testing a strategy only on the companies that exist, or are in an index, today. The companies that failed, were taken over or were removed are left out, so the test sees a history with fewer disasters than the real one.

Why does survivorship bias matter?

It can make a strategy look better than it was. A rule that buys falling stocks never meets the falls that ended in bankruptcy, and a test of buying and holding today’s members excludes the companies that failed along the way.

Does survivorship bias always flatter a strategy?

Not always. It removes the worst outcomes, which flatters rules that would have held those stocks. A rule that sells a falling stock early may be affected differently, because the stocks it would have sold before a collapse are also missing.

How do you avoid survivorship bias?

Test on the stocks that were in the index at each date (point-in-time membership), with price data for the companies that later left, and decide in advance what happens to a position when its stock is delisted.

What is a point-in-time universe?

The list of stocks a strategy could actually have chosen from on each date: for the S&P 500, the members on that day. A stock enters the test when it joins the index and leaves when it is removed.

Is survivorship bias the same as look-ahead bias?

They are related. Survivorship bias is the selection problem: the failures are missing. Choosing the historical universe from today’s index members also brings in look-ahead information: the test knows which companies would survive and grow large enough to join the index.

Do free data sources include delisted stocks?

Often not, or only partly. Many free price feeds drop a ticker once it stops trading, and some reuse old tickers for new companies, so a delisted stock’s history can be missing or attached to the wrong company.