Survivorship bias in backtests: delisted stocks and reused tickers

A common first stock backtest starts with today's list of index members and runs it back ten years. That universe only contains companies that survived, and grew enough to be in the index today. The ones that went bankrupt, were acquired or shrank out of the index are missing. A real investor ten years ago couldn't have known which companies those would be, so the backtest knows something they didn't.

The fix is well known: use point-in-time membership and keep delisted stocks. A less obvious problem is the ticker symbol itself. Symbols get reused, and many data sources map a symbol to whichever company holds it today. This page covers both, with code you can run on synthetic data.

All results on this page are hypothetical and come from simulated prices.

How big is the effect? A simulation

Here every stock draws its daily returns from the same distribution, with an expected return of exactly zero, so no stock is better than another. Any difference between the universes below is caused by how they were chosen.

import numpy as np
import pandas as pd

rng = np.random.default_rng(7)
n_days, n_stocks = 2520, 1000  # about ten years of trading days
daily = rng.normal(-0.0002, 0.02, (n_days, n_stocks))  # log returns; expected simple return is 0
prices = 10 * np.exp(np.cumsum(daily, axis=0))
returns = pd.DataFrame(np.expm1(daily))

# A stock is delisted after the first day it closes below $1.
crossed = np.cumsum(prices < 1, axis=0) > 0
listed = pd.DataFrame(np.vstack([np.zeros((1, n_stocks), bool), crossed[:-1]]) == 0)
survivors = listed.iloc[-1].to_numpy()
top_today = np.argsort(np.where(survivors, prices[-1], 0))[-100:]  # "today's index": the 100 largest survivors

def annual_return(daily_returns):
    return 100 * ((1 + daily_returns).prod() ** (252 / len(daily_returns)) - 1)

round(annual_return(returns.where(listed).mean(axis=1)), 1)    # -> 0.0
round(annual_return(returns.loc[:, survivors].mean(axis=1)), 1)  # -> 1.3  (hypothetical)
round(annual_return(returns[top_today].mean(axis=1)), 1)       # -> 18.9  (hypothetical)
int((~survivors).sum())                                          # -> 65

Three equal-weighted portfolios of the same kind of stock, as hypothetical annual returns in percent:

• Point-in-time (every stock while it was listed, delisted ones included): 0.0%, the right answer for stocks with no expected return.
• Survivors only (stocks still listed at the end): 1.3%, because the 65 stocks that fell below $1 are gone.
• Today's largest 100: 18.9%, because being large today is the result of having gone up.

The last one is the "today's S&P 500 list" backtest in miniature. Nothing about these stocks was predictable; the universe was picked with hindsight. The academic literature has measured the same effect in real data, for example Brown, Goetzmann, Ibbotson and Ross, "Survivorship Bias in Performance Studies" (Review of Financial Studies, 1992), and Elton, Gruber and Blake on mutual funds ("Survivor Bias and Mutual Fund Performance", Review of Financial Studies, 1996).

Point-in-time membership

Store index membership as intervals, one row per company and stint in the index: a start date and an end date (empty if it's still a member). A company is in your universe on day t only if t falls inside one of its intervals and it has a price on t.

membership = pd.DataFrame({
    "company_id": [101, 102, 103],
    "start": pd.to_datetime(["2010-01-04", "2012-06-01", "2019-04-02"]),
    "end": pd.to_datetime(["2017-09-01", None, None]),  # NaT: still a member
})

def members_on(day, membership):
    day = pd.Timestamp(day)
    m = membership
    current = (m["start"] <= day) & (m["end"].isna() | (day < m["end"]))
    return sorted(m.loc[current, "company_id"].tolist())

members_on("2015-01-02", membership), members_on("2020-01-02", membership)  # -> ([101, 102], [102, 103])

Use half-open intervals (start <= day < end) so a company that leaves on the day another joins is never counted twice. Index providers publish additions and removals; reconstructing the history from announcements is tedious but possible, and it's the part most free tutorials skip.

Delisted stocks: keep the last return

When a stock leaves the market, keep its returns through the last trading day. Delistings happen for very different reasons:

• Acquisitions: the last price usually reflects the deal, often at a premium. Dropping these removes gains.
• Failures (bankruptcy, falling below listing standards): the last trade can be far above what holders eventually received. Dropping these removes losses.

If your data has no return for the delisting itself, don't treat a missing value as zero. A common convention in academic work, following Shumway, "The Delisting Bias in CRSP Data" (Journal of Finance, 1997), is to assume a large negative return for performance-related delistings (around −30% for NYSE and AMEX stocks; Shumway and Warther, "The Delisting Bias in CRSP's Nasdaq Data and Its Implications for the Size Effect", Journal of Finance, 1999, estimated larger losses on Nasdaq). Applying the same penalty to acquisitions would be wrong in the other direction, so classify the reason first.

Reused tickers

A ticker is a label, and exchanges hand labels out again. Three examples involving US large caps:

If you join prices to membership by ticker alone, DOW in 2018 matches nothing, and a data source that maps STI to the current holder can attach another company's history to a bank. The fix is to key everything on a permanent identifier (a vendor's permanent security ID, the SEC's CIK for the issuer, or a CUSIP history) and treat the ticker as an attribute with dates:

ticker_map = pd.DataFrame({
    "ticker": ["XYZ", "XYZ"],
    "valid_from": pd.to_datetime(["2005-01-03", "2021-03-01"]),
    "valid_to": pd.to_datetime(["2019-12-06", None]),  # NaT: still in use
    "company_id": [201, 305],  # two unrelated companies, one symbol
})
rows = pd.DataFrame({"date": pd.to_datetime(["2018-05-01", "2020-06-01", "2022-05-02"]), "ticker": "XYZ"})

mapped = pd.merge_asof(rows.sort_values("date"), ticker_map.sort_values("valid_from"),
                       left_on="date", right_on="valid_from", by="ticker")
expired = mapped["valid_to"].notna() & (mapped["date"] >= mapped["valid_to"])
mapped["company_id"] = mapped["company_id"].where(~expired)
mapped["company_id"].tolist()  # -> [201.0, nan, 305.0]

The 2020 row maps to no company, which is correct: nobody held the symbol then. A plain merge on ticker would have given that row, and every row, two matches. Label the later company distinctly in your own tables (for example XYZ#2) so the two never get mixed up in a chart or a log.

Fundamentals have survivorship bias too

Financial-statement datasets are often built for the companies that exist today. Coverage for companies that later disappeared can be much thinner, and a missing balance sheet quietly removes the stock from any factor that needs it. Measure coverage separately for current and for delisted names before you rely on a factor result that depends on it.

Checklist

1. Universe: point-in-time membership, not today's list. Report how many names were members on a few sample dates and compare with the index's stated count.
2. Delistings: count them, and check that their last returns are in the data. Zero delisted names over ten years means the data is survivor-only.
3. Identifiers: join on a permanent ID. List every ticker that maps to more than one company over your sample.
4. Coverage: the share of member-days with a price, and with fundamentals if you use them, split by "still listed" and "delisted".
5. Spot checks: pick a few known events (a bankruptcy, an acquisition, a spin-off) and make sure each looks right in your data.

Checking code for this

Backtest Auditor (in development) flags code that builds its universe from a current constituent list, such as a table of today's index members read from the web. It reads the code, so it can't see whether your data files include delisted stocks; the checklist above covers that part. It will be available after release.

Related on this site: lookahead bias in pandas, the deflated Sharpe ratio in Python, and our help articles.