What are the three legs of point-in-time discipline?
Universe leg: the tradable set at time t must contain exactly the securities listed at t — including future delistees. Backtests on today's index constituents inject survivorship bias: bankruptcies vanish, and the surviving winners flatter returns by 1–3% annually in US equities. The fix is a survivorship-free membership table (entry/exit dates per security) joined as-of t.
Price leg: prices must be adjusted exactly as a trader at t would see them — splits/dividends applied with as-traded factors, no future-adjusted closes leaking corporate actions early. Intraday timing matters too: decisions stamped at close must trade at next open (or with explicit slippage), never at the same close.
Feature leg: every feature value must be knowable at t. Fundamentals lag by reporting delay plus a safety buffer (earnings released in March describe December; using December values in January is look-ahead). Macro series must use vintages (ALFRED), not revised histories — GDP and payrolls revise for years.
What is the audit checklist?
One: universe reconciliation — count securities per date vs an external constituent history; confirm delisted names appear with terminal returns (typically −100% or the delisting return, not NaN silently forward-filled). Two: corporate-action replay — recompute adjusted closes from raw closes and the action calendar; any mismatch is a leak or a bug, both fatal.
Three: feature timestamp proof — for each feature, show max(feature_asof) ≤ decision_time across the panel (the code below automates this). Four: macro vintage check — join at least one series (e.g., payrolls) from ALFRED vintages and confirm the backtest moves when vintages replace revised histories; if it does not move at all, the join is probably broken.
Five: delisting-return sensitivity — rerun with delisting returns set to 0% vs −100% vs −30%; if the conclusion flips, the strategy's edge lives in the delisting assumption, and that assumption needs its own evidence.
What is the reproducible code?
Two checks: a feature-timestamp proof over a panel (fails loudly on any future-dated feature) and a survivorship comparator (today-constituents vs full-universe cumulative return gap).
python# pit_audit.py # Point-in-time audit checks: feature timestamps + survivorship gap. # Run: python pit_audit.py import numpy as np import pandas as pd def assert_point_in_time(features: pd.DataFrame, decisions: pd.Series) -> None: """Every feature's as-of time must be <= its decision time. features: MultiIndex (date, asset) with column 'asof' (Timestamp). decisions: Series indexed (date, asset) with decision timestamps. Raises on first violation with a sample of offenders. """ merged = features[["asof"]].join(decisions.rename("decision"), how="inner") bad = merged[merged["asof"] > merged["decision"]] if len(bad): print(bad.head(10).to_string()) raise AssertionError( f"PIT VIOLATION: {len(bad)} feature rows post-date their decisions." ) print(f"PIT OK: {len(merged)} rows, all features knowable at decision time.") def survivorship_gap( full_universe_rets: pd.Series, survivor_rets: pd.Series ) -> float: """Annualized return gap: survivors-only minus full universe.""" gap = (1 + survivor_rets).prod() ** (252 / len(survivor_rets)) - ( (1 + full_universe_rets).prod() ** (252 / len(full_universe_rets)) ) return float(gap) if __name__ == "__main__": rng = np.random.default_rng(9) dates = pd.date_range("2020-01-01", periods=252 * 3, freq="B") assets = ["AAA", "BBB", "CCC", "DEAD"] idx = pd.MultiIndex.from_product([dates, assets], names=["date", "asset"]) # Clean features: asof == decision date. Flip one row to demo the tripwire: asof = pd.Series(idx.get_level_values("date"), index=idx, name="asof") decisions = pd.Series(idx.get_level_values("date"), index=idx) try: assert_point_in_time(pd.DataFrame({"asof": asof}), decisions) except AssertionError as e: print(e) # Survivorship demo: survivors +8%/yr, full universe incl. -100% delisting. surv = pd.Series(rng.normal(0.08 / 252, 0.01, len(dates)), index=dates) full = surv.copy() full.iloc[-1] += -1.0 / len(dates) * 40 # delisting drag proxy print(f"survivorship gap: {survivorship_gap(full, surv):+.2%} /yr")
What mistakes does this catch?
Forward-filled delisting gaps (NaN → last price) which quietly grant immortality to bankrupt names. The audit forces explicit delisting returns — and the sensitivity rerun prices the assumption.
Same-day fundamentals: joining quarterly EPS by fiscal quarter end instead of report date plus buffer. The timestamp proof catches every instance mechanically, including vendor fields whose 'date' column is the period end, not the release.
Revised macro histories in training features with live decisions on first releases. The ALFRED join makes the mismatch visible because first-release and latest-vintage values differ on exactly the dates that matter.
What are the honest limits?
PIT-clean data costs money (CRSP/Compustat PIT, survivorship-free vendor feeds) or serious engineering (open-source constituent histories are patchy before ~2000). Name the coverage honestly: a PIT-clean US post-2000 backtest beats a contaminated 1871-present one for deployment decisions.
The audit verifies timestamps, not economics: correctly-stamped but misinterpreted data (e.g., restated financials used at original release values without restatement handling) passes the timestamp proof and still misleads. Restatement policy needs its own note.
Delisting-return assumptions remain assumptions. −100% for bankruptcies, −30% for acquisitions-at-unknown-terms, CRSP delisting returns where available — document the mapping per exit type or the sensitivity analysis is theatre.