Meta Description: Build a pairs trading strategy with cointegration testing (Engle-Granger, ADF), hedge-ratio estimation, z-score signals, and a walk-forward backtest in Python. Includes half-life exits and the failure modes that kill most pairs.

Pairs trading is a mean-reversion strategy on the spread between two cointegrated assets: you go long the underperformer and short the outperformer when the spread deviates, and exit when it reverts. Cointegration — not correlation — is the statistical foundation, because correlation breaks exactly when you need it while a cointegrated spread is stationary by construction.

This article is part of the quantitative finance knowledge cluster. For the single-asset OU formulation see OU mean reversion; for validation see walk-forward optimisation and purged CV.

1. Cointegration, not correlation, is the entry criterion

Two price series $P_t^A, P_t^B$ are cointegrated if some linear combination $S_t = P_t^A - \beta P_t^B$ is stationary even though each leg is non-stationary (unit root). Correlation measures co-movement over a window and drifts; cointegration is a long-run equilibrium property testable with the Engle-Granger two-step procedure: regress $P^A$ on $P^B$, then run an Augmented Dickey-Fuller (ADF) test on the residuals. Reject the unit-root null (p < 0.05) and the spread is tradable mean-reversion fuel; fail to reject and you have correlated drift, not a pair.

import numpy as np
import pandas as pd
import statsmodels.api as sm
from statsmodels.tsa.stattools import adfuller

def engle_granger(price_a: pd.Series, price_b: pd.Series) -> dict:
    """Step 1: OLS hedge ratio. Step 2: ADF on residuals."""
    X = sm.add_constant(price_b.values)
    ols = sm.OLS(price_a.values, X).fit()
    beta = float(ols.params[1])
    spread = price_a.values - beta * price_b.values
    adf_stat, p_value, *_ = adfuller(spread, autolag="AIC")[:2]
    return {"beta": beta, "spread": spread,
            "adf_stat": float(adf_stat), "p_value": float(p_value),
            "cointegrated": bool(p_value < 0.05)}

A common mistake is testing cointegration on the full history and trading on the same window. Split first: estimate $\beta$ and the ADF on a formation period (e.g., 12 months), then freeze $\beta$ and trade the following 6 months. Re-estimate on a roll, never on the trading window.

2. The hedge ratio converts two prices into one stationary spread

The hedge ratio $\beta$ is the OLS slope of $P^A$ on $P^B$ over the formation window. The spread $S_t = P_t^A - \beta P_t^B$ is dollar-neutral by construction at formation: a $1 move in the spread corresponds to the hedged portfolio, not to either leg alone. Signals come from the spread z-score $z_t = (S_t - \mu_S)/\sigma_S$ with $\mu_S, \sigma_S$ estimated on formation data only — never on the trading window, or the thresholds already know the future.

Enter when $|z_t| > 2$ (long the cheap leg, short the rich leg), exit when $|z_t| < 0.5$. The asymmetry between a wide entry and a tight exit is deliberate: it harvests the full reversion while cutting time spent in flat noise. Position size should be inversely proportional to $\sigma_S$ so each pair contributes comparable risk.

3. Half-life sets the holding horizon and the stop-loss

The spread's half-life $\tau_{1/2} = \ln 2 / \theta$, with $\theta$ from the AR(1) regression $S_{t+1} - S_t = \alpha + \beta S_t + \varepsilon$, tells you how long reversion takes. If $\tau_{1/2}$ exceeds ~20 trading days the pair is too slow for a statistical-arbitrage book and behaves like a buy-and-hold tilt. Cap holding at $3 \times \tau_{1/2}$: a spread that has not reverted in three half-lives has likely decoupled (takeover, regime break, constituent change), and hope is not a risk model.

def spread_half_life(spread: np.ndarray) -> float:
    """Half-life of mean reversion via AR(1) OLS. Returns inf if no reversion."""
    lag = spread[:-1]
    delta = np.diff(spread)
    X = sm.add_constant(lag)
    beta = sm.OLS(delta, X).fit().params[1]
    theta = -beta
    return float(np.log(2) / theta) if theta > 1e-8 else float("inf")

def pair_signals(spread: np.ndarray, mu: float, sigma: float,
                 z_entry: float = 2.0, z_exit: float = 0.5) -> np.ndarray:
    """Integer positions {-1, 0, 1} on the spread. +1 = long spread."""
    z = (spread - mu) / sigma if sigma > 1e-12 else np.zeros_like(spread)
    pos, holding = np.zeros(len(spread), dtype=int), 0
    for i in range(len(spread)):
        if holding == 0:
            if z[i] < -z_entry: pos[i], holding = 1, 1
            elif z[i] > z_entry: pos[i], holding = -1, -1
        else:
            if abs(z[i]) < z_exit: pos[i], holding = 0, 0
            else: pos[i] = holding
    return pos

4. Backtest the hedged portfolio, not the spread line

A spread chart that reverts is not a backtest. The backtest must simulate both legs with borrow costs on the short, realistic fills (next-open after the signal bar, never same-close), and a portfolio constraint (e.g., max 10 concurrent pairs, equal risk per pair). Report net-of-cost Sharpe with a stationary block bootstrap confidence interval, and run the whole pipeline walk-forward: formation → frozen $\beta$ → 6-month trade → roll.

The three failure modes that kill most pairs: (1) decoupling — cointegration estimated on history breaks (test rolling 60-day ADF p-values; abandon the pair after 3 consecutive months above 0.10); (2) crowding — famous pairs (KO/PEP class) decay post-publication, consistent with McLean & Pontiff (2016); (3) cost blindness — daily rebalancing of a 2%-edge spread with 2 bps per leg per day erases the edge, so gate every pair on net Sharpe, never gross.

5. Worked example: formation, signals, and attribution

Take two large-cap consumer staples names over 2022–2024 daily closes (illustrative numbers). Formation year 2022: OLS of $P^A$ on $P^B$ gives $\beta = 0.82$ with ADF p-value 0.021 — cointegrated, trade approved. Formation spread volatility \sigma_S = \1.40$around$\mu_S \approx $0$. Half-life from the AR(1) fit: 9 trading days — fast enough for a stat-arb book, with a 27-day maximum hold.

Trading H1 2023 frozen at $\beta = 0.82$: the spread hits $z = -2.3$ in March (A sells off on earnings while B holds flat) → long spread (long A, short $0.82 \times$ B notional). It reverts to $z = -0.2$ eleven days later for +1.9% on deployed spread notional, +1.6% after 2 bps per leg plus borrow. Two more round-trips in H1 print +1.1% and −0.4% (the loser: a holding that hit the 27-day cap during a sector rotation — exactly the decoupling case). H1 net: +2.3% on the pair sleeve with 4.1% volatility, gross Sharpe ≈ 1.1, net ≈ 0.8.

Attribution matters more than the total: of the +2.3%, +2.9% came from reversion captures and −0.6% from the capped decoupling — the stop did its job. Rolling 60-day ADF p-values stayed below 0.08 throughout H1; had they printed above 0.10 for three straight months, the protocol kills the pair regardless of P&L. Re-formation in July 2023 re-estimated $\beta = 0.79$ (drift is normal; jumps are not — a $\beta$ jump beyond ±20% triggers a fundamentals review before re-approval).

6. From one pair to a book

A single pair is a concentrated coin-flip on one equilibrium. Production pairs books hold 20–100 pairs across sectors with three overlays: a market-neutrality check (net beta ≈ 0 daily), a concentration cap (no pair > 5% of risk), and a kill switch (portfolio-level stop at −3 ATR-equivalents of daily P&L). Validate the book — not just each pair — with CPCV and the deflated Sharpe, because selecting the best 20 of 500 tested pairs is exactly the multiple-testing problem those tools exist for.