Meta Description: Build a pairs trading strategy with cointegration testing (Engle-Granger, ADF), hedge-ratio estimation, z-score signals, and a walk-forward backtest in Python. Includes half-life exits and the failure modes that kill most pairs.

Pairs trading is a mean-reversion strategy on the spread between two cointegrated assets: you go long the underperformer and short the outperformer when the spread deviates, and exit when it reverts. Cointegration not correlation is the statistical foundation, because correlation breaks exactly when you need it while a cointegrated spread is stationary by construction.

This article is part of the quantitative finance knowledge cluster. For the single-asset OU formulation see OU mean reversion; for validation see walk-forward optimisation and purged CV.

1. Cointegration, not correlation, is the entry criterion

Two price series $P_t^A, P_t^B$ are cointegrated if some linear combination $S_t = P_t^A - \beta P_t^B$ is stationary even though each leg is non-stationary (unit root). Correlation measures co-movement over a window and drifts; cointegration is a long-run equilibrium property testable with the Engle-Granger two-step procedure: regress $P^A$ on $P^B$, then run an Augmented Dickey-Fuller (ADF) test on the residuals. Reject the unit-root null (p < 0.05) and the spread is tradable mean-reversion fuel; fail to reject and you have correlated drift, not a pair.

import numpy as np
import pandas as pd
import statsmodels.api as sm
from statsmodels.tsa.stattools import adfuller

def engle_granger(price_a: pd.Series, price_b: pd.Series) -> dict:
    """Step 1: OLS hedge ratio. Step 2: ADF on residuals."""
    X = sm.add_constant(price_b.values)
    ols = sm.OLS(price_a.values, X).fit()
    beta = float(ols.params[1])
    spread = price_a.values - beta * price_b.values
    adf_stat, p_value, *_ = adfuller(spread, autolag="AIC")[:2]
    return {"beta": beta, "spread": spread,
            "adf_stat": float(adf_stat), "p_value": float(p_value),
            "cointegrated": bool(p_value < 0.05)}

A common mistake is testing cointegration on the full history and trading on the same window. Split first: estimate $\beta$ and the ADF on a formation period (e.g., 12 months), then freeze $\beta$ and trade the following 6 months. Re-estimate on a roll, never on the trading window.

2. The hedge ratio converts two prices into one stationary spread

The hedge ratio $\beta$ is the OLS slope of $P^A$ on $P^B$ over the formation window. The spread $S_t = P_t^A - \beta P_t^B$ is dollar-neutral by construction at formation: a $1 move in the spread corresponds to the hedged portfolio, not to either leg alone. Signals come from the spread z-score $z_t = (S_t - \mu_S)/\sigma_S$ with $\mu_S, \sigma_S$ estimated on formation data only never on the trading window, or the thresholds already know the future.

Enter when $|z_t| > 2$ (long the cheap leg, short the rich leg), exit when $|z_t| < 0.5$. The asymmetry between a wide entry and a tight exit is deliberate: it harvests the full reversion while cutting time spent in flat noise. Position size should be inversely proportional to $\sigma_S$ so each pair contributes comparable risk.

3. Half-life sets the holding horizon and the stop-loss

The spread's half-life $\tau_{1/2} = \ln 2 / \theta$, with $\theta$ from the AR(1) regression $S_{t+1} - S_t = \alpha + \beta S_t + \varepsilon$, tells you how long reversion takes. If $\tau_{1/2}$ exceeds ~20 trading days the pair is too slow for a statistical-arbitrage book and behaves like a buy-and-hold tilt. Cap holding at $3 \times \tau_{1/2}$: a spread that has not reverted in three half-lives has likely decoupled (takeover, regime break, constituent change), and hope is not a risk model.

def spread_half_life(spread: np.ndarray) -> float:
    """Half-life of mean reversion via AR(1) OLS. Returns inf if no reversion."""
    lag = spread[:-1]
    delta = np.diff(spread)
    X = sm.add_constant(lag)
    beta = sm.OLS(delta, X).fit().params[1]
    theta = -beta
    return float(np.log(2) / theta) if theta > 1e-8 else float("inf")

def pair_signals(spread: np.ndarray, mu: float, sigma: float,
                 z_entry: float = 2.0, z_exit: float = 0.5) -> np.ndarray:
    """Integer positions {-1, 0, 1} on the spread. +1 = long spread."""
    z = (spread - mu) / sigma if sigma > 1e-12 else np.zeros_like(spread)
    pos, holding = np.zeros(len(spread), dtype=int), 0
    for i in range(len(spread)):
        if holding == 0:
            if z[i] < -z_entry: pos[i], holding = 1, 1
            elif z[i] > z_entry: pos[i], holding = -1, -1
        else:
            if abs(z[i]) < z_exit: pos[i], holding = 0, 0
            else: pos[i] = holding
    return pos

4. Backtest the hedged portfolio, not the spread line

A spread chart that reverts is not a backtest. The backtest must simulate both legs with borrow costs on the short, realistic fills (next-open after the signal bar, never same-close), and a portfolio constraint (e.g., max 10 concurrent pairs, equal risk per pair). Report net-of-cost Sharpe with a stationary block bootstrap confidence interval, and run the whole pipeline walk-forward: formation → frozen $\beta$ → 6-month trade → roll.

The three failure modes that kill most pairs: (1) decoupling cointegration estimated on history breaks (test rolling 60-day ADF p-values; abandon the pair after 3 consecutive months above 0.10); (2) crowding famous pairs (KO/PEP class) decay post-publication, consistent with McLean & Pontiff (2016); (3) cost blindness daily rebalancing of a 2%-edge spread with 2 bps per leg per day erases the edge, so gate every pair on net Sharpe, never gross.

5. Worked example: formation, signals, and attribution

Take two large-cap consumer staples names over 2022–2024 daily closes (illustrative numbers). Formation year 2022: OLS of $P^A$ on $P^B$ gives $\beta = 0.82$ with ADF p-value 0.021 cointegrated, trade approved. Formation spread volatility \sigma_S = \1.40$around$\mu_S \approx $0$. Half-life from the AR(1) fit: 9 trading days fast enough for a stat-arb book, with a 27-day maximum hold.

Trading H1 2023 frozen at $\beta = 0.82$: the spread hits $z = -2.3$ in March (A sells off on earnings while B holds flat) → long spread (long A, short $0.82 \times$ B notional). It reverts to $z = -0.2$ eleven days later for +1.9% on deployed spread notional, +1.6% after 2 bps per leg plus borrow. Two more round-trips in H1 print +1.1% and −0.4% (the loser: a holding that hit the 27-day cap during a sector rotation exactly the decoupling case). H1 net: +2.3% on the pair sleeve with 4.1% volatility, gross Sharpe ≈ 1.1, net ≈ 0.8.

Attribution matters more than the total: of the +2.3%, +2.9% came from reversion captures and −0.6% from the capped decoupling the stop did its job. Rolling 60-day ADF p-values stayed below 0.08 throughout H1; had they printed above 0.10 for three straight months, the protocol kills the pair regardless of P&L. Re-formation in July 2023 re-estimated $\beta = 0.79$ (drift is normal; jumps are not a $\beta$ jump beyond ±20% triggers a fundamentals review before re-approval).

6. From one pair to a book

A single pair is a concentrated coin-flip on one equilibrium. Production pairs books hold 20–100 pairs across sectors with three overlays: a market-neutrality check (net beta ≈ 0 daily), a concentration cap (no pair > 5% of risk), and a kill switch (portfolio-level stop at −3 ATR-equivalents of daily P&L). Validate the book not just each pair with CPCV and the deflated Sharpe, because selecting the best 20 of 500 tested pairs is exactly the multiple-testing problem those tools exist for.