perf(spillover): sparse cKDTree branch for staggered nearest-treated distances

[igerber]

Summary

• The staggered cohort loop in _compute_nearest_treated_distance_staggered always built a dense (n_units, n_treated_by_onset) distance matrix per cohort. It now dispatches per cohort to the same cKDTree helper the static path already uses (_compute_nearest_treated_distance_sparse) — auto-activated when n_units > _CONLEY_SPARSE_N_THRESHOLD, built-in metrics only, with cutoff_km = _effective_d_bar (the outermost ring edge), mirroring the static call site exactly. Closes the Wave-B TODO row.
• Semantics-preserving by construction: within-cutoff distances are exact (the sparse helper recomputes the true great-circle/planar metric for in-range matches); beyond-cutoff units get inf, and every staggered d_it consumer — ring membership, S_it, the far-away-control check, and the event-study d_bar trigger — compares against thresholds ≤ that cutoff, so inf vs large-finite is indistinguishable downstream (same argument as the documented static-path behavior).
• New TestStaggeredSparseKDTreeBranch: helper-level equality (within-cutoff distances match dense at 1e-8; beyond-cutoff all inf; the d_bar trigger array identical incl. NaN pattern) and end-to-end fit identity (att/SE/per-ring effects at atol 1e-12) with the threshold monkeypatched to force the sparse arm.

Methodology references (required if estimator / math changes)

• Method name(s): Butts (2021) ring-exposure construction (distance-computation routing only; ring/exposure/trigger semantics unchanged)
• Paper / source link(s): REGISTRY.md § SpilloverDiD (ring membership + far-away contract; the static sparse-path convention this mirrors is already documented at the static call site)
• Any intentional deviations from the source (and why): None

Validation

• Tests added/updated: tests/test_spillover.py::TestStaggeredSparseKDTreeBranch (2 tests); full tests/test_spillover.py passes (321).
• Backtest / simulation / notebook evidence (if applicable): N/A

Security / privacy

• Confirm no secrets/PII in this PR: Yes

---

Generated with Claude Code

[github-actions]

Overall Assessment: ✅ Looks Good

Executive Summary

• No P0/P1 methodology or correctness findings.
• Affected method: SpilloverDiD staggered nearest-treated distance routing in diff_diff/spillover.py.
• The sparse branch preserves the registry contract: ring membership, S_it, far-away controls, and event-study trigger all compare only against d_bar, which is validated equal to the outer ring edge.
• This is an implementation choice for distance computation, not a methodology deviation from Butts ring-exposure construction. Butts supports local spillover exposure, distance bins/rings, and staggered settings. (arxiv.org)
• I could not run the tests locally because pytest is not installed in this environment.

Methodology

• Severity: P3 informational
• Impact: The PR affects SpilloverDiD ring-exposure distance routing only: _compute_nearest_treated_distance_staggered now uses the existing sparse helper when cutoff_km=self._effective_d_bar, n_units exceeds the sparse threshold, and the metric is built-in. The estimator’s exposure definition remains aligned with docs/methodology/REGISTRY.md: nearest currently-treated unit, time-varying staggered rings, Omega_0 = {D_it = 0 AND S_it = 0}, and far-away controls defined by d_it > d_bar.
• Concrete fix: None required. The inf substitution for beyond-cutoff distances is not user-visible under the current contract because all affected downstream checks are threshold comparisons at <= d_bar or > d_bar: diff_diff/spillover.py:L484-L486, diff_diff/spillover.py:L1073-L1130, diff_diff/spillover.py:L1891-L1908, diff_diff/spillover.py:L2604-L2624.

Code Quality

• No findings.

Performance

• No findings. The change removes the documented TODO.md performance backlog item for staggered sparse nearest-treated distances and scopes the sparse path to built-in metrics only: diff_diff/spillover.py:L449-L459.

Maintainability

• No findings. The staggered path reuses the existing static sparse helper rather than adding a second implementation: diff_diff/spillover.py:L234-L332.

Tech Debt

• No findings. The corresponding backlog row was removed from TODO.md after implementation.

Security

• No findings. No secrets, external execution, or unsafe input handling changes observed.

Documentation/Tests

• Severity: P3

• Impact: Minor changelog typo: cutoff_km = has a stray space/backtick gap, which slightly weakens the release note clarity.

• Concrete fix: Change cutoff_km = the outermost ring edge to ``cutoff_km = <outermost ring edge>`` or ``cutoff_km set to the outermost ring edgeinCHANGELOG.md:L437-L446`.

• Severity: P3 informational

• Impact: New helper-level and end-to-end tests cover sparse vs dense parity for staggered distances and fit outputs: tests/test_spillover.py:L8906-L8998.

• Concrete fix: None required. I was unable to execute them here because pytest / python -m pytest is unavailable.