Skip to content

feat(rust): one-way HC2 robust vcov kernel#651

Open
igerber wants to merge 1 commit into
mainfrom
feat/rust-hc2-vcov
Open

feat(rust): one-way HC2 robust vcov kernel#651
igerber wants to merge 1 commit into
mainfrom
feat/rust-hc2-vcov

Conversation

@igerber

@igerber igerber commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Summary

  • The Rust vcov path supported only HC1/CR1 — HC2 fell through to NumPy (the TODO row's large-n gap). Adds compute_robust_vcov_hc2 mirroring the NumPy hc2 branch exactly: hat diagonals off the same bread, u²/max(1−h, 1e-10) leverage meat, no n/(n−k) factor — matching NumPy at ~1e-15 across a seed grid, per the Python-canonical convention.
  • Guard semantics preserved Python-side: the kernel signals near-singular hat diagonals (sentinel error) and the dispatcher reproduces the documented warn-and-fall-back-to-HC1 exactly as the NumPy branch does; the HC1 dispatch's numerically-unstable fallback is mirrored too (warn + NumPy HC2, so guard decisions stay in one place). Both plumbing paths are locked by monkeypatched-sentinel tests; an exact-h=1 design locks clamp-path parity (that case legitimately does not trip the >1+1e-6 guard in either backend).
  • Malformed input (residual/row length mismatch) fails loudly in the kernel — no silent zip truncation (local-review P1, regression-tested).
  • Mixed-version safe: the symbol imports independently (same pattern as demean_map / the batched ridge kernel) — a stale extension degrades HC2 to NumPy without disabling older Rust accelerations. return_dof / weighted / clustered / CR2-BM requests stay on NumPy; the TODO row is narrowed to the CR2-BM remainder.

Methodology references (required if estimator / math changes)

  • Method name(s): HC2 leverage-corrected heteroskedasticity-robust variance (MacKinnon-White; sandwich::vcovHC type="HC2" convention already documented for the NumPy branch) — backend port only, no formula change
  • Paper / source link(s): REGISTRY.md linalg/vcov notes (HC2 contract); Python implementation is canonical per the repo convention
  • Any intentional deviations from the source (and why): None — ~1e-15 agreement with the canonical NumPy branch; identical guard/fallback behavior.

Validation

  • Tests added/updated: tests/test_rust_backend.py::TestRustHC2Vcov (7 tests: seed-grid parity, kernel shape/symmetry/PD, exact-h=1 clamp parity, sentinel→HC1-fallback plumbing, instability→NumPy-HC2 fallback plumbing, length-mismatch rejection, DOF/weights dispatch-decline equivalence).
  • Evidence: tests/test_rust_backend.py + tests/test_linalg.py pass (258); cargo check/fmt/clippy clean on the new code.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Generated with Claude Code

@igerber igerber force-pushed the feat/rust-hc2-vcov branch 3 times, most recently from ae6098f to 66bd128 Compare July 8, 2026 00:16
The Rust vcov path supported only HC1/CR1; HC2 fell through to NumPy
(noticeable at large n). Adds compute_robust_vcov_hc2 mirroring the
NumPy hc2 branch exactly: hat diagonals off the same bread,
u^2/max(1-h, 1e-10) leverage meat, no n/(n-k) factor — matches NumPy at
~1e-15 across a seed grid. The near-singular hat-diagonal guard stays
Python-side: the kernel returns a sentinel error and the documented
warn-and-fall-back-to-HC1 fires in the dispatcher, identical to the
NumPy branch (locked by a monkeypatched-sentinel test + an exact-h=1
clamp-parity test). The symbol imports independently (mixed-version
safe: a stale extension degrades HC2 to NumPy without disabling older
Rust accelerations). return_dof / weighted / clustered / CR2-BM
requests stay on NumPy; the TODO row is narrowed to CR2-BM.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@igerber igerber force-pushed the feat/rust-hc2-vcov branch from 66bd128 to fef4197 Compare July 8, 2026 00:17
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: fef419713ff9ef554fe00f455b6cb05dd6b9e21c


Overall Assessment

✅ Looks good — no unmitigated P0/P1 findings.

Executive Summary

  • Affected method: one-way, unweighted HC2 robust vcov backend path only.
  • The Rust kernel matches the documented HC2 formula: hat diagonals, u^2 / max(1-h, 1e-10), and no HC1 n/(n-k) factor.
  • Weighted, clustered, return_dof, and HC2-BM paths remain on the existing NumPy / clubSandwich-derived implementations.
  • Edge-case handling is covered for sentinel fallback, numerical-instability fallback, exact h=1 clamp parity, and residual length mismatch.
  • I could not execute tests locally because this environment lacks numpy.

Methodology

  • Severity: P3 informational
    Impact: No methodology mismatch found. The new Rust path is scoped to analytical-sandwich HC2, which REGISTRY documents as applicable to hat-matrix-bearing designs, and it mirrors the canonical NumPy HC2 branch. See docs/methodology/REGISTRY.md:L410-L418, diff_diff/linalg.py:L1600-L1603, diff_diff/linalg.py:L2832-L2869, and rust/src/linalg.rs:L239-L305.
    Concrete fix: None.

Code Quality

  • Severity: P3 informational
    Impact: The dispatcher preserves existing fallback semantics: hat diagonal sentinel falls back to HC1, while Rust numerical instability falls back to NumPy HC2. See diff_diff/linalg.py:L1716-L1776. The Rust kernel also rejects residual/design length mismatch instead of relying on broadcast failure. See rust/src/linalg.rs:L267-L273.
    Concrete fix: None.

Performance

  • Severity: P3 informational
    Impact: The PR removes the documented Rust HC2 performance gap for the supported unweighted one-way path while keeping CR2-BM in TODO. See TODO.md:L50-L51 and CHANGELOG.md:L436-L445.
    Concrete fix: None.

Maintainability

  • Severity: P3 informational
    Impact: Independent symbol import follows the existing mixed-version-safe pattern, so a stale extension can fall back to NumPy HC2 without disabling older Rust accelerations. See diff_diff/_backend.py:L85-L94.
    Concrete fix: None.

Tech Debt

  • Severity: P3 informational
    Impact: Deferred work is properly narrowed from Rust HC2+CR2-BM to Rust CR2-BM only. This is tracked in TODO.md:L50-L51, so it is not a blocker.
    Concrete fix: None.

Security

  • Severity: P3 informational
    Impact: No secrets, unsafe input handling, or security-sensitive changes observed in the diff.
    Concrete fix: None.

Documentation/Tests

  • Severity: P3 informational
    Impact: Tests cover Rust/NumPy parity, direct kernel shape/symmetry, exact h=1, sentinel fallback, numerical-instability fallback, length mismatch, and dispatch decline for DOF/weights. See tests/test_rust_backend.py:L3512-L3624.
    Concrete fix: None required; CI should run the added Rust backend tests plus existing HC2 linalg tests.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jul 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant