Skip to content

test(se-audit): G2 hetero + cluster machine-precision fixest locks on unbalanced heteroskedastic DGP#644

Merged
igerber merged 1 commit into
mainfrom
test/fixest-golden-hetero-dgp
Jul 8, 2026
Merged

test(se-audit): G2 hetero + cluster machine-precision fixest locks on unbalanced heteroskedastic DGP#644
igerber merged 1 commit into
mainfrom
test/fixest-golden-hetero-dgp

Conversation

@igerber

@igerber igerber commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Summary

  • Completes the deferred SE-audit G2(a) item: fixest_did_twfe_golden.json gains two appended scenarios (heteroskedastic — error sd varies by arm and period — and unbalanced, ~15% rows dropped), so fixest's hetero (HC1) no longer collapses to iid. The original balanced scenarios' RNG draws precede the new blocks and reproduce value-identically on regeneration (verified).
  • DiD plain-OLS path — machine-precision locks for iid, hetero, AND cluster CR1 (observed ≤1e-15; asserted atol=1e-10): the CR1 cluster SE turns out to match fixest exactly on plain OLS (balanced and unbalanced), so the former ~0.5% DiD cluster band-pin is tightened to a machine lock. The documented ~0.25% fixest-CR1 DOF-convention deviation is an absorbed-FE (within-transform) phenomenon only — confirmed by explicit fixest ssc() probing (fixef.K=full/nested + adj reproduce the gap only under absorbed FE).
  • TWFE — the cluster SE stays band-pinned (rel 0.005), re-scoped to the documented non-nested-FE ssc deviation (fixest counts the non-nested time FE in the (n−1)/(n−k) denominator; ~0.3% on the unbalanced scenario; exact-parity follow-up already tracked under "Needs external reference"). TWFE hetero has no public unclustered Python surface (documented auto-cluster-at-unit convention), so scenario 4 locks iid — which pins the SE-audit D4 full-K rescale on an unbalanced panel for the first time. The fixest hetero block is stored for reference.
  • Required golden blocks now assert (not skip) when missing — a regeneration that dropped them fails loudly instead of silently shedding coverage. TODO row 69 updated (sub-item (a) recorded DONE; (b)-(d) + C6-C8 remain).

Methodology references (required if estimator / math changes)

  • Method name(s): 2x2 DiD (pooled OLS) and TWFE within-transform HC1/CR1 small-sample conventions vs fixest::feols ssc() (no estimator code changed — golden fixture, tests, TODO/CHANGELOG only)
  • Paper / source link(s): REGISTRY.md within-transform notes (D4 full-K rescale; non-nested-FE CR1 deviation); fixest 0.14.2 / R 4.5.2 as the reference.
  • Any intentional deviations from the source (and why): the TWFE non-nested-FE CR1 band-pin reflects the already-documented deviation (REGISTRY + Deferred TODO row); everything newly lockable is now machine-locked.

Validation

  • Tests added/updated: tests/test_fixest_did_twfe_parity.py (7 tests: 2 existing iid locks unchanged, DiD cluster tightened to machine precision across both scenarios, TWFE cluster ATT exact + SE band, new hetero class with a discriminating hetero≠iid assertion).
  • Evidence: regeneration reproduces scenarios 1-2 value-identically; live Python-vs-fixest diffs before pinning: DiD iid/hetero/cluster ≤1e-15 rel, TWFE iid 0.0, TWFE cluster 3.0e-3 (inside the documented band).

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Generated with Claude Code

… unbalanced heteroskedastic DGP

Adds two appended scenarios to the fixest golden (error sd varying by
arm/period, ~15% rows dropped; the original balanced scenarios' RNG
draws precede them and reproduce value-identically). On the plain-OLS
DiD path, hetero (HC1) no longer collapses to iid and is locked at
machine precision — and the CR1 cluster SE matches fixest EXACTLY on
plain OLS (balanced + unbalanced), so the former DiD cluster band-pin
is tightened to a machine lock: the documented ~0.25% fixest-CR1
DOF-convention deviation is absorbed-FE-only. The TWFE cluster band is
retained and re-scoped to that documented non-nested-FE ssc deviation;
TWFE hetero has no public unclustered surface (auto-cluster-at-unit),
so its scenario locks iid — pinning the D4 full-K rescale on an
UNBALANCED panel for the first time. Required golden blocks assert
(not skip) when missing. Closes SE-audit G2(a) in the TODO.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown

Overall Assessment: ✅ Looks Good

Executive Summary

  • No unmitigated P0/P1 findings.
  • Affected methods: 2x2 DifferenceInDifferences OLS/HC1/CR1 parity and TwoWayFixedEffects classical/CR1 parity.
  • The TWFE clustered-SE mismatch remains documented in REGISTRY.md and tracked in TODO.md, so it is not a blocker.
  • The golden JSON is internally consistent: all added data vectors match n_obs.
  • I could not run tests because pytest is not installed in this review environment.

Methodology

  • Severity: P3 informational
    Impact: tests/test_fixest_did_twfe_parity.py:L97-L115 retains a relative band for TWFE clustered SEs rather than machine precision. This matches the documented absorbed-FE/non-nested-FE CR1 deviation in docs/methodology/REGISTRY.md:L342-L358 and the tracked TODO entry at TODO.md:L65.
    Concrete fix: None required.

Code Quality
No findings.

Performance
No findings.

Maintainability
No findings.

Tech Debt
No findings. The remaining SE-audit items are explicitly tracked in TODO.md:L65.

Security
No findings. The diff only changes tests, benchmark fixtures/scripts, and docs/changelog material.

Documentation/Tests
No PR findings. The added tests assert required golden blocks loudly (tests/test_fixest_did_twfe_parity.py:L87-L115, L132-L173) and add discriminating heteroskedastic coverage for DiD HC1. Static checks passed for JSON shape and git diff --check; test execution was not possible here because pytest was unavailable.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Jul 7, 2026
@igerber igerber merged commit d332aab into main Jul 8, 2026
25 of 26 checks passed
@igerber igerber deleted the test/fixest-golden-hetero-dgp branch July 8, 2026 02:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant