Converters: SACC→2pt-FITS (byte-compared) + SACC↔OneCovariance glue

[cailmdaley]

Closes #246. PR 3 of the PRD #241 series (row 3 of §7); stacked on #245 (sacc_io).

What

Adds the two converters that let SACC become the standard data-product format while the existing inference chain runs untouched behind it (PRD #241 §2):

• sp_validation.twopoint_convert — sacc_to_twopoint_fits(s, path, *, rho_stats_hdu=None, tau_stats_hdu=None, n_bins=1): converts an analysis SACC into the CosmoSIS 2pt-FITS that 2pt_like (and the ρ/τ 2pt_like_xi_sys fork) reads, reproducing today's cosmo_inference/scripts/cosmosis_fitting.py assembly HDU-for-HDU.
• sp_validation.one_covariance_io — the two pieces of SACC↔OneCovariance glue: write_nz / nz_table / nz_config_stanza / read_nz (SACC source_i n(z) → OneCovariance's combined n(z) text file + [redshift] config stanza), and covariance_blocks (OneCovariance's flat covariance_list_*.dat output → dense covariance blocks paired with SACC selectors for sacc_io.assemble_covariance, reusing statistics.cov_from_one_covariance).

Purely additive: no existing file is modified.

Why

The converter is the bridge in the PRD's "converter first, native SACC likelihood later" plan: it lets PR 4 migrate the per-statistic writers to SACC without touching CosmoSIS, and the byte-compare against a current product is the validation the PRD calls for. The OneCovariance glue replaces the hand-rolled n(z)→config and covariance-reshaping we do today.

How it's built

• Type-major permutation. SACC stores each pair as pair-major [ξ+; ξ−]; the 2pt-FITS is type-major (all ξ+, then all ξ−). The data vector and its covariance are permuted via s.indices.
• Single-bin v1, fail-fast. The converter emits the single-bin 2pt-FITS today's pipeline consumes and raises loudly on n_bins != 1 or any ξ tracer pair beyond (source_0, source_0) — no silent truncation. Tomographic emission lands with the tomographic round.
• The τ covariance is one contiguous [τ_0+; τ_2+] block, not block-diagonal — covdat_to_fits truncates the 3-statistic CosmoCov τ covariance to 2 blocks and keeps the τ_0↔τ_2 cross-correlation. Pinned by a dedicated test.
• ρ/τ uses the sidecar FITS, by design. The 2pt-FITS RHO_STATS table carries per-mode varrho_* variances the SACC layout does not store, so the converter copies RHO_STATS verbatim from the sidecar (never fabricates variances), exactly as rho_to_fits does. This is the settled contract, not a stopgap: the PSF likelihood (cosmo_inference/scripts/xi_sys_psf.py:33-49) reads only the ρ_k± values — never the variances — so there is no consumer that would want the variances lifted into SACC. ξ±, Cℓ, n(z) and the covariance are fully rebuilt from SACC alone.
• CELL_EE only. Today's __main__ builds a CELL_BB HDU and discards it, appending CELL_EE alone; the converter matches. The SACC still carries EE/BB/EB with windows for the harmonic B-mode path.

How it's validated

Byte equality against the current cosmosis_fitting.py (astropy writes FITS deterministically), on both synthetic and real inputs.

• test_twopoint_convert.py (committed CI, 9 tests): build a reference with the current script's own HDU-builders on deterministic synthetic inputs, build a SACC from the same inputs, convert, and assert byte equality — for all three product shapes (plain-xi; xi+Cℓ; xi+ρ/τ). Plus the τ_0↔τ_2 cross-correlation, a perturbation teeth test, sidecar fail-fast, and the adversarial-review hardening below.
• test_twopoint_convert_realdata.py (candide-local, skipif on path existence, 4 tests): build a SACC from a real product's own contents, convert, byte-compare against the current writer. Observed byte-equal for SP_v1.4.6_leak_corr and a glass_mock sibling. The stale on-disk files differ only by an extra CELL_BB HDU (they predate its removal) and ~1e-17 float noise on n(z) bin edges — documented and guarded, no real data committed.
• test_one_covariance_io.py (9 tests): reshape to a hand-built matrix + gaussian/gauss+ng column selection + perturbation teeth; blocks feed assemble_covariance and round-trip s.covariance.dense; n(z) write/read round-trip, the UNIONS z_directory key variant, and fail-fast on mismatched grids / missing bin / bad value_loc.

An independent adversarial review before opening found one HIGH (a tomographic SACC + n_bins=2 silently emitted a plausible-looking file carrying only the (0,0) data — now a loud error), one MEDIUM (ξ-less SACC wrote an empty product), and one coverage gap (every covariance test exercised the identity permutation; a (row,col)-encoded-covariance test now pins the exact np.ix_ gather so any transposition or block swap fails). All fixed in 700a885.

Full fast suite: 163 passed, 2 skipped, 2 failed — both failures pre-existing and environmental (test_bmodes_workflow_dry_runs needs snakemake in the local venv; test_configured_paths_exist_on_candide checks data paths that resolve only from the main checkout), unrelated to this branch.

Note on the on-disk references

The committed 2pt-FITS products on candide were written by an older cosmosis_fitting.py (they carry a CELL_BB HDU and order COVMAT before NZ_SOURCE), so the byte-compare targets the current writer, which PR 4 will use to regenerate them. The direct diff against the on-disk files is verified to be exactly the CELL_BB addition, with every shared HDU matching to float precision.

— Claude (Fable, with Opus implementation agents) on behalf of Cail.

🤖 Generated with Claude Code

https://claude.ai/code/session_016Ko89uZF84ez6HEDcTWbJe