Local telemetry archives for Maple's embedded store (depends on #129)#194
Draft
robbiemu wants to merge 67 commits into
Draft
Local telemetry archives for Maple's embedded store (depends on #129)#194robbiemu wants to merge 67 commits into
robbiemu wants to merge 67 commits into
Conversation
Export acquireCheckpointPin / releaseCheckpointPin and withMaintenanceLock from the checkpoint module so the dependent archive branch can pin an immutable checkpoint against retention and serialize its work against checkpoint creation, restore, and reset. Pin acquisition resolves and validates the checkpoint and writes a UUID-named durable pin record under backups/pins/<id>/; release removes only the exact owned record and fails closed on identity mismatch. The maintenance lock wraps the existing sibling ownership lock with the same dead-PID reconciliation and live-PID refusal semantics. Add the archive module skeleton: ArchiveError typed error, the six raw-telemetry signal map with event-time columns, and the archive path model with strict ID and range-date validation, recursive symlink rejection, root containment, and a live-data-directory separation guard. Twelve pin and lock tests cover acquire/release, retention protection, identity mismatch, escape refusal, and concurrent-owner rejection.
Add a validated, documented tuning model for the seven machine-sensitive archive knobs: Parquet writer threads, row-group rows, maximum shard rows, maximum estimated shard bytes, target chunk bytes, minimum free-space reserve, and the archive and scratch roots. Defaults are the measured research baselines (max_threads=1, 10,000-row groups, ~500k rows / ~256 MiB per shard). The parser rejects non-positive or fractional values, a row group larger than a shard, a shard byte budget too small for one row group, a free-space reserve larger than the chunk target, an implausible thread count, and missing roots. A tuning record captures the effective values for every generation manifest so a generation is reproducible and deployment drift is visible.
Add the versioned, strict archive generation manifest and active-pointer formats with location-bound parsing that rejects unknown versions, signal/range/ generation mismatch, negative counts, malformed shard names, and missing tuning. A read helper binds a manifest to its on-disk (signal, range, generation) directory. Add the Parquet shard exporter: one bounded UTC-hour slice per shard, written via SELECT ... INTO OUTFILE FORMAT Parquet directly on the restored scratch chDB (result consumed as a write side effect, never returned into JavaScript). Each shard is validated for row count, byte bound, and SHA-256 before the generation is sealed. Add the generation write lifecycle: acquire maintenance lock, resolve and pin the checkpoint, restore to sacrificial scratch, export bounded shards, validate row counts against the source, durably move the owned building generation into place, atomically select it through the active pointer (reporting supersession while retaining the old generation), append the catalog, release the pin, and remove only owned building output. Free-space preflight runs at operation time. Twenty manifest, pointer, path-model, promotion, supersession, and catalog tests cover the pure and filesystem-level state machine; the full chDB export path is exercised by the native smoke.
Add the archive read-side. listActiveGenerations reports the active generation per signal/range with archived row count, shard count, total shard bytes, and the active generation's Parquet shard paths in order. Superseded generations are retained on disk but never appear in listings or active-path output, so late-arrival history cannot be double-counted. A malformed active pointer for one range is skipped without hiding the others, and the pointer file is left untouched. activeParquetPaths resolves the machine-readable DuckDB-ready shard paths for one signal across all sealed ranges in ascending order, excluding superseded generations. rebuildCatalog regenerates the per-signal catalog.jsonl from the authoritative generation manifests, so a truncated or missing catalog is recoverable without rescanning Parquet bytes. All retained generations (active and superseded) appear once; a generation directory missing its manifest is skipped. Seven listing and catalog-rebuild tests cover active selection, supersession exclusion, cross-range ordering, malformed-pointer tolerance, and catalog rebuild including superseded and manifest-less generations.
Register a 'maple archive' command group with three subcommands following the server.ts Effect-CLI house style: - 'maple archive create <range-date> <signal>': seal one UTC day of one signal into a validated Parquet generation from the current or a specified checkpoint, using the centralized tuning config and reporting supersession. - 'maple archive list [--format summary|paths|json] [--signal <name>]': report active generations with sizes and paths, or emit machine-readable active Parquet shard paths (excluding superseded generations) for DuckDB. - 'maple archive rebuild <signal>': rebuild a signal's catalog from manifests. All commands are local-only by construction (filesystem/checkpoint work, never WarehouseExecutor), map failures to ArchiveError for non-zero exit, and accept the inherited shared flags without acting on them.
Add docs/local-telemetry-archives.md covering the hot store, immutable checkpoint source, pinning, shared scratch lifecycle, Parquet generations, catalog, calibration, and the independent DuckDB query path with memory-limit and spill guidance. Document the happy path from checkpoint through DuckDB, every major off-happy-path outcome (unavailable/incompatible checkpoint, stale pin, interrupted restore, partial shard, validation mismatch, full volume, pointer or catalog corruption, late arrivals, interrupted GC, insufficient budget, failed calibration), which failures leave the live store untouched vs. require action, the capacity/resource model with its one-machine caveat, and the v1 non-goals. Add the archive create/list/rebuild commands to the local-mode CLI reference, cross-linking to the architecture document.
Add native-archive-smoke.sh: ingest markers into all six raw tables, create two checkpoints, archive a sealed UTC day per signal, list active generations, query each archive with DuckDB for exact source counts (2), prove the live store is unchanged after archiving, and rebuild every signal catalog. Two consecutive runs pass. Fix two real defects the smoke exposed: 1. JSONEachRow count parsing. count() results from chDB's FFI path are newline-delimited JSON objects, not a JSON array. The day-count, hour-count, and time-bounds helpers were JSON.parse-ing the whole string as an array and indexing [0], yielding undefined -> 0 rows. Now split on newlines and read the count() column by name, matching the checkpoint module's readJsonRows. 2. Date predicate correctness. chDB's bundled ClickHouse miscounts aggregate count() over a toDateTime64-vs-DateTime predicate (per-row comparison is correct but the aggregate returns zero). Switched all range predicates to toDate()/toHour() equality, which normalizes both second-precision DateTime and nanosecond DateTime64(9) event-time columns. Also rename the archive list format flag from --format to --output to avoid collision with the global --format (json|table) shared flag, and update the docs accordingly.
…nfig Add 'maple archive calibrate' and its internal 'calibrate-run' worker. The calibrator runs a bounded matrix of Parquet writer/row-group/shard candidates against a pinned checkpoint restored into sacrificial scratch, measuring peak RSS, wall time, and output bytes per candidate in a fresh child process (so peak RSS is measured per-candidate, not accumulated in-process). It selects the lowest-RSS candidate within the operator's memory and time budgets, reports high/low confidence, and writes a versioned configuration document only when --write-config is passed (never auto-applies). A native calibration smoke verified the full round-trip: four candidates ran at ~320MB RSS each, the matrix selected writerThreads=1/rowGroupRows=20000 within a 1GB budget with high confidence, and the generated config recorded the measured peak RSS, output bytes, and effective tuning. An impossible budget fails closed with low confidence and no config mutation. Per D-017, true external-volume deployment-scale calibration under the deployment chDB/user/filesystem is a Phase 3 dependency; Phase 2 proves the mechanism and generated-config round-trip on a local scratch volume.
…ocation (C-1, H-7) The prior approval missed a reproducible external-write vulnerability: symlinked archive descendants (signal root, catalog, manifest, pointer) could be followed by mkdir/write, placing state outside the configured archive root. Path safety (C-1): - ensurePrivateDirectory now walks each ancestor with lstat BEFORE creating, refusing to cross a symlink at any depth (mkdir -p + final lstat was unsafe). - A sync assertNoSymlinkSync variant is added for the synchronous read-side. - promoteGeneration, appendCatalog, and rebuildCatalog now assert no-symlink on every path they create or overwrite, immediately before use. - assertSafeRealPath verifies type + containment + no-symlink at read/write sites. Strict state binding and fail-closed malformed handling (H-7): - parseArchiveActivePointer binds signal/range to its directory, rejecting a pointer copied or moved to the wrong range. - listActiveGenerations surfaces malformed pointers/manifests as errors instead of silently skipping them, while still listing unaffected ranges. - Catalog entries now carry formatVersion 1. Eight external-sentinel tests prove the escapes are closed: symlinked signal root, catalog, manifest, pointer, and ancestor; outside targets are verified untouched. Malformed-pointer surfacing and pointer-directory binding are tested.
… review) The R1 fix closed the C-1 write side but left the read side open: a non-racing attacker who plants one symlinked descendant (generation dir, shard file, or range dir) could make listActiveGenerations/activeParquetPaths/rebuildCatalog follow it out of the archive root, feeding attacker-controlled Parquet to DuckDB and attacker-controlled manifest fields to the catalog. The fix's own guard primitives were written but never wired into read paths. This closes the asymmetry: - readArchiveGenerationManifest (the single chokepoint for manifest reads) now asserts no-symlink + real-file before readFileSync, so a symlinked signal/range/generation/manifest chain is refused. - listActiveGenerations asserts no-symlink on the active-pointer path before reading it and on every shard path before returning it to DuckDB. - The dead assertSafeRealPath helper (never called) is removed. Three read-side sentinel tests prove the escapes are closed: a symlinked generation dir is refused by readArchiveGenerationManifest and rebuildCatalog (the attacker manifest is not trusted); a symlinked shard file is refused by listActiveGenerations and surfaced as an error rather than returned to DuckDB. Outside targets are verified unread.
…, fresh-root creation (H-7/R7-R8) Close four defects from the Gate 1 conditional review: 1. activeParquetPaths now THROWS when any relevant range for the requested signal has a malformed pointer/manifest/shard, rather than returning a partial path list that would silently feed DuckDB incomplete data. Errors for other signals do not block a clean signal's paths. 2. rebuildCatalog now PREFLIGHTS every manifest before writing. If any generation is missing its manifest, is malformed, or is on a symlinked path, the existing catalog is PRESERVED untouched and the call throws. The old code silently skipped corrupt generations and wrote a partial catalog. 3. Pointer reads now require a real regular file (assertRealFileSync), matching the manifest-read check, so a non-file entry cannot feed undefined content. 4. ensurePrivateDirectory's root parameter is now MANDATORY (was optional and degenerate when omitted). Fresh archive-root creation is repaired: the root is created first before walking children, fixing the ENOENT on a fresh root. assertRealFileSync is moved to paths.ts as a shared sync primitive. Three new tests cover the fail-closed activeParquetPaths (malformed same-signal range throws; different-signal error does not block), rebuild catalog preservation on missing manifest, and fresh-root nested directory creation.
…dation (H-1, H-2) R3 — bounded sharding: a sealed UTC day is now partitioned by UTC-hour windows, then within each hour by LIMIT/OFFSET sub-splits when the hour exceeds maxShardRows or the estimated uncompressed-byte budget (maxShardBytes). The prior code aborted if a single hour exceeded the bound instead of splitting. Shard names encode their slice: HH-NNNN.parquet (hour + sequence). R2 — pre-promotion Parquet validation: after writing each shard, it is REOPENED via chDB file(path, Parquet) and its row count, min/max event time, and column list are READ BACK from the Parquet file. A 19-byte invalid 'Parquet' file fails this reopen because file() cannot parse it. The shard record now carries the REOPENED counts and columns, not the source counts — closing the tautology where the record always matched the source query. An empty (0-row) reopen also fails. The ArchiveShardRecord gains a columns field (schema round-trip proof); the manifest parser strictly validates it. Test fixtures updated to the new shard name format and columns field.
…ath escaping (CR-1, H-A, H-B, H-C, M-1, M-2) Close the adversarial R2/R3 review's release blockers: CR-1 — sub-sharding now uses ORDER BY (_part, _part_offset) + LIMIT/OFFSET, making each hour's sub-shards an EXACT partition (no overlaps, no gaps). The prior LIMIT/OFFSET without ORDER BY was nondeterministic and could silently duplicate or drop rows. The last shard gets the remainder via min(rowsPerShard, hourRows - offset), and the expected row count is computed and passed to validation. H-A — DESCRIBE file() failures are no longer swallowed. A schema that did not round-trip fails the shard (columns.length === 0 throws), so the manifest's columns field is a real guarantee, not best-effort. H-B — validateShard now checks the reopened row count against the intended slice size AND verifies all rows fall within the expected hour window (toHour(min) === toHour(max) === hour), so a shard containing rows from the wrong hour is rejected. H-C — the byte bound is now enforced on UNCOMPRESSED bytes via parquet_metadata().uncompressed_size, not compressed on-disk size. Compression could previously keep an oversized shard under the compressed ceiling. M-1 — operator paths containing single quotes or backslashes are rejected before export (assertSafePath); both INTO OUTFILE and file() literals use the same sqlLiteral escaping. M-2 — the promoteGeneration test no longer passes a stale 7th argument.
…lash rejection) The adversarial R2/R3 review flagged that no test exercised the path-rejection (M-1) primitives. Add two pure-logic tests proving exportSignalShards rejects a shardsDir containing a single quote or backslash before any chDB query runs. The full Parquet write+reopen+validation path is exercised by the native smoke.
…Gate 2 NO-GO)
Fix two confirmed chDB runtime incompatibilities the native smoke exposed:
- formatRow(RowBinary, *) -> formatRow('RowBinary', *): the bare token RowBinary
is an unknown identifier in chDB (code 47); it must be a quoted string literal.
- parquet_metadata() -> file(path, ParquetMetadata).total_uncompressed_size:
parquet_metadata() is DuckDB syntax, not ClickHouse; bundled chDB exposes
uncompressed size via the file() ParquetMetadata interface.
Strengthen validation to compare against source intent (not just 'parseable'):
- captureSourceSchema captures name+type before export; validateShard compares
the reopened Parquet schema against it (base type match), so a schema that did
not round-trip fails the shard.
- validateShard verifies the UTC DATE of all rows (toDate min==max==rangeDate)
in addition to the hour window.
- estimateBytesPerRow and validateShardBytes now use the corrected chDB APIs.
Strengthen manifest strictness (H-7):
- sha256 must be 64 hex chars; counts must be safe non-negative integers.
- Cross-field: unique shard names, shard-row sum == archivedRowCount,
sourceRowCount == archivedRowCount, rangeEndExclusive == next-midnight.
- parseShardRecord requires nonempty columns; min <= max event time.
Other repairs:
- Strict calendar date validation (reject 2026-02-31; JS Date normalizes it).
- rangeEndExclusive is now the next midnight (exclusive), not 23:59:59 inclusive.
- Free-space preflight checks the actual destination volume (climbs to the
closest existing ancestor when the root is missing) and includes working bytes.
- rebuildCatalog is async and uses durableWrite (atomic temp+fsync+rename) so an
interruption cannot destroy the prior catalog.
- Active shard paths are asserted to be real existing regular files.
- Pin-release failures are surfaced to stderr, not silently swallowed.
133/133 tests pass; typecheck, lint, format clean.
The native smoke exposed three runtime issues after the API fixes: 1. Timezone: toDate()/toHour() without an explicit 'UTC' argument used the host session timezone, causing the export query and the validation query to shard by different hours (12 UTC stored, 08 EDT computed). All toDate/toHour calls now pass 'UTC' explicitly in export.ts and generation.ts. 2. Schema normalization: Parquet export strips LowCardinality(...) wrappers and widens DateTime to DateTime64. The schema comparison now unwraps LowCardinality/Nullable and normalizes DateTime to DateTime64 before comparing base types, so legitimate Parquet type transformations pass while real schema loss still fails. 3. Smoke determinism: the insert markers now use toDateTime64(..., 9, 'UTC') at a fixed UTC noon timestamp within RANGE_DATE, so the archive's toDate/toHour predicates find the rows regardless of the host timezone. The prior now()/ now64(9) inserts landed on a different UTC day when local and UTC diverged. Two consecutive native archive smokes PASS: 6 signals archived, DuckDB exact counts (2 each), live store unchanged, catalog rebuilt. 133/133 unit tests, typecheck, lint, format clean.
…TICAL cursor defect) Replace the broken ORDER BY (_part, _part_offset) LIMIT/OFFSET pagination with a true static-snapshot physical plan that is merge-safe: 1. For each UTC hour, enumerate the active parts and their _part_offset ranges (freezing the layout into a shard plan). 2. Split each part's offset range into bounded intervals (one Parquet file each). 3. Export each shard via WHERE _part = '<part>' AND _part_offset BETWEEN <lo> AND <hi> — a physical predicate targeting exact immutable rows, not a relative OFFSET that drifts when parts merge. 4. After all shards for the hour, re-enumerate and verify every planned part still exists with the same row count. If a merge removed or changed any part, the export fails closed. The prior OFFSET approach was confirmed corrupting data: an injected merge between shard queries produced [5,6,7,8,5,6,7,8], omitting [1,2,3,4] while all local validations and the aggregate count passed. The part-interval plan targets exact rows by immutable location, so a merge of a different part cannot affect this shard's rows, and if THIS part merges away, the post-export verifyPartsUnchanged detects it. A multi-part adversarial native probe (native-archive-merge-probe.sh) reproduces the cross-check's corruption scenario and confirms the fix: 2 parts → 2 shards, exact ID match [t0..t7], no duplicates. The dead estimateBytesPerRow is removed; the byte bound is enforced post-write via parquet_metadata uncompressed size.
…amper detection) listActiveGenerations now hashes each shard file and compares its SHA-256 and byte size against the manifest before returning the path to DuckDB. A tampered regular file whose actual hash or size disagrees with the manifest is rejected (D-015's checksum-mismatch fail-closed rule). The manifest is authoritative; a mismatched file is surfaced as a listing error, not silently returned. Test fixtures updated to compute real SHA-256 and byte size from placeholder content, and to use the new HH-NNNN.parquet shard name format. 133/133 pass.
… comparison Three remaining cross-check blockers closed: 1. Merge freeze: exportSignalShards now runs SYSTEM STOP MERGES before enumerating parts and SYSTEM START MERGES in a finally. Without this, a background merge on the restored scratch store changed part names between enumeration, export, and source-interval validation, causing stale-part false failures (and on the old OFFSET approach, silent corruption). The freeze makes the physical layout stable for the entire export. 2. Source-interval validation: validateShard now re-queries the SOURCE table for the same (_part, _part_offset) interval and compares count + nanosecond time bounds (via toUnixTimestamp64Nano(toDateTime64(...))) with the reopened Parquet. This proves each shard contains exactly the source rows for its physical interval — a wrong-but-valid row set can no longer pass. Time bounds use nanosecond comparison to avoid session-timezone string mismatches. 3. Exact schema comparison: compareSchema now enforces exact column count, name, and order (positional match), plus base-type compatibility (LowCardinality/ Nullable unwrapped, DateTime normalized to DateTime64). Extra/dropped/ reordered columns fail closed. Nested type parameters (Map key/value types) are still normalized to the base token — documented as a compatibility limitation. Also: ExportSettings gains an afterShardValidated callback for the adversarial merge-injection probe. The sha256FileSync comment is corrected (reads whole file, not streaming). verifyPartsUnchanged is replaced by verifyHourTotalUnchanged (robust to part-name churn from merges, detects data loss/gain). Two native smokes PASS: six-signal (with source-interval validation on all six tables) and merge-safety (2 parts → 2 shards, exact ID match). 133/133 unit tests.
…apleTechLabs#5) A standalone probe that reproduces the exact cross-check corruption scenario: two parts for the same UTC hour, with an OPTIMIZE TABLE ... FINAL injected between shard exports via the afterShardValidated hook. SYSTEM STOP MERGES blocks the OPTIMIZE (Code 236, Cancelled merging parts), the physical layout is frozen, and the exported shards contain the exact source row set [t0..t7] with no duplicates or omissions. This is the definitive proof that the static-snapshot part-interval plan is merge-safe: the prior OFFSET approach produced [5,6,7,8,5,6,7,8] under the same scenario. The probe uses the real bundled chDB via FFI (MAPLE_LIBCHDB).
added 29 commits
July 9, 2026 08:14
…ration (Gate 3b) Raise the operation journal format v2 → v3 with a discriminated kind field (create | gc) so reconcile can dispatch on operation type. The strict parser now validates kind first, then create-vs-gc fields; create intents carry the same fields as before plus kind:'create'. GC intents record the frozen deletion set, keep, manifest+shard evidence per target, and progress. A v2 (pre-kind) create intent left by a Gate 3a binary is migrated to v3 under the maintenance lock by migrateActiveIntentIfLegacy before reconcile reads it, so a stranded 3a intent is reconciled rather than blocking all future archive work. Migration is a mechanical, re-validated field addition; a corrupt v2 record fails closed rather than being silently lifted. The v3 parser rejects raw v2.
…tion (Gate 3b) reconcileArchiveGeneration now dispatches on operation kind: create ops run the existing 3a lifecycle recovery; gc ops delegate to reconcileGcOperation in gc.ts. A crashed GC shares the single operations/active/ slot, so it MUST reconcile or it blocks all future archive work. planArchiveGc enumerates superseded generations under the maintenance lock and builds a frozen deletion set: --keep N (default 1) retains the newest N superseded per range; the active generation is never selected; a signal whose catalog cannot be authoritatively reconstructed is excluded ENTIRELY before any mutation (never delete a range then discover reconstruction is impossible); malformed/symlinked/ambiguous state fails toward over-retention. Collection is crash-safe via tombstone rename — never in-place recursive delete: each target is atomically renamed source→tombstone (manifest intact, no half-deleted window), then only the journal-owned tombstone is removed. A SIGKILL mid-collection leaves whole owned state that reconcile proves it owns. Each target revalidates the pointer (CAS — must equal the recorded active, not merely 'some other gen') and the source (manifest SHA + per-shard SHA) before rename; a pointer that came back onto a target stops collection and preserves it. Reconciliation resumes from the recorded progress cursor and NEVER expands the frozen set.
maple archive reconcile [--dry-run]: converge an interrupted create or gc op to its intended state without a fresh export, inside the maintenance lock. Idempotent; ambiguous/malformed state exits nonzero without mutation; no-op when nothing is active. maple archive gc [--keep N] [--dry-run]: reclaim superseded generations via the tombstone collector, retaining the newest N per range (default 1; 0 reclaims all). --dry-run reports the exact delete/retain sets WITHOUT mutating anything (no journal, no deletion, no catalog change). Rejects negative/fractional --keep. Both acquire the maintenance lock and reconcile any prior op before planning new work. Adds a red() style helper for the over-retention/excluded warning lines.
…trix (Gate 3b) native-archive-gc-probe.sh injects a REAL SIGKILL at each of 5 GC lifecycle boundaries (intent-durable, first-rename, during-removal, after-all-removals, after-catalog) via a committed gc worker paused at fault seams, then converges via the real maple archive reconcile CLI and verifies: only frozen targets removed, active generation intact + DuckDB-queryable, pointer unchanged, catalog exactly matches manifests, no tombstone retains a generation, completed journal retained, second reconcile is a no-op, and a subsequent archive create succeeds (a crashed GC never blocks future work). Plus a dry-run-mutates-nothing check. archive-gc.test.ts covers keep-N/keep-0 ordering, never-active selection, fail-closed on malformed/symlinked/ambiguous, signal-level exclusion, and dry-run filesystem state unchanged. archive-journal.test.ts adds v2→v3 migration tests (clean lift, corrupt-v2 fail-closed, stranded-on-disk lift, no-op). The adversarial matrix gains a 'Garbage collection (Gate 3b)' section (invariant 10) recording every boundary, the tombstone topology, the CAS pointer guard, and the over-retention working rule.
…ng-pointer (Gate 3b repair) Repair the seven NO-GO blockers from the Gate 3b round-1 review: 1. GC state machine: introduce a nonterminal gc-collecting phase (allows a full cursor — the legitimate post-final-deletion state). Progress is persisted as gc-collecting per target, NEVER complete. complete is written only after all targets are absent + every affected catalog passes assertCatalogExact. Fixes the premature-complete defect where reconcile archived a journal mid-collection. 2. Terminal invariant proof: a complete GC journal is verified before archival (completedTargets === targets.length, every target source + tombstone absent, every affected pointer unchanged, every catalog exact). A failed invariant preserves the active journal (fail closed). reconcileGcOperation re-reads the durable record (not the stale in-memory cursor) before verifying. 3. Phase-strict parser: the v3 union rejects kind-incompatible phases (a GC intent at pin-acquired), aborted for GC, and inconsistent phase/cursor combinations (complete requires a full cursor; intent requires cursor 0). Duplicate-target detection. 4. Locked explicit reconcile: runArchiveReconciliation acquires the maintenance lock before migrating/reconciling; the CLI calls only this (no racing create/GC planning, v2 migration, or pointer/catalog repair). 5. Nonmutating dry-runs: a shared planArchiveReconciliation planner is consumed by both dry-run and apply. gc --dry-run never reconciles (the reconcile-first call is apply-only); if an active op is present, dry-run reports the blocker rather than predicting a deletion set from unstable state. planArchiveReconciliation inspects journals WITHOUT migrating (migration is an apply action). 6. Missing pointers fail closed: a range with no active pointer is uncertain — excluded entirely, nothing targeted, no empty-string sentinel that strands the journal. 7. Faithful crash boundary: a dedicated awaited afterTargetProgress(index, total) seam replaces the duplicate during-removal label; the nonfinal-progress boundary (index < total-1) now exposes the premature-complete defect. The oracle verifies EXACT active/pointer/frozen-target/journal/catalog state after CLI reconcile.
…ed, shared plan (Gate 3b repair r3) Repair the five round-2 NO-GO blockers, all in the explicit reconciliation wrapper (no GC redesign requested): 1. Apply ALWAYS acquires the maintenance lock FIRST, then re-plans under it — never returns success for unsafe state without locking. 2. Valid v2 migration is an ACTION (migrate + reconcile), not a blocker. The planner reports needsMigration=true; apply migrates under the lock. A corrupt v2 intent is fail-closed (will not migrate). 3. Malformed/ambiguous/unreadable/unknown-debris state FAILS CLOSED (throws) — never reported as successful reconciliation. listActiveOperationIdsSafe (which silently filtered debris into absence) is replaced by a strict enumerator that surfaces unknown entries and read errors. 4. dry-run and apply share ONE strict nonmutating planner using the authoritative journal reader (parseArchiveOperationIntent), so the plan apply consumes is the same validated plan dry-run reports — no shallow raw reads. 5. Oracle tightened: a missing catalog fails (not skipped); the completed GC journal's target IDs are compared with the pre-crash frozen set (never re-expanded). Adds archive-reconcile.test.ts (8 tests) covering the locked-wrapper contract: v2-through-wrapper, malformed/multi/debris fail-closed, corrupt-v2 fail-closed, dry-run nonmutation with a v2 intent, apply-throws-on-unsafe, no-op success.
…tions (Gate 3b r4) Replace the parallel weaker reader + metadata-only planner with one authoritative protocol per the round-3 review's explicit guidance: 1. ONE authoritative V2/V3 inspector (inspectActiveOperation in journal.ts): reuses listActiveOperationIds (root containment + no-symlink + per-entry debris/prefix/ validateArchiveId) + the guarded readIntent path (assertNoSymlinkSync + assertRealFileSync before readFileSync) + directory/record operationId binding. Reads v2 in-memory (no write); returns no-op | fail-closed | a validated snapshot. DELETE the parallel listActiveOperationIdsStrict + shallow planArchiveReconciliation reader from generation.ts. 2. migrateActiveIntentIfLegacy consumes the inspector: all pre-write checks (no-symlink, real-file, directory/record binding, clean v2 lift) run BEFORE any durableJson write. A symlinked intent, a mismatched ID, or a corrupt v2 record is detected before mutation. Closes the v2-mutates-before-detecting-identity/symlink blockers at the mutation site (defense in depth). 3. Concrete discriminated-union plan (ReconciliationPlan = no-op | fail-closed | create | gc) with EXACT ordered actions (CreateAction/GcAction) and preconditions. No opaque reconcile-create/reconcile-gc rediscovery: buildCreatePlan/buildGcPlan make the branch decision ONCE from observed topology; the executor dispatches per-action via named helpers, revalidating each precondition before mutation. 4. One locked protocol: both dry-run and apply acquire the maintenance lock before inspection; dry-run performs NO mutation (no migration, no reconcile) and a fail-closed plan returns NONZERO; apply executes only the planned actions and captures the terminal postcondition WHILE HOLDING THE LOCK (no post-unlock re-plan → no reporting TOCTOU). Adds state-space tests: v2/v3 × valid/mismatch/symlink/debris × dry-run/apply, lock contention (live owner blocks dry-run), symlinked-intent outside-target survival through both modes, v2-mismatch-never-rewritten-before-rejection.
…sition table (Gate 3b r5) Introduce the ONE pure transition-table function (decideReconciliation) that is the sole branch logic for archive reconciliation. No I/O, no mutation. Both dry-run and apply will consume the SAME ReconciliationDecision; there is no second if-phase implementation to diverge (the defect that recurred across rounds 2–4). inspectReconciliationState (to follow in the wiring commit) will produce a ReconciliationInspection = null | FailClosed | ValidSnapshot, so validation failures (root/pin/scratch/pointer-CAS/owned-path/identity) enter the SAME decision function as valid state — every unsafe state is decided uniformly. The decision union: NoOp | FailClosed | CreateVerifyComplete | CreateAbortPrepublication | CreateFinishPublication | GcVerifyComplete | GcResume. Each carries exact identities (operationId, journalDigest), observed preconditions, affected signals/targets, and migrationRequired. Uses the canonical PHASE_ORDER/phaseAtLeast from journal.ts (no duplicated sequence). Includes the building+final impossible-topology gate. Preserves the lifted v2 record's exact phase (a v2 record can hold any create-eligible phase, not just intent). Exports PHASE_ORDER from journal.ts for reuse. 21 exhaustive transition-table tests cover every (kind, phase, topology) row and every FailClosed gate, asserting the exact decision. Post-promotion is always CreateFinishPublication (unconditional repair — selectActiveGeneration + rebuildCatalog — NOT phase-conditional verify). Complete phases are verify-only. Multi-signal GC carries explicit affectedSignals.
… decision, one executor (Gate 3b r5) Wire the pure decideReconciliation as the SOLE branch logic, deleting the parallel r4 action engine entirely. All reconciliation now flows through one path: inspectReconciliationState → decideReconciliation → execute branch via proven helpers - inspectReconciliationState: runs inspectActiveOperation (symlink-safe V2/V3 inspector) + the full proven validation (assertReconciliationRoots, validateReconciliationTopology, validateOwnedPinState). Any failure → FailClosed (zero mutation). For v2, lifts in-memory preserving the EXACT phase. Returns a complete validated ReconciliationSnapshot. - decideReconciliation: the ONE pure transition table (no I/O). Maps the snapshot to NoOp | FailClosed | CreateVerifyComplete | CreateAbortPrepublication | CreateFinishPublication | GcVerifyComplete | GcResume. - reconcileArchiveGenerationUnderLock: inspect → decide → (dry-run: return; apply: migrate-v2 if needed, then execute via the proven reconcileArchiveGeneration, capture postcondition under lock). - runArchiveReconciliation (CLI): acquires the lock, calls the under-lock function. FailClosed → nonzero in both modes. DELETE: buildCreatePlan, buildGcPlan, executeCreateAction, executeGcPlan, observeCreateTopology, quarantineBuildingStep, planFromInspection, the CreateAction/GcAction unions, the old ReconciliationPlan/ReconcileTopology types. The parallel engine is gone; there is one decision function. 238/238 tests pass (21 pure decision + 13 protocol + 204 existing). 0 lint. Typecheck, format, diff-check clean.
…n it directly (Gate 3b r6) The core fix: reconcileArchiveGeneration no longer independently re-branches. It inspects → decides → switches on the computed ReconciliationDecision, executing ONLY the branch-specific helpers for that decision. The decision IS the operative state machine. All three entry points (CLI, automatic create, automatic GC) call this same function — there is one call graph, one decision function, one executor. The inspection now includes branch-significant read-only preconditions: - create post-promotion: verifyPublishedGeneration (manifest SHA + shards); - create terminal: verifyCompletedOperationInvariants; - GC: assertReconciliationRoots + verifyCompletedGcInvariants for complete. So dry-run returns FailClosed for the same state apply rejects (only for valid topology; impossible topologies are gated by the decision function from snapshot fields before any branch-specific verification runs). 238/238 tests pass. 0 lint. Typecheck, format, diff-check clean.
…e 3b r7) Close the four round-6 bounded defects (no redesign): 1. Create post-promotion inspection now includes assertPointerConsistent — a conflicting pointer is FailClosed in dry-run AND apply (parity). 2. GC resume preflight: preflightGcTargets validates EVERY remaining target's source/tombstone topology, manifest/shard evidence, pointer CAS, and both-present detection read-only before the decision authorizes any mutation. Prevents partial deletion where target 1 succeeds but target 2 fails. 3. GC executor split: GcVerifyComplete → verifyCompleteAndArchiveGc (verify + archive, no repair); GcResume → resumeFrozenTargetsAndCompleteGc (resume + complete + verify + archive). The executor dispatches directly to each — no phase re-branch. The decision IS operative for GC as well as create. 4. Destination collision preflight: completed-op dir + quarantine dir existence checked in inspection before the decision authorizes earlier actions. Adds a dry-run/apply parity test: conflicting pointer → FailClosed in both modes, zero mutation, state preserved. 239/239 tests. 0 lint. Typecheck, format, diff-check clean.
…ks + parity tests (Gate 3b r8)
Complete the round-7 GC preflight gaps:
1. preflightGcTargets now validates the ENTIRE frozen set:
- prefix (index < cursor): source absent, tombstone absent, pointer CAS;
- current/suffix (index >= cursor): all documented crash topologies with
evidence validation. Prevents partial deletion when a prefix target is
corrupted but suffix targets are valid.
2. revalidateTombstone: for source-absent/tombstone-present targets, proves the
tombstone is a real non-symlinked directory with manifest/shard evidence
matching the frozen set. A symlinked or tampered tombstone → FailClosed before
any mutation.
3. Collision probes replaced with lstatSync-based pathExistsIncludingSymlinks:
catches broken/dangling symlinks at deterministic destinations that existsSync
misses.
Adds two GC multi-target hostile parity tests:
- cursor-ahead (completedTargets=1 but target 0 source exists): dry-run FailClosed,
apply nonzero, zero mutation (byte-identical snapshot).
- source-absent/tombstone-present with symlinked tombstone: dry-run FailClosed,
apply nonzero.
241/241 tests. 0 lint. Typecheck, format, diff-check clean.
… states + ENOENT-only collision (Gate 3b r9) Close the three round-8 bounded defects: 1. Replace ALL existsSync source/tombstone topology checks in gc.ts with a shared lstat-aware tri-state classifier (classifyPath): absent only on ENOENT; real-directory/real-file validated; symlink or non-ENOENT error → fail closed. Used in preflightGcTargets, collectOneTarget, and verifyCompletedGcInvariants. Dangling symlinks are no longer treated as absent. 2. Enforce distinct prefix/current/suffix topology in preflightGcTargets: - prefix (i < cursor): source absent, tombstone absent, pointer CAS; - current (i === cursor): documented crash topologies (source/tomb states); - suffix (i > cursor): source-present + tombstone-absent + evidence + CAS only. A suffix target that is already absent or tombstoned → FailClosed (impossible out-of-order mutation). 3. pathExistsIncludingSymlinks: returns false ONLY on ENOENT; every other lstat error propagates as fail-closed (permission, I/O, malformed path). Adds three hostile parity tests with complete durable-state snapshots: - dangling source symlink → FailClosed, zero mutation (byte-identical); - dangling tombstone symlink → FailClosed, zero mutation; - impossible suffix (target 2 absent while cursor=0) → FailClosed, zero mutation. 244/244 tests. 0 lint. Typecheck, format, diff-check clean.
Expand docs/local-telemetry-archives.md from a user overview into the full operator and architecture reference required by the plan: the six signals and their event-time columns, the complete directory layout, the manifest (v3), active pointer (v1), and catalog.jsonl formats, the full tuning field reference with defaults and validation constraints, the calibration workflow (candidate matrix, worst-case aggregation, margin-inside-ceiling, disjoint held-out validation, no-recommendation cases), recovery and the single reconciliation decision function, the GC lifecycle, and the complete off-happy-path catalog with the untouched/debris/reconcile/intervention classification. Add docs/pr-local-telemetry-archives.md as the fresh long-form draft PR description (what/why, dependency on PR MapleTechLabs#129, resource and adoption implications, happy path, failure outcomes, validation evidence, known limits, follow-up). This is written against the branch head and does not reuse prior stale draft PR materials.
… volume binding Architectural repair for the R9/R10 fresh-review NO-GO (D-024), in one commit atop 5d891e83. Manifest v3 strictness (finding 1). parseArchiveGenerationManifest is now exact-key (assertExactOwnKeys over MANIFEST_KEYS) and rejects unknown top-level fields. The tuning block is parsed by parseTuningRecord, which enforces positive integers and the same cross-field invariants as the writer. Omitting tuningConfig now fails closed instead of normalizing to null; zero/invalid tuning is rejected. tuningConfig accepts the two known config format identities (legacy v1 + verified v2) since the manifest stores an opaque hash-bound identity; only the loader refuses v1 for new writes. Calibration intent-retention wedge (finding 2). reconcileCalibration catches a resolveCheckpoint failure and, when the record is a provably inert intent (phase intent, pinPath null, every exact derived checkpoint/pin/scratch/ sample resource absent via classifyArchivePathSync), retires the record without resolving the retired source. Any ambiguity preserves the record. Calibration/config acceptance contract (finding 3). Calibration is now a parent-owned session: the parent resolves ONE checkpoint, acquires ONE pin, and threads the exact checkpointId+fingerprint to every child. Children (assertCalibrationSession) refuse current/previous, bind to the parent's operation/checkpoint/fingerprint, verify the parent pin is live, and clean only their own sample (cleanupCalibrationSample); the parent owns full reconcileCalibration in a finally. Held-out results, worstCase, comparisons, tolerances, and a heldOutAttempts log are retained in the recommendation and written to the config. Tolerances are one canonical exported constant (HELD_OUT_TOLERANCES) and the loader recomputes the comparison, rejecting a forged withinTolerance. The two non-candidate knobs are DERIVED (deriveTargetChunkBytes = max(4*maxShardBytes, reserve+maxShardBytes); minFreeSpaceReserve = budget.freeSpaceReserve) and the derivation formula + recomputed effective block are re-verified by the loader. Config format bumped to 2; confidence must be high to load. A new calibrate-session command (open/close) makes the parent session lifecycle explicit so a single child or SIGKILL probe can run against an open session and an operator can retire a wedged one. Environment + volume identity at create (findings 5,7). archive create --config enforces the recorded mapleVersion/chdbVersion/schemaFingerprint/ executionUser/platform/arch/cpuModel/cpuCount/totalMemoryBytes and the archive-volume fsid/type + canonical path against the live host (assertCalibrationEnvironment), and re-checks the volume identity immediately before publication (beforePublicationVolumeRecheck seam). archiveVolumeIdentity no longer climbs ancestors: it requires an existing real non-symlink canonical directory. R9 documentation corrections. Recovery, quarantine, catalog, calibration cleanup, operation paths, free-space, list output, volume, and capacity claims are corrected to match actual behavior; both docs pass oxfmt. Tests/probes: 303/303 unit (+2 env/volume negative tests; manifest and semantic-rewrite coverage). The calibration probes are rewritten to the session model: the crash probe covers a SIGKILLed sampling child and the retired-intent wedge; the calibrate probe adds a forged-volume rejection check.
…int held-out scope
Closes the four R9/R10 repair-review blockers (D-024, round 2) in one commit
atop 3a7ac948. Findings 1, 2, and 4 plus the core of finding 3.
Canonical tolerances + triggers (finding 1). HELD_OUT_TOLERANCES and
RECALIBRATION_TRIGGERS now live once in config.ts
(CALIBRATION_HELD_OUT_TOLERANCES / CALIBRATION_RECALIBRATION_TRIGGERS) and the
loader requires the document's heldOut.tolerances and recalibrationTriggers to
exactly match them (exactJson), so a document can no longer redefine its own
comparison policy to pass a 1000x wall regression. Tolerances tightened so
every metric is < 1.0.
Strict parent-pin identity (finding 2). assertCheckpointPinIdentity is one
shared strict validator (exact keys, formatVersion, pinId, checkpointId,
purpose regex, parseable createdAt) used by both releaseCheckpointPin and
assertCalibrationSession. assertCalibrationSession now parses the derived pin
and requires exact id/checkpoint/purpose, so a substituted same-path regular
file (e.g. '{}') is no longer accepted as the live parent pin.
Larger, persisted, auditable held-out scope (finding 3). The held-out window
is now STRICTLY LARGER than training and disjoint: training covers ordered rows
[0, sampleRows); held-out covers [sampleRows, sampleRows + 2*sampleRows). Every
training and held-out result carries a persisted sample scope
({checkpointId, checkpointManifestFingerprint, rangeDate, role, startRow,
requestedRows, rowCount}) emitted by the child (the authoritative source) and
attached by the parent. The config document records a samplePolicy
(trainingRows, heldOutMultiplier=2, heldOutRows, training/heldOut windows), and
the loader re-verifies: all scopes bind to ONE checkpoint/range; training is
role training, startRow 0, requestedRows=budget.sampleRows; held-out is role
held-out, startRow=sampleRows, requestedRows=heldOutRows; rowCount matches
metrics.rowCount; failed results record no scope. Representative-data confidence
now matches all four candidate fields (isSameCalibrationCandidate), not just
writerThreads. Metric-coherence validation rejects forged
compressionRatio/throughput; positive freeSpaceReserve enforced; recalibration
triggers exact.
R9 documentation (finding 4). Both docs corrected to config v2, 306 tests, the
canonical <1.0 tolerances, null-selected throws (no config written), the new
samplePolicy/scope fields, and the larger disjoint held-out window. The stale
'separate validation report' and 'low-confidence config may be written' claims
are removed.
Native calibrate probe asserts the larger persisted held-out scope and
samplePolicy for every result. Unit tests add five hostile scope-forgery cases
(forged checkpoint, role, non-disjoint startRow, rowCount mismatch, samplePolicy
multiplier) and a held-out-larger-and-disjoint property. 306/306 unit;
typecheck/lint/format/diff-check clean.
…mparison
Closes finding 3 (held-out contract) on top of the round-2 repair. The held-out
sample is now strictly LARGER than training and disjoint: training covers ordered
rows [0, sampleRows); held-out covers [sampleRows, sampleRows + 2*sampleRows)
(heldOutSampleRows = HELD_OUT_SAMPLE_MULTIPLIER * sampleRows). Every training and
held-out result carries a persisted sample scope
({checkpointId, checkpointManifestFingerprint, rangeDate, role, startRow,
requestedRows, rowCount}) emitted by the child and attached by the parent; the
config records a samplePolicy and the loader proves every scope binds to ONE
checkpoint/range and that the two windows are disjoint and correctly sized.
Hybrid predicted-vs-observed comparison (per the calibrator contract): because a
2x-larger held-out naturally takes ~2x wall time and bytes, wallMs and
physicalBytes predictions are rescaled by heldOut.logicalBytes /
training.logicalBytes before the two-sided check; throughput and compressionRatio
(size-invariant rates) compare directly; peak RSS and peak temp disk compare as
absolute peaks (NOT scaled by row count — they are process peaks, not per-row
costs). The raw values, scale ratio, training/held-out logical bytes, adjusted
predictions, and comparison results are all persisted; the loader recomputes the
ratio and every comparison from the recorded evidence using the fixed canonical
tolerances only (no run-specific widening). comparePredictedObserved gains an
optional sizeScaling param mirrored by the loader's expectedComparisons.
Native calibrate probe asserts the larger persisted held-out scope and
samplePolicy for every result (training=10 held-out=20, disjoint, single source)
and the loop closes through the hybrid comparison. Unit tests add hostile
forged-scale-ratio and forged-scope cases plus a held-out-larger-and-disjoint
property. 306/306 unit; typecheck/lint/format/diff-check clean.
Closes the final D-026 contract: the held-out comparison is now PER-SIGNAL and
like-for-like, replacing the cross-signal aggregate ratio that could mask a
single-signal regression.
Each signal's held-out result is paired with the same candidate's TRAINING
result for that signal (compareHeldOutPerSignal). wallMs/physicalBytes are
scaled by THAT signal's own heldOut.logicalBytes/training.logicalBytes ratio;
throughput and compressionRatio are compared directly (size-invariant rates);
peak RSS and peak temp disk are compared as absolute peaks. The attempt passes
only when ALL SIX signals pass; cross-signal aggregate extrema (worstCase) are
recorded only as a descriptive summary and never decide acceptance.
Data model (config format stays v2): heldOut and each heldOutAttempts entry
carry signalComparisons [{signal, scaleRatio, comparisons, passed}] in canonical
six-signal order — no duplicated raw metrics (the loader re-derives pairs by
exact candidate + signal identity). A complete attempt has six entries even
when it fails; an incomplete/over-budget/short-window attempt has
signalComparisons: [], worstCase: null, passed: false. Training and held-out
logicalBytes must both be strictly positive (an undefined ratio makes the
attempt incomplete, never silently ratio 1).
The loader recomputes every per-signal ratio, adjusted prediction, relative
delta, and pass flag via expectedSignalComparisons and requires exactJson
agreement. The runner-level compareHeldOutPerSignal is pure and unit-tested
with the reviewer's exact cross-signal counterexample (signal A regresses 1.0
under per-signal but aggregate reports delta 0). Hostile cases updated for the
per-signal shape (forged comparison, forged scaleRatio, forged canonical
tolerances with recomputed per-signal comparisons).
calibration-validation-compare.ts (probe reference) now imports canonical
HELD_OUT_TOLERANCES and applies the per-signal logical-byte ratio. Native probe
asserts six signalComparisons in canonical order, each with its own ratio.
R9 hybrid section documents signalComparisons, per-signal scaling, and the
complete-vs-incomplete shape. 309/309 unit; typecheck/lint/format/diff-check clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Local telemetry archives for Maple's embedded store (draft)
Summary
This PR adds long-term, portable Parquet telemetry archives to Maple's local
mode. It exports the six raw telemetry tables (
logs,traces,metrics_sum,metrics_gauge,metrics_histogram,metrics_exponential_histogram) fromimmutable checkpoints into sealed UTC-day Parquet generations, queryable
independently with DuckDB — without reloading history into the live store, adding
a live export endpoint, or running a second always-on database.
It reuses the hardened checkpoint infrastructure from #129: the scoped
restored-checkpoint API, the maintenance lock, and a new persistent pin API that
protects the source checkpoint during export.
What the feature is
maple archive create <range-date> <signal>: seal one UTC day of one signalinto a validated Parquet generation from the current (or a specified)
checkpoint.
maple archive list [--output summary|paths|json]: report active generationswith sizes and paths, or emit machine-readable active Parquet shard paths for
DuckDB.
maple archive rebuild <signal>: rebuild a signal's catalog from manifests.maple archive calibrate: run a candidate matrix against a pinned checkpointand emit a versioned tuning configuration.
Why each major choice was made
Checkpoint-restored scratch, not a live copy
The only proven safe archive source is a native chDB checkpoint restored into
sacrificial scratch. A raw copy of the live data directory captures an
inconsistent on-disk state and races concurrent ingest. Archive export restores
one checkpoint into a private scratch chDB (reusing #129's
withRestoredCheckpoint), exports from it, then removes the scratch. The livestore is never opened for export.
Immutable generations with structural supersession, not TraceId dedup
There is no universal deduplication key across the six raw tables (
TraceIdisshared by many spans, absent from some logs, and nonexistent on metrics). An
archive seals a fixed UTC-day range into an immutable generation; a late-arrival
re-export creates a new generation that supersedes the old through an atomic
active.jsonpointer flip. The old generation is retained on disk but excludedfrom listings and queries. This avoids scanning all generations to dedup and
makes each generation independently reproducible.
One sacrificial chDB for validation and export
Checkpoint validation and archive export share #129's scoped
withRestoredCheckpointinstance, so archive export does not add a secondconcurrent chDB memory term — rotation adds duration and disk I/O to that
scratch working set, not another full in-memory OLAP copy.
Persistent pins inside the maintenance lock
The research prototype left a resolve-vs-GC race open. This PR acquires a
persistent pin on the source checkpoint inside the maintenance lock, so
retention cannot delete the snapshot between resolution and export. A stale pin
(from a crashed export) safely over-retains data rather than risking deletion.
toDate()/toHour() predicates, not toDateTime64 ranges
chDB's bundled ClickHouse miscounts aggregate
count()over atoDateTime64-vs-DateTimepredicate (the per-row comparison is correct butthe aggregate optimizer returns zero). All range predicates use
toDate()/toHour()equality, which normalizes both second-precisionDateTime(
logs.TimestampTime) and nanosecondDateTime64(9)event-time columns.Dependency on #129
This branch is created from #129's final head
178b8c49and consumes:withRestoredCheckpoint— the scoped restored-checkpoint primitive.resolveCheckpoint/readCheckpointState— strict checkpoint resolution.withMaintenanceLock).retireCheckpointIfEligible's existing pin-honoring retention.It adds
acquireCheckpointPin/releaseCheckpointPin/withMaintenanceLockto the checkpoint module. These are small, additive exports; they do not change
existing checkpoint behavior.
Resource and adoption implications
same working set checkpoint validation uses).
over speculative deletion).
archive volume, and memory budget.
Complete happy-path behavior
maple checkpointto create a validated checkpoint.maple archive create 2026-06-01 traces(and the other signals).maple archive list --output paths --signal tracesfor DuckDB-ready paths.read_parquetandunion_by_name=true.Major failure outcomes
Validation evidence
Final local branch head:
5ad77dfbTwo consecutive exact-commit native archive smokes passed. Each run:
A calibration smoke verified the candidate matrix round-trip: four candidates at
~320MB RSS, selected
writerThreads=1, rowGroupRows=20000within a 1GB budgetwith high confidence, and wrote a versioned config recording the measured peak
RSS and effective tuning.
Known limits and follow-up work
generated-config round-trip on a local scratch volume. True external-volume,
deployment-scale calibration under the deployment chDB/user/filesystem is a
Phase 3 dependency.
(
_part,_part_offset) split is a follow-up for very-high-volume signalsthat exceed the per-shard bounds.
after repeated successful runs and measured checkpoint pause.
retained). A safe, ownership-aware GC is future work.
(117 tests). A full child-process kill/restart fault matrix at every durable
boundary, matching local chDB checkpoints for Maple’s embedded chDB store #129's depth, is a follow-up.
Non-goals (v1)