diff --git a/CHANGELOG.md b/CHANGELOG.md index 13d1e0b..e3f5c5e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,33 @@ Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html) --- +## [0.13.0] - 2026-07-04 + +Drift-fixes release. Closes the SDK-side items on `docs/drift.md` (2026-07-04); no on-wire breaking change — backends on `1.0.0` keep working unchanged. + +### Added + +- **Idempotency-key propagation to `/track` v3 single-event** — new `nullrun.context._server_minted_idempotency_key_var` + `get_/set_/reset_/clear_server_minted_idempotency_key` helpers. `_capture_server_minted_execution_id` now also reads `response["operation_id"]` (which equals the `/check` `idempotency_key` per `runtime.py:1260`); `_enrich_event` stamps it onto `wire_event` for `llm_call`; `_build_v3_track_payload` propagates it onto the v3 `/track` body with a contextvar fallback for tests and direct callers. Without this, transport-level retry on the same event either 503'd with `RESERVATION_NOT_FOUND` (reservation key DEL'd after first consume per CLAUDE.md §25) or double-billed the underlying budget. + +### Changed + +- `runtime.py` module docstring now distinguishes **SDK-side transport failure** (network / 5xx / breaker open → fail-OPEN on the `/check` path) from **wire 4xx/5xx that names an enforcement failure** (`BUDGET_REDIS_UNAVAILABLE` → 402 fail-CLOSED, `RATE_LIMIT_REDIS_UNAVAILABLE` → 503 fail-CLOSED). The previous README claim "Fail-OPEN na infrastructure failures" was conflating the two — the SDK code is now correctly documented in the docstring; the README rewrite is tracked under `drift.md` P0-1 (deferred to a separate doc PR). + +### Fixed + +- **Wire `status_code` preserved on every decision exception** — `NullRunBlockedException`, `NullRunBudgetError`, `NullRunChainError`, `NullRunWorkflowInactiveError`, `NullRunConsumeOverbudgetError` now all accept `status_code: int | None = None`. `_parse_v3_error_envelope` populates it from `response.status_code` for every branch (402 budget, 403 workflow/chain cross-org, 422 `CONSUME_OVERBUDGET`, 503 `RATE_LIMIT_REDIS_UNAVAILABLE`, ...). FastAPI exception handlers reading `exc.status_code` previously got `None` / 500 for budget blocks because the backend's 402 was lost in the constructor chain. +- **Patch-coverage gap from 0.12.2 closed** — `tests/test_v3_wire_contract.py::TestGateCacheRuntimeFlow` (3 tests) drives `NullRunRuntime.check_workflow_budget` inside `with chain(...)` and exercises the `cache_enabled` / cache-hit / cache-miss / cache-bypass-via-env branches in `runtime.py:1287-1310` that were previously uncovered (was dragging codecov/patch below the 70% floor on PR #52). + +### Tests + +- `tests/test_drift_fixes_2026_07_04.py` — 15 new tests: 5 idempotency-key contextvar lifecycle + payload-shape, 8 status_code on every decision exception, 2 fail-CLOSED on wire 503 `RATE_LIMIT_REDIS_UNAVAILABLE`. All pass on the 0.13.0 source. +- `tests/test_v3_wire_contract.py::TestGateCacheRuntimeFlow` — 3 runtime-level chain-mode cache tests as described above. + +### Audit + +- New `docs/drift.md` records the six P0 + P1 items that turned up during pre-publish review of 0.12.2 (idempotency-key wiring, status_code on exceptions, fail-CLOSED honesty, plus four P0/P1 README issues that are deferred to a README rewrite PR and explicitly NOT in this release). + + ## [0.12.2] - 2026-07-04 Bug-fix release. Two related correctness fixes layered on top of 0.12.1; no wire-format change. diff --git a/docs/drift.md b/docs/drift.md new file mode 100644 index 0000000..c38530d --- /dev/null +++ b/docs/drift.md @@ -0,0 +1,74 @@ +# SDK 0.12.2 → 0.13.0 drift audit (2026-07-04) + +This document records the drift that was discovered between the +**documented** behaviour in `SDK_README.md` / `CLAUDE.md` and the +**actual** runtime behaviour surfaced during pre-publish manual +review of 0.12.2. It is the companion audit to `sdk-v3-migration-gaps.md` +(which captured the earlier `execution_id` propagation gaps). + +**Scope.** SDK-side code only. Backend drift is owned by the +`nullrun-backend` repo under a separate audit log. PR review caught +the bulk of items; runtime tests + manual repro surfaced the rest. + +**Why a document, not a CHANGELOG.** The CHANGELOG entry describes +what shipped in 0.13.0; this document is the trail of breadcrumbs so +the next person who reads the SDK can see *why* each item was filed +the way it was filed, and which items are deferred. + +--- + +## Severity model + +| Tier | Definition | Disposition | +|---|---|---| +| **P0** | Bug-shape — backend actually returns one status, SDK reports another, *or* a wire contract documented in CLAUDE.md is silently violated on a happy path. | Fix immediately, ship in next patch release. | +| **P1** | Wire-contract honour that the SDK cheats on — the *intent* is documented, the code path doesn't quite get there. | Fix in current minor release (0.13.0). | +| **P2** | Cosmetic / docs-only — README claims feature X, code does X but README says it does Y; no user-visible regression. | Defer to README rewrite PR; do not block release. | +| **Q?** | Open question — agreed intent but ambiguous wire spec, owner undecided. | Track; resolve before next wire spec bump. | + +--- + +## Findings (closed in 0.13.0) + +| ID | Tier | Surface | Description | Fix in 0.13.0 | +|---|---|---|---|---| +| **P1-5 + open Q4** | P1 | `/track` v3 single-event (`Transport.track_single`) | `idempotency_key` not wired onto the v3 /track payload. Without it, transport-level retry on the SAME event either (a) re-runs `CONSUME_SCRIPT` → 503 `RESERVATION_NOT_FOUND` since the reservation key is DEL'd after the first consume per CLAUDE.md §25, or (b) double-bills the underlying budget. | New contextvar `get_server_minted_idempotency_key` + symmetric `set_/reset_/clear_`; `_capture_server_minted_execution_id` also reads `response["operation_id"]` (which equals the /check `idempotency_key`, runtime.py:1260); `_enrich_event` stamps it onto `wire_event` for `llm_call`; `_build_v3_track_payload` propagates onto the v3 /track payload (contextvar fallback for tests / direct callers). | +| **P1-1** | P1 | Decision exception class hierarchy | `NullRunBlockedException` / `NullRunBudgetError` / `NullRunChainError` / `NullRunWorkflowInactiveError` / `NullRunConsumeOverbudgetError` did not accept `status_code`. `_parse_v3_error_envelope` had no place to put the wire `response.status_code`. FastAPI exception handlers reading `exc.status_code` previously got `None` / 500 for budget blocks (the backend's 402 was lost in the constructor chain). | All five exception classes now accept `status_code: int \| None = None`; `_parse_v3_error_envelope` populates it from `response.status_code` for every branch — 402 budget, 403 workflow/chain cross-org, 422 `CONSUME_OVERBUDGET`, 503 `RATE_LIMIT_REDIS_UNAVAILABLE`, etc. | +| **P1-2** | P1 | `runtime.py` module docstring | The README claim "Fail-OPEN on infrastructure failures" was half-wrong — it conflated SDK-side transport failure (network/5xx/breaker open → fail-OPEN on the /check path per `check_workflow_budget`) with wire 4xx/5xx that names an *enforcement* failure (`BUDGET_REDIS_UNAVAILABLE` → 402 fail-CLOSED; `RATE_LIMIT_REDIS_UNAVAILABLE` → 503 fail-CLOSED). The two paths must be distinguished. | `runtime.py` top-of-file docstring now carries a table distinguishing (a) SDK-side transport failure → fail-OPEN, from (b) wire 4xx/5xx that names an enforcement failure → fail-CLOSED with the matching status code. | + +--- + +## Findings (deferred — docs only) + +| ID | Tier | Surface | Description | Why deferred | +|---|---|---|---|---| +| **P0-1** | P0 → DOCS | `SDK_README.md` §"Error handling" | README claim "SDK falls back to allow on any transport error" is a *partial* misstatement. The truth is in the runtime.py docstring table added by P1-2 above. | SDK code is now correct (P1-2); the README rewrite is a documentation PR of its own. Blocked on its own unrelated doc PR. | +| **P0-2** | P0 → DOCS | `SDK_README.md` §"Budget enforcement" | README does not mention the new `_GATE_CACHE` 5s in-process TTL chain-mode debounce (added in 0.12.2). Readers who instrument chain-mode calls will be confused about why they see only 1 /gate roundtrip per 100 calls. | Same as P0-1 — doc PR. | +| **P0-3** | P0 → DOCS | `SDK_README.md` §"Wire contract" | README still references the pre-0.11.0 `/api/v1/execute` endpoint and ignores the v3 single-event `/api/v1/track` path entirely. | Same as P0-1 — doc PR. | +| **P0-4** | P0 → DOCS | `SDK_README.md` §"Idempotency" | README does not document the new `idempotency_key` field on /track added by P1-5 above. | Same as P0-1 — doc PR. | +| **P1-3** | P1 → DOCS | `SDK_README.md` §"Circuit breaker" | Lists open-vs-half-open states but does not mention the `NULLRUN_GATE_CACHE_DISABLE` opt-out for the chain-mode cache. | Same as P0-1 — doc PR. | +| **P1-4** | P1 → DOCS | `SDK_README.md` §"Status codes" | Does not enumerate the 402 / 403 / 422 / 503 codes that decision exceptions now carry post-P1-1. | Same as P0-1 — doc PR. | + +--- + +## Tests added (`tests/test_drift_fixes_2026_07_04.py`) + +- **F1 / P1-5 + Q4** — 5 tests pinning the idempotency-key contextvar lifecycle (`get_/set_/reset_/clear_` × payload-shape assertion) + v3 /track wire propagation +- **F2 / P1-1** — 8 tests pinning that `status_code` survives the constructor chain for every decision exception class +- **F3 / P1-2** — 2 tests pinning that wire `RATE_LIMIT_REDIS_UNAVAILABLE` → 503 is classified as fail-CLOSED on the runtime side (i.e. caller raises, does *not* fall through to the SDK-side fail-OPEN transport-error handler) + +## Patch coverage follow-up + +0.13.0 also closes the 0.12.2 patch-coverage gap that previously dragged +codecov/patch below the 70% floor. New `TestGateCacheRuntimeFlow` class +in `tests/test_v3_wire_contract.py` drives `NullRunRuntime.check_workflow_budget` +inside `with chain(...)` and exercises the cache_enabled / cache-hit / +cache-miss / cache-bypass branches that were previously uncovered +(`runtime.py:1287-1310`). + +## Cross-references + +- `docs/sdk-v3-migration-gaps.md` — earlier audit that motivated 0.12.1 +- `CHANGELOG.md` 0.13.0 entry — release-facing summary of F1/F2/F3 +- `CLAUDE.md` §25 — wire contract for /track consume-vs-reserve +- `CLAUDE.md` §33 — fail-CLOSED exceptions and corresponding wire codes diff --git a/pyproject.toml b/pyproject.toml index 82c22f0..eb99f7d 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -4,12 +4,12 @@ build-backend = "hatchling.build" [project] name = "nullrun" -# Version bump: 0.12.1 → 0.12.2 in `release(0.12.2)` (fresh -# execution_id per /check + in-process chain-mode gate cache). The -# runtime commit bumped `src/nullrun/__version__.py` together with -# this field — same drift prevention as #50, but proactive this -# time (caught during pre-merge audit, not after a publish error). -version = "0.12.2" +# Version bump: 0.12.2 → 0.13.0 in `release(0.13.0)` (drift-fixes +# release: idempotency_key on /track + status_code on every +# decision exception + fail-CLOSED/OPEN honesty in module docstring). +# No on-wire breaking change; backends on 1.0.0 keep working +# unchanged. See docs/drift.md for the full audit trail. +version = "0.13.0" # Long form used by PyPI page meta-description and search snippets. # Kept under the 200-char preview threshold so the full line is visible # without an "expand" click. Keywords are matched against likely search diff --git a/src/nullrun/__version__.py b/src/nullrun/__version__.py index fe45207..eacffdb 100644 --- a/src/nullrun/__version__.py +++ b/src/nullrun/__version__.py @@ -81,7 +81,66 @@ 1.0.0 keep working unchanged. Pinning unchanged: SDK_MIN_VERSION_FOR_V3 = "0.12.0". Recommended upgrade path: 0.12.1 -> 0.12.2. + +--- + +v3.13 / 0.13.0 (2026-07-04) — drift-fixes release: closes the SDK-side +items left over from the docs-vs-code audit captured in +`docs/drift.md`. + + 1. ``idempotency_key`` wired onto the v3 /track single-event + payload. New contextvar + ``nullrun.context._server_minted_idempotency_key_var`` + + ``get_/set_/reset_/clear_server_minted_idempotency_key``; + ``_capture_server_minted_execution_id`` now also captures + ``response["operation_id"]`` (which equals the /check + idempotency_key, runtime.py:1260); ``_enrich_event`` stamps + the value onto the ``wire_event`` for ``llm_call``; + ``_build_v3_track_payload`` propagates it onto the v3 /track + body with a contextvar fallback for tests + direct callers. + Without this, transport-level retry on the same event either + 503'd with ``RESERVATION_NOT_FOUND`` (reservation key DEL'd + after the first consume per CLAUDE.md §25) or double-billed + the underlying budget. + + 2. Wire ``status_code`` preserved through every decision + exception class. ``NullRunBlockedException``, + ``NullRunBudgetError``, ``NullRunChainError``, + ``NullRunWorkflowInactiveError``, + ``NullRunConsumeOverbudgetError`` now all accept + ``status_code: int | None = None``; ``_parse_v3_error_envelope`` + sets it from ``response.status_code`` for every branch — + 402 budget, 403 workflow/chain cross-org, 422 + ``CONSUME_OVERBUDGET``, 503 ``RATE_LIMIT_REDIS_UNAVAILABLE``, + etc. FastAPI exception handlers reading ``exc.status_code`` + previously got ``None`` / 500 for budget blocks (the backend's + 402 was lost in the constructor chain). + + 3. The runtime.py module docstring now distinguishes + SDK-side transport failure (network/5xx/breaker open → + fail-OPEN on /check) from wire 4xx/5xx that names an + enforcement failure (``BUDGET_REDIS_UNAVAILABLE`` → 402 + fail-CLOSED; ``RATE_LIMIT_REDIS_UNAVAILABLE`` → 503 + fail-CLOSED). The README had conflated the two with a single + "fail-OPEN on infra failures" claim. + +Tests: + * ``tests/test_drift_fixes_2026_07_04.py`` — 15 tests (5 idempotency, + 8 status_code on every decision exception, 2 fail-CLOSED on + wire 503 RATE_LIMIT_REDIS_UNAVAILABLE). + * ``tests/test_v3_wire_contract.py::TestGateCacheRuntimeFlow`` — 3 + runtime-level chain-mode cache tests that close the 0.12.2 + patch-coverage gap (dragged codecov/patch below the 70% floor + on PR #52). Drives ``NullRunRuntime.check_workflow_budget`` + inside ``with workflow(...) + with chain(...)`` to exercise + cache_enabled / cache-hit / cache-miss / + cache-bypass-via-env branches (runtime.py:1287-1310). + +Backends on 1.0.0 keep working unchanged. Pinning unchanged: +SDK_MIN_VERSION_FOR_V3 = "0.12.0". Recommended upgrade +path: 0.12.2 -> 0.13.0 (no on-wire breaking change; the SDK +will pick up the new idempotency_key stamping automatically). """ -__version__ = "0.12.2" +__version__ = "0.13.0" __platform_version__ = "1.0.0" diff --git a/src/nullrun/breaker/exceptions.py b/src/nullrun/breaker/exceptions.py index c36e96a..d01e49e 100644 --- a/src/nullrun/breaker/exceptions.py +++ b/src/nullrun/breaker/exceptions.py @@ -426,11 +426,17 @@ def __init__( chain_id: str | None = None, backend_code: str | None = None, details: dict[str, Any] | None = None, + status_code: int | None = None, **kwargs: Any, ) -> None: self.chain_id = chain_id self.backend_code = backend_code or self.error_code self.details = details or {} + # 2026-07-04 (drift.md P1-1): preserve the wire HTTP + # status. Chain errors map to 402/403/404 depending on + # the specific code — FastAPI handlers reading + # ``exc.status_code`` should see the right one. + self.status_code = status_code super().__init__(message, **kwargs) @@ -476,6 +482,7 @@ def __init__( max_allowed_cents: int | None = None, actual_cost_cents: int | None = None, epsilon_cents: int | None = None, + status_code: int | None = None, **kwargs: Any, ) -> None: self.execution_id = execution_id @@ -483,6 +490,10 @@ def __init__( self.max_allowed_cents = max_allowed_cents self.actual_cost_cents = actual_cost_cents self.epsilon_cents = epsilon_cents + # 2026-07-04 (drift.md P1-1): CONSUME_OVERBUDGET maps to + # 422 on the wire (CLAUDE.md §25) — surface it so FastAPI + # handlers don't fall back to 500. + self.status_code = status_code super().__init__(message, **kwargs) @@ -514,9 +525,14 @@ def __init__( message: str, *, workflow_id: str | None = None, + status_code: int | None = None, **kwargs: Any, ) -> None: self.workflow_id = workflow_id + # 2026-07-04 (drift.md P1-1): WORKFLOW_INACTIVE maps to + # 403 on the wire (CLAUDE.md §13) — surface it so FastAPI + # handlers don't fall back to 500. + self.status_code = status_code super().__init__(message, **kwargs) @@ -687,6 +703,18 @@ class NullRunBlockedException(NullRunDecision): ``None`` when the block is workflow-scoped rather than tool-scoped. details: Free-form structured payload forwarded by the caller. + status_code: HTTP status code the backend sent on the wire + when the block was derived from a server response (e.g. + 402 for ``BUDGET_HARD_BLOCKED``, 403 for + ``TOOL_BLOCKED`` / ``WORKFLOW_INACTIVE``, 429 for + ``RATE_LIMIT_EXCEEDED``). ``None`` for client-side + blocks (sensitive-tool pre-check, loop detection, retry + storm) where there was no wire response. Lets FastAPI / + Starlette exception handlers map to the correct HTTP + status without re-deriving it from + ``type(exc).__name__``. Drift fix 2026-07-04 + (drift.md P1-1: SDK_README's NR-B004 → 429 claim was + wrong; the real wire status is 402). """ error_code = "NR-X001" # generic block; subclasses override @@ -703,12 +731,19 @@ def __init__( reason: str, action: str = "block", tool_name: str | None = None, + status_code: int | None = None, **details: Any, ) -> None: self.workflow_id = workflow_id self.reason = reason self.action = action self.tool_name = tool_name + # 2026-07-04 (drift.md P1-1): wire HTTP status preserved + # so FastAPI exception handlers can return the correct + # status without re-deriving from the error class. ``None`` + # when the block fired client-side (loop detection, retry + # storm, sensitive-tool pre-check). + self.status_code = status_code self.details = details tool_suffix = f", tool={tool_name}" if tool_name else "" # ``code`` / ``user_action`` / ``retryable`` can be overridden @@ -722,7 +757,7 @@ def __init__( retryable = self.retryable super().__init__( f"Workflow {workflow_id} blocked: {reason} " - f"(action={action}{tool_suffix}, details={details})", + f"(action={action}{tool_suffix}, status_code={status_code}, details={details})", error_code=error_code, user_action=user_action, retryable=retryable, diff --git a/src/nullrun/context.py b/src/nullrun/context.py index ca5ff40..78c1f15 100644 --- a/src/nullrun/context.py +++ b/src/nullrun/context.py @@ -207,6 +207,22 @@ def set_chain_op(op: str) -> None: _server_minted_reservation_at_var: ContextVar[float] = ContextVar( "server_minted_reservation_at", default=0.0 ) +# 2026-07-04 (drift.md P1-5): /track idempotency anchor. +# The /check request carries ``idempotency_key = operation_id`` (UUID v4); +# the backend's /track handler (handlers.rs:4654-4725) accepts the same +# key and replays the original response on hit (200 + ``idempotent_replay: +# true``). Without forwarding the key from /check onto the /track payload, +# a transport-level retry on the SAME event either re-runs CONSUME_SCRIPT +# (→ 503 RESERVATION_NOT_FOUND, since the reservation key was DEL'ed by +# the first successful consume per §25) or double-bills. +# +# Captured into a contextvar at the same instant as +# ``server_minted_execution_id`` so the two values always refer to the +# same /check. ``None`` when the /check didn't supply one (legacy or +# capability-disabled backend) — the /track payload then omits the field. +_server_minted_idempotency_key_var: ContextVar[str | None] = ContextVar( + "server_minted_idempotency_key", default=None +) def get_server_minted_execution_id() -> str | None: @@ -234,6 +250,22 @@ def get_server_minted_reservation_at() -> float: return _server_minted_reservation_at_var.get() +def get_server_minted_idempotency_key() -> str | None: + """Return the /check ``idempotency_key`` for the in-scope + reservation, or ``None`` if none captured. + + Read by ``NullRunRuntime._enrich_event`` to tag the /track + v3 single-event payload. The /check request sets + ``idempotency_key = operation_id`` (a UUID v4) at + runtime.py:1260; the /track handler honors it for replay + per CLAUDE.md §23. + + Pairs with :func:`get_server_minted_execution_id` and shares + the same capture token; ``None`` on the legacy v1/v2 path. + """ + return _server_minted_idempotency_key_var.get() + + def set_server_minted_execution_id(value: str | None) -> Token[str | None]: """Capture the server-minted execution_id returned by /check. @@ -263,6 +295,19 @@ def set_server_minted_reservation_at(value: float) -> Token[float]: return _server_minted_reservation_at_var.set(value) +def set_server_minted_idempotency_key(value: str | None) -> Token[str | None]: + """Capture the /check ``idempotency_key`` (the operation_id UUID v4 + on the v3 path) alongside the matching execution_id. + + Lifetime is symmetric with + :func:`set_server_minted_execution_id` — the runtime captures + both at the same instant and resets both at the matching + /track emission (or workflow/chain block exit). Returns the + matching Token. + """ + return _server_minted_idempotency_key_var.set(value) + + def reset_server_minted_execution_id(token: Token[str | None]) -> None: """Restore the previous server-minted execution_id value. @@ -282,6 +327,14 @@ def reset_server_minted_reservation_at(token: Token[float]) -> None: _server_minted_reservation_at_var.reset(token) +def reset_server_minted_idempotency_key(token: Token[str | None]) -> None: + """Restore the previous /check idempotency_key value. + + Pair with :func:`set_server_minted_idempotency_key`. + """ + _server_minted_idempotency_key_var.reset(token) + + def clear_server_minted_execution_id() -> None: """Erase the captured server-minted execution_id + timestamp. @@ -290,6 +343,7 @@ def clear_server_minted_execution_id() -> None: _server_minted_execution_id_var.set(None) _server_minted_reservation_at_var.set(0.0) + _server_minted_idempotency_key_var.set(None) Use :func:`reset_server_minted_execution_id` instead when you have a Token to consume — that path restores the previous @@ -297,6 +351,7 @@ def clear_server_minted_execution_id() -> None: """ _server_minted_execution_id_var.set(None) _server_minted_reservation_at_var.set(0.0) + _server_minted_idempotency_key_var.set(None) def set_attempt_index(index: int) -> None: diff --git a/src/nullrun/runtime.py b/src/nullrun/runtime.py index e62fc8d..fc3ba2f 100644 --- a/src/nullrun/runtime.py +++ b/src/nullrun/runtime.py @@ -22,6 +22,31 @@ | `_enforce_sensitive_tool` (default `_fallback_mode=permissive`) | CLOSED -- body MUST NOT run when `decision_source` is any `FALLBACK_*` | n/a (body did not run) | `NULLRUN_SENSITIVE_FAIL_OPEN=1` -- explicitly documented as "OPEN-when-engine-unavailable" | | `_enforce_sensitive_tool` (`_fallback_mode=strict`) | CLOSED -- transport returns `decision=block, decision_source=FALLBACK_*` | n/a | none | | `_emit_span_start` / `_emit_span_end` | n/a -- never blocks | n/a | n/a | +| `/track` batch path (legacy) | OPEN-on-network-error (event dropped, no retry) | n/a -- circuit breaker backoff applies | none | + +**Drift fix 2026-07-04 (drift.md P1-2):** the SDK_README.md claim +"Fail-OPEN на инфраструктурных сбоях. Если backend недоступен, бюджет +не блокирует агента" is **partially wrong** — it conflates SDK-side +transport failure with backend-side budget-enforcement failure. The +honest split is: + +* **SDK-side transport failure** (network timeout, 5xx, breaker open) + → fail-OPEN on the *check* path so a dead backend doesn't freeze + the user's agent loop (this is what the README describes). +* **Backend-side budget-enforcement failure** (the /gate or /track + handler actually returned a wire response, just one indicating a + Redis outage or aggregate rate-limit Redis unavailable) → the + wire response is what it is, and the SDK raises the corresponding + exception. ``BUDGET_REDIS_UNAVAILABLE`` → 402 ``NullRunBudgetError`` + (fail-CLOSED, the backend rejected the request because Redis was + unreachable for the budget counter — this is the authoritative + enforcement signal, not a transport blip). ``RATE_LIMIT_REDIS_UNAVAILABLE`` + → 503 ``NullRunRateLimitRedisError`` (fail-CLOSED for the same + reason). The SDK does NOT silently fall-OPEN on a wire 4xx/5xx + that names an enforcement failure. + +The table above is authoritative; if any of these change, the +README claim must be updated in lockstep. The "Opt-out" column makes it explicit that `NULLRUN_SKIP_BUDGET_CHECK=1` is a **different category** of action than @@ -2179,6 +2204,26 @@ def _enrich_event(self, event: dict[str, Any]) -> dict[str, Any]: else: enriched["execution_id"] = smid + # 2026-07-04 (drift.md P1-5): propagate the in-scope + # /check idempotency_key onto the wire_event so the v3 + # /track single-event payload carries the same anchor and + # the backend's replay branch returns 200 + + # ``idempotent_replay: true`` on retry (handlers.rs: + # 4654-4725). Without this, a transport-level retry on the + # SAME event either re-runs CONSUME_SCRIPT (→ 503 + # RESERVATION_NOT_FOUND, since the reservation key was + # DEL'ed after the first successful consume per §25) or + # double-bills. Read via the same contextvar written at + # ``_capture_server_minted_execution_id`` time — symmetric + # lifetime with ``execution_id`` (cleared together on + # /track emit and on workflow/chain block exit). + if "idempotency_key" not in enriched: + from nullrun.context import get_server_minted_idempotency_key + + idem_key = get_server_minted_idempotency_key() + if idem_key: + enriched["idempotency_key"] = idem_key + # Add type if not present if "type" not in enriched: enriched["type"] = "event" @@ -2547,6 +2592,7 @@ def _capture_server_minted_execution_id(response: dict[str, Any]) -> str | None: from nullrun.context import ( clear_server_minted_execution_id, set_server_minted_execution_id, + set_server_minted_idempotency_key, set_server_minted_reservation_at, ) @@ -2586,6 +2632,18 @@ def _capture_server_minted_execution_id(response: dict[str, Any]) -> str | None: set_server_minted_execution_id(raw) set_server_minted_reservation_at(_time.monotonic()) + # 2026-07-04 (drift.md P1-5): capture the /check + # idempotency_key so the matching /track event can carry the + # same anchor (handlers.rs:4654-4725 — replay returns 200 + + # idempotent_replay: true on key hit). We look at the + # request body via the response's ``operation_id`` field + # when the server echoes it (the /check request sets + # ``idempotency_key = operation_id`` at runtime.py:1260); + # when absent, fall back to None and let the /track wire + # payload drop the field. + op_id = response.get("operation_id") if isinstance(response, dict) else None + if isinstance(op_id, str) and op_id: + set_server_minted_idempotency_key(op_id) logger.debug( "_capture_server_minted_execution_id: captured %s", raw, @@ -2687,6 +2745,34 @@ def _build_v3_track_payload( if k in wire_event and wire_event[k] is not None: payload[k] = wire_event[k] + # Wire idempotency_key (CLAUDE.md §23, drift.md P1-5): the + # backend's /track handler (``handlers.rs:4654-4725``) accepts + # ``idempotency_key: Option`` and, on hit of the same + # key, replays the original response with 200 OK + + # ``idempotent_replay: true``. Without this, a transport-level + # retry (5xx, timeout) on the SAME event would re-call the v3 + # CONSUME_SCRIPT and either double-bill or get 503 + # ``RESERVATION_NOT_FOUND`` (because the reservation key was + # DEL'ed after the first successful consume per §25). + # + # Source of truth: ``check_req.idempotency_key`` (set in + # ``check_workflow_budget`` to the operation_id UUID v4, see + # runtime.py:1260) is captured into a contextvar by + # ``_capture_server_minted_execution_id`` and stamped onto the + # wire_event by ``_enrich_event``. We accept EITHER source — + # ``wire_event`` takes precedence (explicit caller override), + # then the contextvar fallback (covers tests / flows that call + # ``_build_v3_track_payload`` directly without going through + # ``_enrich_event``). When both are absent, omit the field and + # the backend falls back to ``execution_id`` only. + idem_key = wire_event.get("idempotency_key") + if not idem_key: + from nullrun.context import get_server_minted_idempotency_key + + idem_key = get_server_minted_idempotency_key() + if idem_key: + payload["idempotency_key"] = str(idem_key) + return payload diff --git a/src/nullrun/transport.py b/src/nullrun/transport.py index 9308219..fe3dc91 100644 --- a/src/nullrun/transport.py +++ b/src/nullrun/transport.py @@ -2167,6 +2167,7 @@ def _parse_v3_error_envelope( max_allowed_cents=details.get("max_allowed_cents"), actual_cost_cents=details.get("actual_cost_cents"), epsilon_cents=details.get("epsilon_cents"), + status_code=status, # 422 per backend mapping ) if backend_code == "CHAIN_MAX_DURATION_EXCEEDED" or backend_code == "CHAIN_CROSS_ORG" or backend_code == "CHAIN_ORG_MISMATCH": @@ -2175,12 +2176,14 @@ def _parse_v3_error_envelope( chain_id=details.get("chain_id"), backend_code=backend_code, details=details, + status_code=status, # 402/403 per backend mapping ) if backend_code == "WORKFLOW_INACTIVE": return NullRunWorkflowInactiveError( full_message, workflow_id=details.get("workflow_id"), + status_code=status, # 403 per backend mapping ) if backend_code == "RATE_LIMIT_REDIS_UNAVAILABLE": @@ -2223,9 +2226,19 @@ def _parse_v3_error_envelope( # workflow_id (str) + reason (str) positional args. Use # the workflow_id / reason from the envelope details if # present, otherwise synthesise from the endpoint label. + # + # 2026-07-04 (drift.md P1-1): forward the wire HTTP + # status so FastAPI exception handlers reading + # ``exc.status_code`` get 402 for BUDGET_HARD_BLOCKED + # (not None / 500). The backend maps each budget + # error_code to a specific HTTP status (error_codes.rs + # 189-233), but the only signal a transport caller + # has is ``response.status_code`` — we propagate it + # here so the exception is self-describing. return NullRunBudgetError( workflow_id=str(details.get("workflow_id") or "unknown"), reason=full_message, + status_code=status, ) if catalog is NullRunRateLimitRedisError: # NullRunError base takes (message, error_code=, user_action=, diff --git a/tests/test_drift_fixes_2026_07_04.py b/tests/test_drift_fixes_2026_07_04.py new file mode 100644 index 0000000..b07c453 --- /dev/null +++ b/tests/test_drift_fixes_2026_07_04.py @@ -0,0 +1,357 @@ +""" +Contract tests for the drift.md 2026-07-04 fixes. + +Background +---------- +drift.md (NULLRUN/drift.md, 2026-07-04) flagged three real +SDK gaps whose wire effect was observable to customers: + + F1 (drift.md P1-5) / open Q4: /track v3 single-event + payload did NOT carry a wire ``idempotency_key``. Backend + (handlers.rs:4654-4725) supports replay on hit, but + without the field the SDK's transport-level retry either + re-ran CONSUME_SCRIPT (→ 503 ``RESERVATION_NOT_FOUND``) + or double-billed. Fix: ``_capture_server_minted_execution_id`` + now captures ``operation_id`` from the /check response + into a contextvar (``get_server_minted_idempotency_key``), + ``_enrich_event`` stamps it on the wire_event, and + ``_build_v3_track_payload`` propagates it onto the v3 + /track payload. + + F2 (drift.md P1-1): NR-B004 → 402 not 429. The wire envelope + parser preserved the HTTP status on ``NullRunBackendError`` + but not on ``NullRunBudgetError`` / + ``NullRunWorkflowInactiveError`` / + ``NullRunChainError`` / + ``NullRunConsumeOverbudgetError``. FastAPI exception + handlers reading ``exc.status_code`` would fall back to 500 + (or None). Fix: each class now accepts ``status_code`` and + ``_parse_v3_error_envelope`` populates it from + ``response.status_code``. + + F3 (drift.md P1-2): SDK_README "Fail-OPEN на инфраструктурных + сбоях" is half-wrong. The honest split (now in the + runtime module-top docstring): + * SDK-side transport error (network/5xx/breaker open): + /check path is fail-OPEN, /track legacy path drops. + * Wire 4xx/5xx that names an enforcement failure + (``BUDGET_REDIS_UNAVAILABLE``, ``RATE_LIMIT_REDIS_UNAVAILABLE``): + fail-CLOSED on the SDK side — the exception is + raised exactly as the backend returned it. + +This file pins each fix with focused unit tests so future +refactors trip CI rather than silently re-introducing the +drift. +""" + +from __future__ import annotations + +import json +from unittest.mock import patch + +import pytest +import respx +from httpx import Response + +from nullrun import context as nullrun_context +from nullrun.breaker.exceptions import ( + NullRunBudgetError, + NullRunChainError, + NullRunConsumeOverbudgetError, + NullRunWorkflowInactiveError, +) + + +# --------------------------------------------------------------------------- +# F1: wire idempotency_key propagation +# --------------------------------------------------------------------------- + +class TestIdempotencyKeyOnTrackPayload: + """F1 (drift.md P1-5): /track v3 single-event carries the + /check operation_id as the wire ``idempotency_key`` so the + backend's replay branch returns 200 + ``idempotent_replay: + true`` on hit. + """ + + def setup_method(self) -> None: + # Defensive: clear any leftover capture between tests so + # assertions aren't poisoned by an earlier /check mock. + nullrun_context.clear_server_minted_execution_id() + + def teardown_method(self) -> None: + nullrun_context.clear_server_minted_execution_id() + + def test_idempotency_key_captured_from_check_response(self): + """``_capture_server_minted_execution_id`` should now also + read ``response["operation_id"]`` and store it via + ``set_server_minted_idempotency_key``. + """ + from nullrun.runtime import _capture_server_minted_execution_id + + captured = _capture_server_minted_execution_id( + { + "reservation_id": "01926e7a-3b3b-7ddd-9bdd-7f0d3b3b7b3b", + "operation_id": "11111111-2222-3333-4444-555555555555", + } + ) + + assert captured == "01926e7a-3b3b-7ddd-9bdd-7f0d3b3b7b3b" + assert ( + nullrun_context.get_server_minted_idempotency_key() + == "11111111-2222-3333-4444-555555555555" + ) + + def test_idempotency_key_missing_when_operation_id_absent(self): + """Backward compat: legacy /check responses without + ``operation_id`` should leave the contextvar at None — + ``_build_v3_track_payload`` then omits the field on the + wire. + """ + from nullrun.runtime import _capture_server_minted_execution_id + + _capture_server_minted_execution_id( + { + "reservation_id": "01926e7a-3b3b-7ddd-9bdd-7f0d3b3b7b3b", + } + ) + + assert nullrun_context.get_server_minted_execution_id() is not None + assert nullrun_context.get_server_minted_idempotency_key() is None + + def test_clear_drops_idempotency_key(self): + """``clear_server_minted_execution_id`` must also clear the + idempotency_key (symmetric lifetime — drift.md P1-5). + """ + from nullrun.runtime import _capture_server_minted_execution_id + + _capture_server_minted_execution_id( + { + "reservation_id": "01926e7a-3b3b-7ddd-9bdd-7f0d3b3b7b3b", + "operation_id": "abcdef00-0000-0000-0000-000000000000", + } + ) + assert nullrun_context.get_server_minted_idempotency_key() is not None + + nullrun_context.clear_server_minted_execution_id() + assert nullrun_context.get_server_minted_idempotency_key() is None + + def test_build_v3_track_payload_includes_idempotency_key(self): + """The v3 /track payload mapper must surface the captured + idempotency_key on the wire_event so /track can carry the + same anchor as the matching /check. + """ + from nullrun.runtime import _build_v3_track_payload + + nullrun_context._server_minted_idempotency_key_var.set( + "11111111-2222-3333-4444-555555555555" + ) + + try: + payload = _build_v3_track_payload( + { + "workflow_id": "wf-123", + "tokens": 100, + "model": "claude-sonnet-4-6", + }, + "01926e7a-3b3b-7ddd-9bdd-7f0d3b3b7b3b", + ) + finally: + nullrun_context.clear_server_minted_execution_id() + + assert payload is not None + assert ( + payload["idempotency_key"] + == "11111111-2222-3333-4444-555555555555" + ) + # Sanity: the rest of the v3 payload shape is preserved. + assert ( + payload["reservation_id"] + == "01926e7a-3b3b-7ddd-9bdd-7f0d3b3b7b3b" + ) + assert payload["workflow_id"] == "wf-123" + assert payload["tokens"] == 100 + + def test_build_v3_track_payload_omits_idempotency_key_when_absent( + self, + ): + """Backward compat: when no /check ran (legacy / track-by-batch + fall-through), the field must be absent (not an empty + string — that would set a stale anchor on the backend). + """ + from nullrun.runtime import _build_v3_track_payload + + nullrun_context._server_minted_idempotency_key_var.set(None) + + payload = _build_v3_track_payload( + { + "workflow_id": "wf-123", + "tokens": 100, + }, + "01926e7a-3b3b-7ddd-9bdd-7f0d3b3b7b3b", + ) + + assert payload is not None + assert "idempotency_key" not in payload + + +# --------------------------------------------------------------------------- +# F2: HTTP status_code on every decision exception +# --------------------------------------------------------------------------- + +class TestStatusCodeOnExceptions: + """F2 (drift.md P1-1): the wire envelope parser preserves + ``response.status_code`` on every decision exception so FastAPI + exception handlers reading ``exc.status_code`` don't fall back + to 500. + """ + + def _build_envelope(self, error_code: str, body_extra: dict | None = None) -> dict: + body: dict = { + "error_code": error_code, + "error_message": f"synthetic {error_code}", + "details": body_extra or {"workflow_id": "wf-123"}, + "retry_after_ms": None, + } + return body + + def _raise_via_parser( + self, error_code: str, status: int, body_extra: dict | None = None + ): + """Drive ``_parse_v3_error_envelope`` through a synthetic + httpx.Response — the real path the transport uses. + """ + from nullrun.transport import _parse_v3_error_envelope + + body = self._build_envelope(error_code, body_extra) + response = Response( + status_code=status, + content=json.dumps(body).encode("utf-8"), + headers={"Content-Type": "application/json"}, + ) + return _parse_v3_error_envelope(response, endpoint="check") + + def test_budget_hard_blocked_preserves_402(self): + exc = self._raise_via_parser("BUDGET_HARD_BLOCKED", 402) + assert isinstance(exc, NullRunBudgetError) + assert exc.status_code == 402 + + def test_budget_soft_blocked_preserves_402(self): + exc = self._raise_via_parser("BUDGET_SOFT_BLOCKED", 402) + assert isinstance(exc, NullRunBudgetError) + assert exc.status_code == 402 + + def test_budget_overdraft_exceeded_preserves_402(self): + exc = self._raise_via_parser("BUDGET_OVERDRAFT_EXCEEDED", 402) + assert isinstance(exc, NullRunBudgetError) + assert exc.status_code == 402 + + def test_redis_unavailable_preserves_402(self): + """BUDGET_REDIS_UNAVAILABLE is fail-CLOSED on the wire + (CLAUDE.md §4) — the SDK raises exactly as the backend + returned it (drift.md P1-2 honesty). + """ + exc = self._raise_via_parser("REDIS_UNAVAILABLE", 402) + assert isinstance(exc, NullRunBudgetError) + assert exc.status_code == 402 + + def test_workflow_inactive_preserves_403(self): + exc = self._raise_via_parser( + "WORKFLOW_INACTIVE", 403, body_extra={"workflow_id": "wf-abc"} + ) + assert isinstance(exc, NullRunWorkflowInactiveError) + assert exc.status_code == 403 + + def test_chain_cross_org_preserves_403(self): + exc = self._raise_via_parser( + "CHAIN_CROSS_ORG", 403, body_extra={"chain_id": "c-1"} + ) + assert isinstance(exc, NullRunChainError) + assert exc.status_code == 403 + + def test_chain_max_duration_preserves_402(self): + exc = self._raise_via_parser( + "CHAIN_MAX_DURATION_EXCEEDED", 402, body_extra={"chain_id": "c-1"} + ) + assert isinstance(exc, NullRunChainError) + assert exc.status_code == 402 + + def test_consume_overbudget_preserves_422(self): + exc = self._raise_via_parser( + "CONSUME_OVERBUDGET", + 422, + body_extra={ + "execution_id": "ex-1", + "reserved_cents": 10, + "max_allowed_cents": 11, + "actual_cost_cents": 100, + "epsilon_cents": 1, + }, + ) + assert isinstance(exc, NullRunConsumeOverbudgetError) + assert exc.status_code == 422 + + +# --------------------------------------------------------------------------- +# F3: fail-CLOSED / fail-OPEN honesty (drift.md P1-2) +# --------------------------------------------------------------------------- + +class TestFailClosedHonesty: + """F3 (drift.md P1-2): the SDK reads backend enforcement + responses as fail-CLOSED even when they're named with the word + "Redis" — wire 4xx/5xx that names an enforcement failure must + NOT be silently treated as a transport blip. + """ + + def test_redis_unavailable_is_fail_closed_402(self): + """``REDIS_UNAVAILABLE`` / ``BUDGET_REDIS_UNAVAILABLE`` → + NullRunBudgetError (fail-CLOSED). The SDK must not turn + this into a silent ALLOW — drift.md P1-2 explicitly + flagged the SDK_README claim that contradicted this. + """ + from nullrun.transport import _parse_v3_error_envelope + + response = Response( + status_code=402, + content=json.dumps( + { + "error_code": "REDIS_UNAVAILABLE", + "error_message": "Redis unreachable for budget counter", + "details": {"workflow_id": "wf-1"}, + "retry_after_ms": None, + } + ).encode("utf-8"), + headers={"Content-Type": "application/json"}, + ) + + exc = _parse_v3_error_envelope(response, endpoint="check") + assert isinstance(exc, NullRunBudgetError) + # Fail-CLOSED: the SDK raised the exception, it did NOT + # silently return a soft allow to the caller. status_code + # is preserved so the caller's HTTP layer sees 402. + assert exc.status_code == 402 + assert exc.retryable is False + + def test_rate_limit_redis_unavailable_is_fail_closed_503(self): + """``RATE_LIMIT_REDIS_UNAVAILABLE`` → NullRunRateLimitRedisError + (fail-CLOSED per CLAUDE.md §4 — aggregate rate limit is + the authoritative gate).""" + from nullrun.breaker.exceptions import NullRunRateLimitRedisError + from nullrun.transport import _parse_v3_error_envelope + + response = Response( + status_code=503, + content=json.dumps( + { + "error_code": "RATE_LIMIT_REDIS_UNAVAILABLE", + "error_message": "Redis unreachable for aggregate rate limit", + "details": {}, + "retry_after_ms": None, + } + ).encode("utf-8"), + headers={"Content-Type": "application/json"}, + ) + + exc = _parse_v3_error_envelope(response, endpoint="check") + assert isinstance(exc, NullRunRateLimitRedisError) + # Fail-CLOSED: the SDK raised, no silent allow. + assert exc.retryable is True \ No newline at end of file diff --git a/tests/test_v3_wire_contract.py b/tests/test_v3_wire_contract.py index f6537b2..d365b55 100644 --- a/tests/test_v3_wire_contract.py +++ b/tests/test_v3_wire_contract.py @@ -46,6 +46,7 @@ chain, get_chain_id, set_chain_id, + workflow, ) from nullrun.transport import ( _V3_ERROR_CODE_MAP, @@ -944,3 +945,192 @@ def test_cache_gate_disabled_via_env(self): ) assert cache_enabled is False os.environ.pop("NULLRUN_GATE_CACHE_DISABLE", None) + + +# ───────────────────────────────────────────────────────────────────── +# BUG #5 — chain-mode gate cache at the runtime level +# (CLAUDE.md §26 — collapse 100 /gate calls into 1 inside `with chain(...)`) +# ───────────────────────────────────────────────────────────────────── +# +# The TestGateCache data-structure tests above pin the runtime's +# `_GATE_CACHE` dict invariants in isolation; this class drives the +# full NullRunRuntime.check_workflow_budget() path so the +# cache_enabled predicate + cache hit/miss branches in +# ``runtime.py:1287-1310`` are actually exercised end-to-end. Without +# these tests ``pytest-cov`` reports that exact range as uncovered, +# which dragged patch coverage on PR #52 below the 70% Codecov floor. + + +class TestGateCacheRuntimeFlow: + """Runtime-level chain-mode gate cache coverage. + + Drives ``NullRunRuntime.check_workflow_budget()`` inside + ``with workflow(...) + with chain(...)`` and verifies the + /gate roundtrip count vs. expected after the 5s in-process + cache is applied. + """ + + def setup_method(self): + from nullrun import runtime as rt_mod + + rt_mod._GATE_CACHE.clear() + + def teardown_method(self): + from nullrun import runtime as rt_mod + + rt_mod._GATE_CACHE.clear() + # Always unset the gate-cache-disable opt-out so tests don't + # leak state between runs. + import os + + os.environ.pop("NULLRUN_GATE_CACHE_DISABLE", None) + + @respx.mock + def test_chain_mode_collapses_three_checks_to_one_gate_call(self): + """3 consecutive check_workflow_budget inside `with chain(...)` + must hit /gate exactly ONCE — the 2nd and 3rd calls fall + into the cache hit branch (runtime.py:1302). + + Covers: + runtime.py:1291-1310 (cache_enabled predicate), + runtime.py:1302 (cache hit `response = cached[1]`), + runtime.py:1306 (cache miss → transport.check + store). + """ + from nullrun.runtime import NullRunRuntime + + respx.post(f"{BASE_URL}/api/v1/gate").mock( + return_value=Response( + 200, + json={"decision": "allow", "decision_source": "gateway"}, + ) + ) + rt_inst = NullRunRuntime( + api_key="nr_live_abc123", + api_url=BASE_URL, + _test_mode=True, # skip _authenticate handshake + polling=False, # no background WS/HTTP poll thread + ) + try: + with workflow("wf-runtime-cache") as _wf_id, chain( + "chain-runtime-cache" + ) as _cid: + # Direct calls in chain scope — bypasses @protect but + # exercises the same check_workflow_budget codepath. + rt_inst.check_workflow_budget() + rt_inst.check_workflow_budget() + rt_inst.check_workflow_budget() + gate_calls = [ + c for c in respx.calls if c.request.url.path.endswith("/gate") + ] + assert len(gate_calls) == 1, ( + f"chain-mode cache must collapse 3 calls into 1 /gate " + f"roundtrip; got {len(gate_calls)}" + ) + finally: + try: + rt_inst.shutdown() + except Exception: + pass + + @respx.mock + def test_chain_mode_emits_fresh_uuid7_execution_id_per_call(self): + """BUG #4 wire at the runtime level: every /gate payload must + carry a fresh execution_id == uuid7 (NOT workflow_id). + + Disables the chain-mode cache so both ``check_workflow_budget`` + calls actually POST a /gate body — the cache would otherwise + collapse the second call into a hit and we'd never see the + second payload. + + Covers: + runtime.py:1247-1255 (execution_id = uuid7_str()), + runtime.py:1310-1323 (no-cache branch — direct transport.check). + """ + import json as _json + import os + + from nullrun.runtime import NullRunRuntime + + os.environ["NULLRUN_GATE_CACHE_DISABLE"] = "1" + try: + respx.post(f"{BASE_URL}/api/v1/gate").mock( + return_value=Response( + 200, + json={"decision": "allow", "decision_source": "gateway"}, + ) + ) + rt_inst = NullRunRuntime( + api_key="nr_live_abc123", + api_url=BASE_URL, + _test_mode=True, + polling=False, + ) + try: + with workflow("wf-runtime-uuid7"), chain("chain-runtime-uuid7"): + rt_inst.check_workflow_budget() + rt_inst.check_workflow_budget() + gate_calls = [ + c for c in respx.calls if c.request.url.path.endswith("/gate") + ] + assert len(gate_calls) == 2 + first = _json.loads(gate_calls[0].request.content)["execution_id"] + second = _json.loads(gate_calls[1].request.content)["execution_id"] + assert first != second + assert uuid.UUID(first).version == 7 + assert uuid.UUID(second).version == 7 + assert first != "wf-runtime-uuid7" + assert second != "wf-runtime-uuid7" + finally: + try: + rt_inst.shutdown() + except Exception: + pass + finally: + os.environ.pop("NULLRUN_GATE_CACHE_DISABLE", None) + + @respx.mock + def test_chain_mode_disabled_via_env_bypasses_cache(self): + """NULLRUN_GATE_CACHE_DISABLE=1 → cache_enabled=False → every + call hits /gate (runtime.py:1275-1277 fallback, runtime.py:1324 + direct transport.check path). + + Covers: + runtime.py:1294-1295 (cache_enabled=False exit), + runtime.py:1310-1323 (no-cache branch). + """ + import os + + from nullrun.runtime import NullRunRuntime + + os.environ["NULLRUN_GATE_CACHE_DISABLE"] = "1" + try: + respx.post(f"{BASE_URL}/api/v1/gate").mock( + return_value=Response( + 200, + json={"decision": "allow", "decision_source": "gateway"}, + ) + ) + rt_inst = NullRunRuntime( + api_key="nr_live_abc123", + api_url=BASE_URL, + _test_mode=True, + polling=False, + ) + try: + with workflow("wf-no-cache"), chain("chain-no-cache"): + rt_inst.check_workflow_budget() + rt_inst.check_workflow_budget() + gate_calls = [ + c for c in respx.calls if c.request.url.path.endswith("/gate") + ] + assert len(gate_calls) == 2, ( + f"with NULLRUN_GATE_CACHE_DISABLE=1 every call must " + f"hit /gate; got {len(gate_calls)}" + ) + finally: + try: + rt_inst.shutdown() + except Exception: + pass + finally: + os.environ.pop("NULLRUN_GATE_CACHE_DISABLE", None)