feat(interceptors): initial gateway interceptor implementation and reference example

[drew]

Summary

Implements #1927

Related Issue

Closes #1919, #1842

Changes

• Add interceptor, binding, and decision labels to gateway interceptor evaluation counters.
• Remove Z3-specific build setup from the governance interceptor smoke script.
• Start a dedicated smoke gateway with isolated ports, state, gateway ID, and scrubbed ambient OpenShell environment.
• Print neutral setup INFO messages and reserve PASS/FAIL for functional checks.
• Dump smoke, gateway, and interceptor logs inline on failures.
• Refresh the governance interceptor README smoke-test wording.

Testing

• CC=clang CXX=clang++ mise run pre-commit passes
• OPENSHELL_GOVERNANCE_KEEP_LOGS=1 examples/governance-interceptor/smoke.sh passes
• Hostile ambient OpenShell env smoke run passes
• Unit tests added/updated (not needed for smoke script hardening)
• E2E tests added/updated (not applicable)

Checklist

• Follows Conventional Commits
• Commits are signed off (DCO)

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-2005.docs.buildwithfern.com/openshell

[johntmyers]

gator-agent

Blocked

Gator is blocked because PR #2005 is still marked as a draft.

Next action: @drew, mark the pull request ready for review when the interceptor implementation is ready for the independent code review and CI-gating workflow.

Head SHA: e6e5a8d35b2c4118496008ba592217201a016ef0

[johntmyers]

gator-agent

PR Review Status

Validation: This is project-valid, maintainer-authored work implementing #1919 and the actively reviewed gateway-interceptors RFC in #1927, with #1842 explicitly folded into the interceptor design.

Head SHA: e6e5a8d35b2c4118496008ba592217201a016ef0

Thanks @drew. I also checked your response on the crate-layout discussion: the current flat, component-prefixed openshell-gateway-interceptors layout is consistent with that stated direction, so I am treating that discussion as resolved.

Review findings:

• Critical — sandbox principals can bypass the signed-governance path. examples/governance-interceptor/src/main.rs:639, crates/openshell-gateway-interceptors/src/routes.rs:30, and crates/openshell-server/src/grpc/policy.rs:2550: sandbox UpdateConfig is allowed before signature validation, and non-interceptable SubmitPolicyAnalysis can auto-approve policy changes. Governed sandboxes need authenticated E2E coverage proving neither path can widen policy without valid authority.
• Critical — an interceptor can self-expand its authority and receive gateway secrets. crates/openshell-core/src/config.rs:529, crates/openshell-gateway-interceptors/src/lib.rs:340, and :381: Describe controls bindings and profile-source authority unless optionally narrowed. This can expose provider credentials or refresh material, including over accepted plaintext http:// endpoints. Require an operator allowlist, secret-field redaction/authorization, and UDS or authenticated TLS for secret-bearing operations.
• Critical — post_commit fail-closed can report failure after a mutation committed. crates/openshell-server/src/multiplex.rs:296-303: a post-commit timeout/invalid result replaces a committed success with PERMISSION_DENIED, inviting unsafe retries. Preserve the committed response and report audit failure separately (or use a transactional outbox), with middleware coverage.
• Critical — policy state and governance provenance are committed non-atomically. crates/openshell-server/src/grpc/policy.rs:1841-1855 and :1998-2016: policy writes happen before the separate annotation CAS. A stale/concurrent update can leave active policy without its signature/provenance while returning an error. Commit these together or store attestation atomically with the policy revision, and test the conflict path.
• Warning — the protobuf transcoder drops oneof semantics. crates/openshell-gateway-interceptors/src/lib.rs:1128-1160, :1209-1223, and :1607-1640: malformed wire input with multiple alternatives can be validated as one shape and re-encoded as another. Preserve oneof metadata or reject multiple alternatives, with a wire-level test.
• Warning — wrong-type patches bypass per-binding fail_open handling. crates/openshell-gateway-interceptors/src/lib.rs:568-579 and :497-500: JSON Patch can succeed before final protobuf encoding fails outside the binding's failure policy. Schema-validate/re-encode each candidate mutation before accepting it.
• Warning — provider profile snapshots are repeatedly fetched and can mix revisions. crates/openshell-server/src/provider_profile_sources.rs:167-180, :200-232, and crates/openshell-server/src/grpc/policy.rs:1317-1334: fetch one immutable catalog per request or use a validated revision-keyed cache.
• Warning — signed hashes are not canonical for nested protobuf maps. examples/governance-interceptor/src/policy_hash.rs:10-30 and examples/governance-interceptor/src/main.rs:895-914: recursively canonicalize nested maps and test insertion-order equivalence.
• Warning — docs advertise an unimplemented append catalog mode. docs/sandboxes/providers-v2.mdx:55 says catalogs can be append or authoritative, while crates/openshell-server/src/provider_profile_sources.rs:129-139 always replaces built-in/user sources when interceptor sources exist. Remove the append claim or implement/configure it.

Docs: Relevant Fern docs and gateway architecture docs are updated, but the append-catalog statement above must be corrected. No navigation change appears necessary for the existing pages.

Tests: The implementation is gateway/policy/provider infrastructure and will require test:e2e once review findings are resolved and pipeline gating begins.

Next state: gator:in-review

[johntmyers]

gator-agent

PR Review Status

Validation: This is project-valid maintainer-authored work implementing #1919 and the active gateway-interceptors RFC in #1927, with #1842 folded into the interceptor/governance design.

Head SHA: 4cf7381d39be0d0dff8191a5ecce8ccb229d24c6

Thanks @drew. I rechecked the July 10 updates against the earlier gator findings. The provider-profile source composition work resolves the append/authoritative docs mismatch and narrows the old interceptor profile-source self-selection concern, but blocking review feedback remains.

Review findings:

• Critical — sandbox principals can still bypass signed/frozen governance. examples/governance-interceptor/src/main.rs:652, crates/openshell-server/src/grpc/policy.rs:1021, crates/openshell-gateway-interceptors/src/routes.rs:30, and crates/openshell-server/src/grpc/policy.rs:2550: the example still unconditionally allows sandbox principals before validating governance annotations, while core only restricts sandbox callers to sandbox-scoped policy sync shape. SubmitPolicyAnalysis also remains non-interceptable and can auto-merge policy changes. Remove the unconditional sandbox allow and apply the same frozen-governance guard to sandbox policy sync and SubmitPolicyAnalysis auto-approval, or make that path interceptable.
• Critical — policy revisions and governance provenance still commit separately. crates/openshell-server/src/grpc/policy.rs:1998, crates/openshell-server/src/grpc/policy.rs:2009, crates/openshell-server/src/grpc/policy.rs:1071, and crates/openshell-server/src/grpc/policy.rs:1841: a policy revision can be activated before the annotation/provenance CAS succeeds, so a conflict can return an error after committing policy state without matching provenance. Store provenance with the policy revision or wrap both writes in one transaction, with conflict-path coverage.
• Critical — post_commit can still report failure after a mutation committed. crates/openshell-server/src/multiplex.rs:296, crates/openshell-server/src/multiplex.rs:309, crates/openshell-gateway-interceptors/src/lib.rs:996, and crates/openshell-gateway-interceptors/src/lib.rs:963: post-commit interceptor failure can replace a successful committed response with PERMISSION_DENIED, inviting unsafe retries. Preserve the committed response and report audit/metrics/outbox failure separately; blocking decisions need to happen before commit.
• Warning — wrong-type JSON patches still bypass per-binding fail_open. crates/openshell-gateway-interceptors/src/lib.rs:533 and crates/openshell-gateway-interceptors/src/lib.rs:450: patch application is under the binding failure policy, but protobuf re-encoding happens later outside that policy. Validate or re-encode each patched candidate under the binding policy before accepting it.
• Warning — provider profile snapshots can still mix revisions within one request. crates/openshell-server/src/provider_profile_sources.rs:266, crates/openshell-server/src/provider_profile_sources.rs:299, and crates/openshell-server/src/grpc/policy.rs:1392: profile lookups can rebuild the effective catalog independently. Fetch one immutable catalog snapshot per request/config build and pass it through.
• Warning — secret-bearing interceptor traffic can use plaintext http://. crates/openshell-core/src/config.rs:538 and crates/openshell-gateway-interceptors/src/lib.rs:849: interceptable operations can include provider credentials, but endpoints may be plaintext HTTP. Require UDS or authenticated TLS for secret-bearing bindings, or redact secrets unless explicitly allowed.
• Warning — signed hashes are still not canonical for nested maps. examples/governance-interceptor/src/policy_hash.rs:10 and examples/governance-interceptor/src/policy_hash.rs:29: only top-level network_policies are sorted before hashing nested protobuf encodings. Use recursive canonicalization or canonical JSON for every map-bearing message before signing.

Docs: The provider-profile source docs now match implementation, including the authoritative-by-omission behavior. No Fern navigation change appears necessary for the existing pages.

Tests: Once review findings are resolved, this gateway/policy/provider infrastructure change should run with test:e2e; add focused coverage for governed UpdateConfig, SubmitPolicyAnalysis auto-approval, annotation CAS conflict after policy write, and post-commit failure after commit.

Next state: gator:in-review