Support passing arbitrary secrets as environment variables to test jobs

[MariusStorhaug]

Context

Module test jobs need a safe way to receive caller-selected secrets and non-secret configuration without hard-coding every possible test credential name in Process-PSModule. Some modules need no test data, while others need module-specific environment variable names for API tokens, app credentials, service URLs, usernames, space keys, or similar test configuration.

Technical Decisions

Interface: Add one optional workflow-call secret named TestData to .github/workflows/workflow.yml. TestData is a single-line JSON object with optional secrets and variables maps:

{ "secrets": { "NAME": "value" }, "variables": { "NAME": "value" } }

Exposure model: Entries from both maps are exposed as environment variables in BeforeAll-ModuleLocal, Test-ModuleLocal, and AfterAll-ModuleLocal. Values under secrets are masked before they are written to GITHUB_ENV; values under variables are not masked.

Security boundary: Do not use broad secrets: inherit. The calling workflow must explicitly list the values it wants to expose in TestData, so the shared workflow only receives the caller-selected test data.

Validation: TestData must parse as a JSON object containing only optional secrets and variables maps. Map values must be scalar values. Keys must be safe environment-variable names matching ^[A-Za-z_][A-Za-z0-9_]*$ and must not override reserved variables such as PATH, CI, GITHUB_*, RUNNER_* or ACTIONS_*.

Masking and log readability: TestData must be passed as a single-line value after YAML folding. Pretty-printed JSON with nested indentation is not safe because YAML preserves more-indented lines inside a folded scalar; brace-only lines can become individual GitHub masks and pollute unrelated logs with ***.

Environment-scoped values: GitHub Environment secrets and variables are usable when the calling job sets environment: and explicitly includes those values in TestData; they are not exposed automatically.

Acceptance Criteria

• Test jobs can read arbitrary, module-defined secret names without changing the shared workflow.
• No module-specific secret names are hard-coded in .github/workflows/workflow.yml or the *-ModuleLocal workflows.
• Secret values are masked in logs before being exposed through GITHUB_ENV.
• Non-secret configuration is supported through the variables map.
• Modules that need no test data work unchanged by omitting TestData.
• Environment-scoped secrets and variables are documented as requiring environment: on the calling job plus explicit inclusion in TestData.
• The duplicated fixed TEST_* pass-through blocks are removed from the main reusable workflow and *-ModuleLocal workflows.

Implementation Plan

• Replace the fixed workflow-call TEST_* secret inputs on .github/workflows/workflow.yml with TestData.
• Pass TestData to BeforeAll-ModuleLocal, Test-ModuleLocal, and AfterAll-ModuleLocal.
• Parse TestData in the *-ModuleLocal workflows and expose selected values through GITHUB_ENV.
• Mask secret values and avoid printing secret key names.
• Validate JSON shape, environment-variable names, reserved names, and scalar values before writing to GITHUB_ENV.
• Document the breaking migration path from fixed TEST_* inputs to TestData.
• Document safe YAML folding, variable syntax, base64 guidance for complex secrets, and environment-scoped values.

Related

• Implemented by PR 🌟 [Major]: Fixed test secret inputs replaced by TestData #365.
• Also addresses the duplicated fixed-secret pass-through concern in Replace duplicated test-secret blocks with secrets: inherit in ModuleLocal workflows #322 without adopting broad secrets: inherit.

[MariusStorhaug]

Specification: Securely Passing Arbitrary Secrets to Reusable GitHub Workflows

Summary

Implement a secure method for passing an unlimited number of arbitrary secrets to GitHub reusable workflows using asymmetric encryption (libsodium), allowing secrets to be managed without explicit declaration within the reusable workflow.

Background

GitHub reusable workflows currently require all secrets to be explicitly declared, limiting flexibility when dynamically managing many secrets. To overcome this, encrypted variables (ciphertexts) will be passed through environment variables (env) and decrypted at runtime with a private key stored securely in GitHub secrets.

Proposed Solution

Workflow

Preparation

• Generate an asymmetric key pair (using libsodium).
• Store the private key securely as a GitHub secret (PRIVATE_KEY_SECRET).
• Encrypt each secret individually using the public key and store them as GitHub variables.

Workflow Structure

Caller Workflow:

name: Caller Workflow
on: [workflow_dispatch]

env:
  ENCRYPTED_SECRET1: ${{ vars.ENCRYPTED_SECRET1 }}
  ENCRYPTED_SECRET2: ${{ vars.ENCRYPTED_SECRET2 }}

jobs:
  call-reusable-workflow:
    uses: org/repo/.github/workflows/reusable.yml@main
    secrets:
      PRIVATE_KEY_SECRET: ${{ secrets.PRIVATE_KEY_SECRET }}

Reusable Workflow:

name: Reusable Workflow
on:
  workflow_call:
    secrets:
      PRIVATE_KEY_SECRET:
        required: true

jobs:
  decrypt-secrets:
    runs-on: ubuntu-latest
    steps:
      - name: Decrypt Secrets
        run: |
          npm install -g sodium-cli
          PRIVATE_KEY="${{ secrets.PRIVATE_KEY_SECRET }}"

          SECRET1=$(echo "$ENCRYPTED_SECRET1" | sodium-cli seal-open "$PRIVATE_KEY")
          SECRET2=$(echo "$ENCRYPTED_SECRET2" | sodium-cli seal-open "$PRIVATE_KEY")

          echo "::add-mask::$SECRET1"
          echo "::add-mask::$SECRET2"

Security Considerations

• Never expose decrypted secrets in logs; always use GitHub Actions' masking feature (::add-mask::).
• Regularly rotate encryption keys.
• Only store ciphertext (encrypted data) publicly or in repository variables.
• Private keys must be securely managed as GitHub secrets.

Benefits

• Enables unlimited, dynamic secret management.
• Ensures secrets are managed securely and flexibly.
• Eliminates the need to explicitly declare each secret in the reusable workflow, reducing workflow complexity.

Acceptance Criteria

• A functional example demonstrating secure encryption, environment variable passing, and runtime decryption.
• Documentation detailing the key generation, encryption, and decryption process.
• Explicit demonstration of secret masking to ensure workflow security compliance.

[MariusStorhaug]

Did this to support this with less input.

PSModule/Sodium#30

With this we only need to store the private key and encrypted secrets in variables.

[MariusStorhaug]

This is basically BYOK for GitHub secrets?

[MariusStorhaug]

Issue: To make new keys, the user needs to have the private key.

Alternatives:

1. Store privateKey locally in a Context, and use this to create new secrets.
2. Use a single (template) script for the key value pair for how to create secrets. This could generate a new private key and upload that to a GitHub secret on a per need basis.

[MariusStorhaug]

Need functionality in Process-PSModule for how to do this "seamlessly". Would work great if we had the BYOK support from the GitHub module.

Just add a switch for PrivateKey? Lets explore...

[MariusStorhaug]

Could potentially just store it in KeyVault?

[MariusStorhaug]

Restructured issue description into the standard three-section format.

• Rewrote into a proper context/request structure preserving the code example
• Updated title to imperative mood (removed emoji prefix)
• Added technical decisions for input format and masking approach
• Created implementation plan with 6 tasks
• Applied Minor label (new feature)

[MariusStorhaug]

Cross-reference: The structural duplication of test secrets across BeforeAll-ModuleLocal.yml, Test-ModuleLocal.yml, and AfterAll-ModuleLocal.yml is tracked in #322 (Replace duplicated test-secret blocks with secrets: inherit). That issue addresses the DRY violation with the current fixed set of secrets, while this issue covers the broader capability of passing arbitrary secrets.

[MariusStorhaug]

Empirical finding: the proposed env: input can't carry secrets — here's what does

I ran an isolated experiment covering every way a caller can expose secrets to a reusable workflow — provider + two independent consumers, full write-up with run links:

• Provider (generic runner): https://github.com/MariusStorhaug/reusable-workflow-demo
• Consumers: https://github.com/MariusStorhaug/reusable-workflow-consumer and https://github.com/MariusStorhaug/reusable-workflow-consumer-b
• Report: https://github.com/MariusStorhaug/reusable-workflow-consumer/blob/main/REPORT.md

The env: map in with: (as proposed) is rejected at validation

A reusable-workflow-call with: block cannot reference the secrets context, so
with: { env: "TEST_PAT: ${{ secrets.TEST_PAT }}" } never even reaches the parsing step. Proven three independent ways:

• GitHub dispatch API → HTTP 422: Unrecognized named-value: 'secrets' ... within expression: secrets.REPO_SECRET
• actionlint: context "secrets" is not allowed here. available contexts are "github","inputs","matrix","needs","strategy","vars"
• a real run failed with "This run likely failed because of a workflow file issue" (0 jobs created)

What works — and needs no new input

Consumers already call us with secrets: inherit, so the entire secrets context is available inside the workflow. A workflow step's env: can reference toJSON(secrets) (a with: can't). Expand it (plus vars) into environment variables on the test job, and existing Pester tests keep reading $env:TEST_* unchanged:

# *-ModuleLocal test job, before Pester
- name: Expose inherited secrets + vars
  shell: pwsh
  env:
    ALL_SECRETS: ${{ toJSON(secrets) }}   # a STEP env CAN reference toJSON(secrets); a reusable-wf with: cannot
    ALL_VARS:    ${{ toJSON(vars) }}       # repo/org vars flow automatically (no inherit needed)
  run: |
    foreach ($p in ($env:ALL_SECRETS | ConvertFrom-Json).PSObject.Properties) {
      if ($p.Name -eq 'github_token') { continue }
      Write-Host "::add-mask::$($p.Value)"
      "$($p.Name)=$($p.Value)" >> $env:GITHUB_ENV
    }
    foreach ($p in ($env:ALL_VARS | ConvertFrom-Json).PSObject.Properties) {
      "$($p.Name)=$($p.Value)" >> $env:GITHUB_ENV
    }

Proven across two unrelated consumers using the same runner with zero secret names in it:

Consumer	Saw secrets	Saw vars
A	REPO_SECRET	CONSUMER_A_REGION, CONSUMER_A_BASE_URL
B	DOMENESHOP_TOKEN, DOMENESHOP_API_SECRET	DOMENESHOP_BASE_URL, DOMENESHOP_TENANT

Neither saw the other's names; the runner was unchanged between them. Modules that need no secrets are a no-op.

Notes / caveats (all verified)

• Inherited secrets are still auto-masked in logs; the ::add-mask:: above also covers values expanded into env vars.
• secrets: inherit carries repo + org secrets only. If a module's test creds live in a GitHub Environment, the test job must also set environment: <name> (inherit alone won't expose environment secrets).
• This folds in Replace duplicated test-secret blocks with secrets: inherit in ModuleLocal workflows #322: drop the ~7 duplicated explicit TEST_* passes and keep secrets: inherit end-to-end.
• If we ever want the required secrets enforced/visible per call site, the alternative is a composite action whose step with:/env: can reference secrets — but the toJSON step above needs no per-module wiring.

I've revised the issue body to this approach and kept the original proposal in a collapsed section for history.