Skip to content

Fix durable execution conformance issues found via cross-SDK testing#2473

Draft
GarrettBeatty wants to merge 1 commit into
devfrom
durable-execution-conformance-fixes
Draft

Fix durable execution conformance issues found via cross-SDK testing#2473
GarrettBeatty wants to merge 1 commit into
devfrom
durable-execution-conformance-fixes

Conversation

@GarrettBeatty

Copy link
Copy Markdown
Contributor

Summary

Fixes five behavioral bugs in the Amazon.Lambda.DurableExecution operations, surfaced by running the multi-language durable-execution conformance suite against the .NET SDK and comparing the emitted execution history / results against the JS, Python, and Java reference SDKs.

All five were confirmed as divergences from the reference SDKs (one matches a known open bug in the Python SDK, issue #279).

Fixes

  1. ConcurrentOperation — missing ParentId on parent CONTEXT checkpoints. Parallel/Map did not set ParentId on their START/SUCCEED checkpoints (every other operation type does). This dropped the parent linkage for a nested parallel/map — its child events lost their ParentId.

  2. CallbackOperation — missing ParentId on the callback START. Nested callbacks (e.g. the inner callback created by WaitForCallbackAsync) lost their ParentId.

  3. WaitForConditionOperationStepStarted only emitted on the first poll. Now emits a fresh StepStarted before every poll iteration, matching Python/JS/Java. On resume the persisted status is READY/PENDING, so each poll gets its own START; the only skip is when the status is already STARTED (crash-recovery, where the original START is authoritative).

  4. WaitForConditionOperationSTART checkpoint was awaited instead of fire-and-forget. Awaiting it forced two back-to-back synchronous flushes (START then terminal FAIL) with no suspension when the check throws on the first attempt, which could hang the invocation. Now fire-and-forget, mirroring StepOperation and the reference SDKs. Replay correctness depends only on the terminal RETRY/SUCCEED/FAIL checkpoints.

  5. ConcurrentOperation/BatchResultTotalCount counted never-dispatched branches. Never-dispatched branches (skipped by a CompletionConfig short-circuit before they ever started) are now excluded from the user-facing All list, so TotalCount = started(in-flight) + succeeded + failed, matching the JS SDK (same bug as Python Support for Binary Data using Razor Pages #279). A dispatched-then-bailed (genuinely in-flight) branch still appears as Started. The completion policy still reasons over the declared branch count (so ToleratedFailurePercentage and the early-stop / min-successful signal are unchanged), and the persisted checkpoint summary still records every declared unit for replay/reconstruct. The replay and reconstruct paths were updated to keep the reconstructed All consistent with the fresh run.

Tests

Updated the affected ParallelOperationTests and MapOperationTests to reflect the corrected never-dispatched semantics. Dispatched-then-bailed (in-flight) cases are deliberately unchanged. All 72 parallel + map operation unit tests pass.

Validated end-to-end against the conformance harness: all implementable .NET conformance tests pass across step, wait, callback, child, invoke, parallel, wait_for_callback, and wait_for_condition operations.

Draft: opening for review of the SDK behavior changes. Three conformance cases (per-operation custom serdes for child/parallel/invoke) remain unimplementable in .NET and are tracked separately as a capability gap, not addressed here.

Fixes five behavioral bugs in the DurableExecution operations, surfaced by
running the multi-language conformance suite against the .NET SDK and comparing
against the JS/Python/Java reference SDKs.

1. ConcurrentOperation: emit ParentId on the parent CONTEXT START/SUCCEED
   checkpoints. Every other operation type sets ParentId; parallel/map did not,
   which dropped the parent linkage for a nested parallel/map (its child events
   lost their ParentId).

2. CallbackOperation: emit ParentId on the callback START checkpoint. Nested
   callbacks (e.g. the inner callback of a WaitForCallback) lost their ParentId.

3. WaitForConditionOperation: emit a fresh StepStarted before every poll
   iteration (not just the first). On resume the persisted status is
   READY/PENDING, so each poll now gets its own START, matching the
   Python/JS/Java SDKs. The only skip is when status is already STARTED
   (crash-recovery: the original START is authoritative).

4. WaitForConditionOperation: make the START checkpoint fire-and-forget instead
   of awaiting it. Awaiting forced two back-to-back synchronous flushes
   (START then terminal FAIL) with no suspension when the check throws on the
   first attempt, which could hang the invocation. Mirrors StepOperation and the
   reference SDKs; replay correctness relies only on the terminal
   RETRY/SUCCEED/FAIL checkpoints.

5. ConcurrentOperation/BatchResult: exclude never-dispatched branches from the
   user-facing All list so TotalCount = started(in-flight) + succeeded + failed,
   matching the JS SDK (same bug tracked in Python issue #279). A
   dispatched-then-bailed (genuinely in-flight) branch still appears as Started.
   The completion policy still reasons over the declared branch count (so
   ToleratedFailurePercentage and the early-stop signal are unchanged), and the
   persisted checkpoint summary still records every declared unit for
   replay/reconstruct. Replay/reconstruct paths updated to keep the reconstructed
   All consistent with the fresh run.

Updated the affected Parallel/Map operation unit tests to reflect the corrected
never-dispatched semantics (dispatched-then-bailed cases are unchanged).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant