Skip to content

Implement client error correction#702

Open
Alan4506 wants to merge 1 commit into
smithy-lang:developfrom
Alan4506:feat/client-error-correction
Open

Implement client error correction#702
Alan4506 wants to merge 1 commit into
smithy-lang:developfrom
Alan4506:feat/client-error-correction

Conversation

@Alan4506
Copy link
Copy Markdown
Contributor

@Alan4506 Alan4506 commented May 26, 2026

Description of changes

This codegen generates Python clients from Smithy service models. For each operation's input and output shape, it produces a Python @dataclass whose fields mirror the model members. Members that the service model marks as @required are emitted as non-default keyword-only arguments, meaning the dataclass cannot be constructed without them.

After an operation completes, the SDK's deserialization layer reads the response body and calls the dataclass constructor with the parsed values. When the response follows the model, this works fine. However, when the response is missing a @required member, the constructor raises a Python TypeError for the missing keyword-only argument. The SDK pipeline catches that, classifies it as non-retryable, and re-raises as smithy_core.exceptions.SmithyError. The user sees something like:

smithy_core.exceptions.SmithyError: SomeOutput.__init__() missing 1 required keyword-only argument: 'foo'

This happens in two practical situations:

  • A successful response is missing one of the response shape's @required members. For example, Connect Health CreateSubscription returns an HTTP 200 response that omits the @required lastUpdatedAt field of CreateSubscriptionOutput. The subscription is created server-side, but the SDK call fails because the output dataclass cannot be constructed without last_updated_at.
  • An error response is missing one of the error shape's @required members. For example, Q Business ChatSync, when rejecting a request with ValidationException, returns an exception body that omits the @required reason field of ValidationException. The user sees a generic message about a missing kwarg instead of the actual service-side error message: the real reason the request was rejected is masked.

The Smithy spec accounts for this in § 3.3.3.1.1 Client error correction: clients MAY substitute a zero-value default for a missing required member to avoid downtime. This PR implements that policy in the Python codegen. Each shape type gets the spec's prescribed default. The implementation lives in a new MemberErrorCorrectionGenerator (a ShapeVisitor) plus two hooks in StructureGenerator: error correction at the end of the generated deserialize_kwargs, and a generated _smithy_default() classmethod used when a parent's required nested-struct member is missing on the wire.

Note: enum and intEnum currently fall back to placeholder zero values ("" and 0). Spec § 3.3.3.1.1 recommends (SHOULD) that these types define a structured unknown variant for receiving wire values not known at SDK-generation time. Implementing it requires redesigning the generated enum/intEnum classes: today they are plain StrEnum / IntEnum subclasses with no room for an unknown member, so if necessary, a follow-up would need to change EnumGenerator / IntEnumGenerator (and likely PythonSymbolProvider, MemberDeserializerGenerator) to emit a wrapper type that exposes something like MyEnum.unknown(value) and is_unknown. The two enum visitor methods in MemberErrorCorrectionGenerator have TODO comments documenting this.

Testing

:core:test, :core:integ, :core:spotlessCheck, the aws-query and rest-json-1 protocol-test suites all pass.

Also, two real-world cases were verified before/after:

Amazon Connect Health CreateSubscription. Before this PR:

Traceback (most recent call last):
  File ".../smithy_http/aio/protocols.py", line 152, in deserialize_response
    return operation.output.deserialize(deserializer)
  File ".../aws_sdk_connecthealth/models.py", line 1491, in deserialize
    return cls(**cls.deserialize_kwargs(deserializer))
TypeError: CreateSubscriptionOutput.__init__() missing 1 required keyword-only argument: 'last_updated_at'

The above exception was the direct cause of the following exception:
  File ".../smithy_core/aio/client.py", line 258, in _execute_request
    raise SmithyError(e) from e
smithy_core.exceptions.SmithyError: CreateSubscriptionOutput.__init__() missing 1 required keyword-only argument: 'last_updated_at'

After this PR, last_updated_at is filled with datetime.fromtimestamp(0, tz=timezone.utc), the call returns normally.

Amazon Q Business ChatSync. Before this PR:

smithy_core.exceptions.SmithyError: ValidationException.__init__() missing 1 required keyword-only argument: 'reason'

After this PR:

aws_sdk_qbusiness.models.ValidationException: Please provide a unique clientToken that can be used as an idempotency attribute.

reason is filled with "", the ValidationException constructs cleanly, and the actual validation message reaches the caller.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@Alan4506 Alan4506 requested a review from a team as a code owner May 26, 2026 05:04
@Alan4506 Alan4506 changed the title Implement error client error correction Implement client error correction May 26, 2026
@Alan4506 Alan4506 force-pushed the feat/client-error-correction branch from 4f2764e to 40c648b Compare May 26, 2026 05:07
@Alan4506 Alan4506 force-pushed the feat/client-error-correction branch from 40c648b to e57b0b7 Compare May 26, 2026 05:55
Copy link
Copy Markdown
Contributor

@alexgromero alexgromero left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Yuxuan, mostly looks good! Would be great to see some testing for the new error correction path though.

* spec: Client error correction</a>
*/
@SmithyInternalApi
public final class MemberErrorCorrectionGenerator extends ShapeVisitor.DataShapeVisitor<Boolean> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

non-blocking nit: any reason this is <Boolean> instead of <Void>? The other DataShapeVisitor subclasses (MemberSerializerGenerator, MemberDeserializerGenerator) use <Void> and the return value here doesn't seem to be consumed.

Copy link
Copy Markdown
Contributor

@jonathan343 jonathan343 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Alan4506! Let a few comments I think are important to address before merging. +1 to testing as well.

Also, we'll want to patch bump the code generator incase there are not other changes when we release this. Otherwise, our release infra won't generate the new SDK clients.

}
writer.write("""
@classmethod
def _smithy_default(cls) -> Self:
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we generate _smithy_default for every structure, including cases where no generated code can call it, such as structures with no required members or structures that are never used as the target of a required nested structure member.

Could we narrow this so the helper is only emitted when it's needed to create a missing required structure member? This will eliminate some dead code that looks like the following:

@classmethod
def _smithy_default(cls) -> Self:
    return cls()

// TODO: the Smithy spec recommends intEnum types define an unknown variant. If a
// future change adds an unknown variant to the generated intEnum class (e.g.
// MyIntEnum.unknown(value)), revisit this to emit it instead of the bare 0.
writer.writeInline("0");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the safer short-term fix is to omit client error correction for enum and intEnum until we have a proper unknown-value representation for them. Longer term, we should add first-class unknown enum/intEnum support and then use that here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants