Implement client error correction#702
Conversation
4f2764e to
40c648b
Compare
40c648b to
e57b0b7
Compare
alexgromero
left a comment
There was a problem hiding this comment.
Thanks Yuxuan, mostly looks good! Would be great to see some testing for the new error correction path though.
| * spec: Client error correction</a> | ||
| */ | ||
| @SmithyInternalApi | ||
| public final class MemberErrorCorrectionGenerator extends ShapeVisitor.DataShapeVisitor<Boolean> { |
There was a problem hiding this comment.
non-blocking nit: any reason this is <Boolean> instead of <Void>? The other DataShapeVisitor subclasses (MemberSerializerGenerator, MemberDeserializerGenerator) use <Void> and the return value here doesn't seem to be consumed.
jonathan343
left a comment
There was a problem hiding this comment.
Thanks @Alan4506! Let a few comments I think are important to address before merging. +1 to testing as well.
Also, we'll want to patch bump the code generator incase there are not other changes when we release this. Otherwise, our release infra won't generate the new SDK clients.
| } | ||
| writer.write(""" | ||
| @classmethod | ||
| def _smithy_default(cls) -> Self: |
There was a problem hiding this comment.
It seems like we generate _smithy_default for every structure, including cases where no generated code can call it, such as structures with no required members or structures that are never used as the target of a required nested structure member.
Could we narrow this so the helper is only emitted when it's needed to create a missing required structure member? This will eliminate some dead code that looks like the following:
@classmethod
def _smithy_default(cls) -> Self:
return cls()| // TODO: the Smithy spec recommends intEnum types define an unknown variant. If a | ||
| // future change adds an unknown variant to the generated intEnum class (e.g. | ||
| // MyIntEnum.unknown(value)), revisit this to emit it instead of the bare 0. | ||
| writer.writeInline("0"); |
There was a problem hiding this comment.
I think the safer short-term fix is to omit client error correction for enum and intEnum until we have a proper unknown-value representation for them. Longer term, we should add first-class unknown enum/intEnum support and then use that here.
Description of changes
This codegen generates Python clients from Smithy service models. For each operation's input and output shape, it produces a Python
@dataclasswhose fields mirror the model members. Members that the service model marks as@requiredare emitted as non-default keyword-only arguments, meaning the dataclass cannot be constructed without them.After an operation completes, the SDK's deserialization layer reads the response body and calls the dataclass constructor with the parsed values. When the response follows the model, this works fine. However, when the response is missing a
@requiredmember, the constructor raises a PythonTypeErrorfor the missing keyword-only argument. The SDK pipeline catches that, classifies it as non-retryable, and re-raises assmithy_core.exceptions.SmithyError. The user sees something like:This happens in two practical situations:
@requiredmembers. For example, Connect HealthCreateSubscriptionreturns an HTTP 200 response that omits the@required lastUpdatedAtfield ofCreateSubscriptionOutput. The subscription is created server-side, but the SDK call fails because the output dataclass cannot be constructed withoutlast_updated_at.@requiredmembers. For example, Q BusinessChatSync, when rejecting a request withValidationException, returns an exception body that omits the@required reasonfield ofValidationException. The user sees a generic message about a missing kwarg instead of the actual service-side error message: the real reason the request was rejected is masked.The Smithy spec accounts for this in § 3.3.3.1.1 Client error correction: clients MAY substitute a zero-value default for a missing required member to avoid downtime. This PR implements that policy in the Python codegen. Each shape type gets the spec's prescribed default. The implementation lives in a new
MemberErrorCorrectionGenerator(aShapeVisitor) plus two hooks inStructureGenerator: error correction at the end of the generateddeserialize_kwargs, and a generated_smithy_default()classmethod used when a parent's required nested-struct member is missing on the wire.Note: enum and intEnum currently fall back to placeholder zero values (
""and0). Spec § 3.3.3.1.1 recommends (SHOULD) that these types define a structured unknown variant for receiving wire values not known at SDK-generation time. Implementing it requires redesigning the generated enum/intEnum classes: today they are plainStrEnum/IntEnumsubclasses with no room for an unknown member, so if necessary, a follow-up would need to changeEnumGenerator/IntEnumGenerator(and likelyPythonSymbolProvider,MemberDeserializerGenerator) to emit a wrapper type that exposes something likeMyEnum.unknown(value)andis_unknown. The two enum visitor methods inMemberErrorCorrectionGeneratorhave TODO comments documenting this.Testing
:core:test,:core:integ,:core:spotlessCheck, theaws-queryandrest-json-1protocol-test suites all pass.Also, two real-world cases were verified before/after:
Amazon Connect Health
CreateSubscription. Before this PR:After this PR,
last_updated_atis filled withdatetime.fromtimestamp(0, tz=timezone.utc), the call returns normally.Amazon Q Business
ChatSync. Before this PR:After this PR:
reasonis filled with"", theValidationExceptionconstructs cleanly, and the actual validation message reaches the caller.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.