Skip to content

Add modular AWS credential resolution#1172

Open
mtdowling wants to merge 16 commits into
mainfrom
aws-auth
Open

Add modular AWS credential resolution#1172
mtdowling wants to merge 16 commits into
mainfrom
aws-auth

Conversation

@mtdowling
Copy link
Copy Markdown
Member

@mtdowling mtdowling commented May 7, 2026

Add modular credential chain with pluggable provider discovery

Adds a modular AWS credential provider chain that separates credential detection from resolution, enabling independent artifact composition.

New modules:

  • aws-config: SEP-conformant INI parser, AwsProfileFile, AwsProfile, AwsConfigCredentialSource sealed types. Standalone module with no chain dependency.
  • aws-credential-chain: CredentialChain, ChainIdentityProvider SPI, ChainSetup, StandardProvider enum with expanded profile-source slots, OrderingConstraint (Standard/Before/After), SharedConfigProvider, StaticKeysHandler, SessionKeysHandler, CredentialProcessHandler.
  • aws-credentials-imds: IMDSv2 credential provider with static stability, extended/legacy API fallback, config-based disable.

Changes to existing modules:

  • auth-api: Add CachingIdentityResolver with async background refresh, static stability, injectable Clock/ScheduledExecutorService. Add invalidate() default method to IdentityResolver.
  • aws-client-core: Add AwsCredentialChainPlugin, auth-failure interceptor, EnvironmentCredentialProvider and
    SystemPropertiesCredentialProvider (both terminal — stop chain assembly when credentials are present).
  • settings.gradle.kts: Include new modules.

Architecture:

  • One interface (ChainIdentityProvider), one SPI, void create() method.
  • Providers are sorted by ordering() then called in order with a shared mutable ChainSetup. They call setup.addResolver() to register, or setup.addTerminalResolver() to register and stop assembly.
  • StandardProvider enum encodes the full chain precedence including per-profile-source slots, matching the Credentials Provider Chain Search Precedence SEP exactly.
  • SharedConfigProvider parses config files and sets profile on setup; downstream profile providers read from setup.profile().
  • Terminal resolvers short-circuit assembly (env vars set → config never parsed, IMDS never created).
  • Actionable errors name the missing module when a source is detected but no provider claims its slot.
  • CachingIdentityResolver provides background refresh; invalidate() propagates through the chain to clear caches on auth failures.

TODO: STS, SSO, ECS, Login providers.


Example usage:

// build.gradle.kts
dependencies {
    implementation("software.amazon.smithy.java:aws-client-core:1.1.0")
    implementation("software.amazon.smithy.java:aws-credential-chain:1.1.0")
    implementation("software.amazon.smithy.java:aws-credentials-imds:1.1.0")
}
var client = CoffeeShopClient.builder()
        .addPlugin(new AwsCredentialChainPlugin())
        .build();

Credentials resolve automatically in order:

  1. JVM system properties (aws.accessKeyId) — terminal
  2. Environment variables (AWS_ACCESS_KEY_ID) — terminal
  3. Config file static/session keys — terminal
  4. Config file credential_process
  5. EC2 IMDS (if aws-credentials-imds is added)

On Lambda, only aws-client-core is needed (env vars are terminal, nothing else is constructed):

dependencies {
    implementation("software.amazon.smithy.java:aws-client-core:1.1.0")
    implementation("software.amazon.smithy.java:aws-credential-chain:1.1.0")
}

@mtdowling mtdowling force-pushed the aws-auth branch 2 times, most recently from c7e4fbe to 3fae9c4 Compare May 11, 2026 20:59
@mtdowling mtdowling marked this pull request as draft May 12, 2026 14:59
@mtdowling mtdowling marked this pull request as ready for review May 12, 2026 19:54
Comment thread aws/aws-credential-chain/README.md
@mtdowling mtdowling force-pushed the aws-auth branch 7 times, most recently from 4a35179 to 459b9c4 Compare May 26, 2026 16:24
@mtdowling mtdowling requested a review from sugmanue May 26, 2026 17:54
mtdowling added 16 commits May 28, 2026 12:30
Adds support for loading AWS credentials from shared config/credentials
files and assembling them into a pluggable credential provider chain.

New modules:
- aws-config: SEP-conformant INI parser, profile data model,
  AwsConfigCredentialSource sealed types, handler SPI with ServiceLoader
  discovery, and built-in handlers for static keys, session keys, and
  credential_process.
- aws-credential-chain: Credential provider chain with builtin slots,
  Before/After relative ordering, SPI-based provider discovery, cheap
  environment detection, and actionable error messages when
  implementation modules are missing.

Changes to existing modules:
- auth-api: Add CachingIdentityResolver with async background refresh,
  static stability support, injectable Clock and
  ScheduledExecutorService. Add invalidate() default method to
  IdentityResolver interface. Will be used by STS, SSO, etc.
- aws-client-core: Add AwsCredentialChainPlugin ClientPlugin, register
  EnvironmentCredentialProvider and SystemPropertiesCredentialProvider
  as chain sources via SPI. Both now read AWS_ACCOUNT_ID / aws.accountId
  per the account ID SEP.
- settings.gradle.kts: Include new modules.

Architecture overview:
- Data model (aws-config) is separated from resolution policy (chain).
- Credential sources are detected cheaply from profile properties
  without needing implementation modules (STS, SSO, IMDS).
- Handlers are discovered via ServiceLoader; missing handlers produce
  errors naming the dependency to add.
- Chain ordering uses a fixed enum for builtins and simple Before/After
  insertion for third-party providers.
- CachingIdentityResolver provides background refresh with a shared
  ScheduledExecutorService passed via ProviderContext.
- invalidate() propagates through the chain to force credential refresh
  on auth failures.

TODO items: add SSO, STS, IMDS, ECS, etc.
Allowing custom provider relative positioning around only builtins
simplifies the design, removes the possibility of cycles, and removes
the need for a topo sort. Also removing aliases.
Automatically add credential chain runtime plugin if AWS auth is
used.
Reduces chain to a single, flat chain of fixed slots, including
config based credentials.
Compat shim for the four services (s3, ses, eventbridge,
cloudfront-keyvaluestore) whose endpoint rule sets emit an authSchemes
property that overrides or replaces the model-driven auth scheme. New
services should use a custom AuthSchemeResolver instead - endpoint-
driven selection is deprecated for new onboards.

AwsRulesExtension.extractEndpointProperties now reads the authSchemes
list from the raw endpoint property map and converts each entry into an
EndpointAuthScheme on the resolved Endpoint, with signingName,
signingRegion, signingRegionSet, and disableDoubleEncoding stored under
typed keys in the new EndpointAuthSchemeSettings class. The conversion
is content-keyed memoized in a ConcurrentHashMap so steady-state cost
collapses to one map lookup; bounded growth (S3 has ~10 unique entries
across its whole rule set).

ClientPipeline.applyEndpointAuthSchemeOverrides extends to swap
schemes, not just merge property overrides. Iterate the emitted entries
and pick the first whose ID is in supportedAuthSchemes; on a match
merge property overrides; on a different scheme re-resolve identity and
apply overrides; throw CallException if no entry is supported. The
second identity resolution is unavoidable in this ordering (identity
must be available before endpoint resolution because endpoints can
depend on account ID), and only fires when the endpoint actually
overrides - i.e. effectively only on S3.

SigV4Signer now reads EndpointAuthSchemeSettings.SIGNING_NAME and
SIGNING_REGION first, falling back to the scheme-level keys
(SigV4Settings.SIGNING_NAME, RegionSetting.REGION). Endpoint override
wins; auth scheme value is the default.

DISABLE_DOUBLE_ENCODING and SIGNING_REGION_SET keys are defined but not
yet consumed; the signer already does single-encoding only, so
disableDoubleEncoding=true is effectively the current behavior.
SigV4a (which would consume signingRegionSet) is not yet implemented.

Adds two ClientPipelineTest cases: scheme swap (resolver picks A,
endpoint emits B, B's signer runs with merged overrides) and unknown
scheme name (endpoint emits an unsupported scheme; CallException with
the scheme ID in the message).
New module aws/aws-sigv4-s3express implements the auth scheme S3 Express
endpoint rules emit on directory-bucket data-plane requests. Required
because S3 Express credentials are bucket-scoped session creds (not STS
creds) that go in x-amz-s3session-token, not x-amz-security-token.

Pieces:
- S3ExpressIdentity: AwsCredentialsIdentity + sessionToken from
  CreateSession
- S3ExpressAuthScheme: schemeId aws.auth#sigv4-s3express; lazy-caches
  the identity provider per base resolver
- S3ExpressSigner: pre-sets x-amz-s3session-token then delegates to
  SigV4 with a token-stripped wrapper identity so the inner signer
  won't write x-amz-security-token. The session header is included in
  the canonical request (the inner signer sees it on the request) so
  the wire signature covers it.
- S3ExpressIdentityProvider: reads BUCKET from request context, gets
  base credentials, looks up a per-(bucket, base creds)
  CachingIdentityResolver. Each per-bucket resolver does its own
  refresh-ahead via the existing CachingIdentityResolver helper.
- S3ExpressIdentityCache: ConcurrentHashMap + per-entry volatile
  lastAccess tick; reads are lock-free; writes/evictions take a
  ReentrantLock and run an O(N) scan for the LRU victim. MAX_SIZE=25,
  matching v2.
- S3ExpressIdentityKey: manual equals/hashCode on bucket +
  accessKeyId + secretAccessKey + sessionToken. expirationTime and
  accountId are intentionally ignored so token rotations that produce
  the same key material reuse the cached session.
- CreateSessionCallback: functional interface; the caller wires it to
  their generated S3 client's createSession() since this module
  doesn't depend on any concrete client.
- S3ExpressContext.BUCKET: per-call context key set by an interceptor
  before auth runs.
- S3ExpressFeatureId.S3_EXPRESS_BUCKET ("J"): emitted to
  CallContext.FEATURE_IDS on successful identity resolution per the
  S3 Express SEP / business-metric registry.

Also adds two endpoint setting keys in S3EndpointSettings:
S3_DISABLE_EXPRESS_SESSION_AUTH and S3_USE_EXPRESS_CONTROL_ENDPOINT.
Not registered as builtins because S3 declares those rule-set
parameters as plain context params, not as @builtIn-tagged ones, and
the rule defaults to false via coalesce.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants