When using the AWS CLI against RunPod's S3-compatible API, any operation that requires paginated ListObjectsV2 calls fails because the endpoint returns the same ContinuationToken on consecutive pages. The AWS SDK detects this as an infinite loop and aborts.
fatal error: Error during pagination: The same next token was received twice:
{'ContinuationToken': ''}
The Base64-decoded token reveals it corresponds to an object key deep in the bucket, suggesting the pagination cursor is not advancing.
Affected operations (all fail):
- aws s3 ls --recursive s3:///
- aws s3 sync s3:/// ./local/
- aws s3 cp --recursive s3://// ./local/
Working operations:
- aws s3 ls s3:/// (non-recursive, uses Delimiter='/')
- aws s3 ls s3://// (non-recursive with prefix + delimiter)
- aws s3 cp s3:///path/to/specific-file ./file (single object GetObject)
Steps to reproduce:
Upload enough objects to trigger pagination (e.g. >1000 keys)
Then run:
aws s3 ls --recursive --region
--endpoint-url https://s3api-.runpod.io
s3:///
Workaround: Download files individually using aws s3 cp without --recursive, since single-object GetObject calls work correctly.
Environment:
AWS CLI: 1.45.6
botocore: 1.43.6
RunPod S3 endpoint region: eu-cz-1
Bucket contains ~1000+ objects (mostly from a .vscode-server directory)
Generated with the help of a chat agent, hope this helps!
When using the AWS CLI against RunPod's S3-compatible API, any operation that requires paginated ListObjectsV2 calls fails because the endpoint returns the same ContinuationToken on consecutive pages. The AWS SDK detects this as an infinite loop and aborts.
fatal error: Error during pagination: The same next token was received twice:
{'ContinuationToken': ''}
The Base64-decoded token reveals it corresponds to an object key deep in the bucket, suggesting the pagination cursor is not advancing.
Affected operations (all fail):
Working operations:
Steps to reproduce:
Upload enough objects to trigger pagination (e.g. >1000 keys)
Then run:
aws s3 ls --recursive --region
--endpoint-url https://s3api-.runpod.io
s3:///
Workaround: Download files individually using aws s3 cp without --recursive, since single-object GetObject calls work correctly.
Environment:
AWS CLI: 1.45.6
botocore: 1.43.6
RunPod S3 endpoint region: eu-cz-1
Bucket contains ~1000+ objects (mostly from a .vscode-server directory)
Generated with the help of a chat agent, hope this helps!