Relationships
#1805 @swamp/s3-datastore disables IMDS on plain EC2, so an attached instance profile cannot resolve
Opened by keeb · 8/24/2026· Shipped 8/25/2026
Found while documenting AWS credential resolution for #1248.
Problem
datastore/s3/extensions/datastores/_lib/s3_client.ts:347-353 disables the AWS SDK instance metadata credential lookup:
if (
!Deno.env.get("AWS_EC2_METADATA_DISABLED") &&
!Deno.env.get("AWS_CONTAINER_CREDENTIALS_RELATIVE_URI") &&
!Deno.env.get("AWS_CONTAINER_CREDENTIALS_FULL_URI")
) {
Deno.env.set("AWS_EC2_METADATA_DISABLED", "true");
}The comment above it says "When not running in a container environment (ECS/EKS)" and "Off-cloud, the metadata endpoint is unreachable". Avoiding the IMDS stall off-cloud is the right goal; it is half of the #803 fix.
But there is no EC2 detection. The only things that suppress the disable are the two ECS/EKS container-credential variables. On a plain EC2 instance drawing credentials from an attached instance profile, neither is set, so the datastore disables the exact credential source it is meant to use. IMDS is where instance-profile credentials come from on EC2; ECS and EKS use the container endpoint instead, which is why only those two cases are covered.
Expected outcome on such a host: credentials do not resolve, and the #803 preflight fails in about 3 seconds with either a CredentialsProviderError classified as session-expired, or:
Credential preflight timed out after 3000ms — verify that AWS credentials are configured (AWS_ACCESS_KEY_ID, AWS_PROFILE, or attached IAM role) and that the credential source is responsivewhich names the attached IAM role as a thing to check, having just disabled it.
This is from reading the code, not from a reproduction on EC2. Worth confirming on a real instance before fixing, because a careless change here re-introduces the off-cloud stall that #803 removed.
Why it has stayed quiet
Anyone running swamp on EC2 with a profile or static env credentials is unaffected, because those providers resolve earlier in the chain than IMDS. It only bites a host whose only credential source is the instance profile.
Documented behaviour disagrees
Both the extension description and content/manual/reference/datastore-configuration.md list "IAM role attached to the instance or task" as a supported credential source. On plain EC2 that is currently false without an explicit opt-out.
Workaround
Set AWS_EC2_METADATA_DISABLED=false. The guard tests truthiness of the string, so any value prevents swamp from setting it to true, and the SDK reads "false" as not-disabled. This is now written down at /manual/how-to/authenticate-to-aws-with-sso and in both extension descriptions, but it should not be necessary.
Suggested fix
Probe IMDS with a short timeout rather than disabling it outright, or gate the disable on an actual EC2 check. Whichever is chosen belongs in the shared credential module proposed in #1804 rather than only in the s3 client, since the same off-cloud stall affects the vault and the generated model family, neither of which carries this code at all.
Upstream repository: https://github.com/systeminit/swamp-extensions
Shipped
Click a lifecycle step above to view its details.
Sign in to post a ripple.