Feat: Enable automatic AWS credential discovery via provider chain

[mdgreenwald]

Add support for AWS credential provider chain across all AWS integrations,
eliminating the need for long-lived IAM access keys in production environments.

Changes:

• Enable aws_credentials library at runtime (remove runtime: false)
• Remove self-hosted mode restriction for task role credentials
• Add task role support to Kafka sinks with AWS MSK IAM authentication
• Add task role support to HttpPushSqsPipeline (BroadwaySQS)
• Update all AWS sink validations to support use_task_role flag

This enables automatic credential discovery from:

• Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
• AWS credentials file (~/.aws/credentials)
• IRSA/EKS Pod Identity (web identity tokens)
• ECS task credentials
• EC2 instance metadata

Benefits:

• Eliminates security risk of long-lived IAM keys
• Supports modern Kubernetes auth patterns (IRSA, Pod Identity)
• Automatic credential rotation for temporary credentials
• Follows AWS security best practices
• Fully backward compatible (use_task_role defaults to false)

Affected services: SQS, SNS, Kinesis, Kafka (MSK IAM), HttpPushSQS

[mdgreenwald]

Claude has this to say about how to idiomatically solve this problem:

ex_aws is the de facto Elixir AWS library and supports pluggable credential providers. IRSA works by EKS injecting two environment variables into your pod — AWS_ROLE_ARN and AWS_WEB_IDENTITY_TOKEN_FILE — plus a projected JWT at that file path. Your application needs to read the token file, call sts:AssumeRoleWithWebIdentity, and refresh the resulting temporary credentials before they expire.

ex_aws_sts provides the AssumeRoleWithWebIdentity action directly. The implementation pattern is:
Write a small GenServer (fits naturally in your OTP supervision tree) that reads the token file on startup and on a refresh timer (tokens default to 24h but you should refresh well ahead of expiry)
Call ExAws.STS.assume_role_with_web_identity/3 with the token and role ARN from the environment

Store the resulting AccessKeyId, SecretAccessKey, and SessionToken in process state or ETS. Configure ex_aws to use a custom credential provider that pulls from that state.

This is the most idiomatic Elixir approach. You stay entirely within the BEAM, leverage OTP supervision for credential refresh resilience, and avoid any external sidecar dependencies. The linked Medium article by Sachin Joshi covers this pattern precisely.

[mdgreenwald]

#2054