Fix all sample CI failures - #272
Closed
peter-smith-phd wants to merge 2 commits into
Closed
Conversation
Every sample that calls awslocal aborts in CI before running any AWS
command:
File ".../awscli/s3transfer/__init__.py", line 140, in <module>
from botocore.vendored.requests.packages.urllib3.exceptions import (
ModuleNotFoundError: No module named 'botocore.vendored'
The cause is not a version pin. awscli 1.46.0 declares no botocore
dependency of its own, and the only other path to botocore is
awscli-local -> localstack-client -> boto3. The runner image already ships
Ubuntu's python3-boto3 1.34.46 in /usr/lib/python3/dist-packages, so pip
reports "Requirement already satisfied: boto3" and never fetches boto3 or
botocore from PyPI. Debian strips vendored code copies from its botocore
package, leaving no botocore/vendored/ directory, and awscli v1 imports
botocore.vendored.requests during startup patching.
Naming boto3 explicitly and passing -U makes pip install boto3/botocore
from PyPI into /usr/local/lib/python3.12/dist-packages, which precedes the
distro path on sys.path. Both parts are required: pip's default
upgrade-strategy leaves the already-satisfied distro package alone, so
naming boto3 without -U changes nothing.
Keeping the [ver1] extra is deliberate. lambda-xray/python-xray-sdk and
java-notification-app use awslocal cloudformation package/deploy, which
depends on the --s3-endpoint-url customization that awslocal's
patch_awscli_libs() adds to awscli v1.
Verified by reproducing the runner environment in an ubuntu:24.04
container (python3-boto3 from apt, then the workflow's pip command).
Before the change awslocal fails with the identical traceback; after it,
s3 mb, s3api list-buckets, sts get-caller-identity, and cloudformation
package + deploy all succeed against a live LocalStack, with the stack
reaching CREATE_COMPLETE.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both samples fail for unrelated reasons, neither caused by the sample code itself; both are drift in something the sample pulls in at build time. emr-serverless-python-dependencies The build now aborts with "process /bin/bash -c python3 -m venv $VIRTUAL_ENV did not complete successfully: exit code 1". The localstack/localstack:latest base image has moved to Debian forky/sid with Python 3.14, and Debian splits ensurepip out of the stdlib into a separate pythonX.Y-venv package, so python3 -m venv cannot bootstrap pip. The base image cannot simply be swapped: this build produces a venv that docker-compose mounts into the LocalStack container as PYSPARK_PYTHON at /tmp/environment, so its interpreter has to match the container's. Instead install the missing package, deriving the version from the image's own interpreter rather than hardcoding 3.14 so this keeps working as the base image moves on. This also covers a second latent break: poetry-plugin-bundle creates its own venv and would have hit the same wall. terraform-resources Creation of aws_api_gateway_rest_api fails with "unexpected state '', wanted target 'AVAILABLE'". The sample declares no required_providers block, so terraform init resolves the latest AWS provider on every run. Bisecting against LocalStack puts the break exactly at 6.57.0, which added a state waiter polling a field LocalStack's CreateRestApi/GetRestApi response does not return; 6.56.0 and everything before it are fine. Pin to ~> 6.56.0, matching the style already used by lambda-function-urls-javascript and emr-serverless-python-dependencies. This is a stopgap that hides the gap rather than closing it. Users on provider >= 6.57.0 still hit this against LocalStack outside this repo, so the API Gateway response is worth fixing upstream. Verified locally against a live LocalStack. The EMR image builds through poetry bundling and export, and the resulting bundle mounted at /tmp/environment inside localstack/localstack:latest runs and imports both typer and the jobs package. terraform-resources applies all 10 resources via its own make run, GetRestApis confirms auth-demo exists, and destroy removes all 10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
peter-smith-phd
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix test failures that occurred due to changes in third-party dependencies.
awslocalaborts on the runner's debundled distro botocoreemr-serverless-python-dependenciesensurepipout of the stdlibterraform-resources🤖 Generated with Claude Code