ci: add scheduled public API e2e tests (CM-968) - #4398
Conversation
There was a problem hiding this comment.
Pull request overview
Adds scheduled Public API end-to-end regression testing against the isolated api-e2e deployment.
Changes:
- Adds the scheduled deployment and test workflow.
- Adds HTTP-based fixture seeding and resource suites.
- Registers and documents the new e2e architecture.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
.github/scripts/public-api-e2e-tests.sh |
Implements the Public API test suite. |
.github/workflows/api-e2e-tests.yml |
Deploys and tests api-e2e nightly. |
scripts/builders/backend.env |
Maps api-e2e to the backend image. |
docs/adr/0011-api-e2e-test-architecture.md |
Documents runtime and isolation decisions. |
docs/adr/0012-api-e2e-test-suite-design.md |
Documents suite organization. |
docs/adr/README.md |
Indexes the new ADRs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
94a8eb5 to
71617ab
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
.github/scripts/public-api-e2e-tests.sh:113
- The Auth0 request is also unbounded, so a stalled token endpoint can consume the entire 90-minute job timeout before any test runs. Give this curl call a short connect and total timeout as well.
response="$(curl -sS -X POST "${AUTH0_STAGING_ISSUER}/oauth/token" \
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (1)
.github/workflows/api-e2e-tests.yml:78
- A job-level timeout cancels the whole
e2e-testsjob, so this later step is never reached when the 90-minute limit is hit. That leaves a hung build/deploy/test without the Slack alert promised by the ADR. Move notification into a separate job thatneeds: e2e-testsand runs whenalways() && needs.e2e-tests.result != 'success', so failures and timeouts are both reported.
if: failure()
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.
Comments suppressed due to low confidence (4)
.github/scripts/public-api-e2e-tests.sh:290
- This extraction can abort the script after the preceding soft
checkrecords a failure: withset -e, invalid or empty response JSON makes the assignment return nonzero. Keep extraction non-fatal so later identity cases still run and the final failure count is reported.
identity_id="$(jq -r '.id' <<<"$BODY")"
.github/scripts/public-api-e2e-tests.sh:341
- A non-JSON failure response here bypasses the suite’s soft-check behavior because this
jqassignment exits nonzero underset -e. Make the ID extraction tolerant; subsequent requests will then be counted as failures instead of terminating the run.
acme_we="$(jq -r '.id' <<<"$BODY")"
.github/scripts/public-api-e2e-tests.sh:354
- This unguarded ID extraction can terminate the suite after a soft
checkfailure when the response body is empty or not JSON. Guardjqso the remaining work-experience cases and final summary still execute.
globex_we="$(jq -r '.id' <<<"$BODY")"
.github/scripts/public-api-e2e-tests.sh:431
- Because the preceding status assertion is soft, a malformed failure body should not make this
jqassignment abort the whole script underset -e. Use tolerant extraction so the DELETE/GET cases record failures and the run prints its aggregate result.
doomed_we="$(jq -r '.id' <<<"$BODY")"
Signed-off-by: Yeganathan S <63534555+skwowet@users.noreply.github.com>
Summary
Adds a scheduled Public API e2e suite against the dedicated
api-e2estack (crowd_api_e2e), so we can catch contract regressions onmainwithout hand-running against shared staging.Changes
.github/scripts/public-api-e2e-tests.sh(HTTP seed + resource suites).github/workflows/api-e2e-tests.yml(deployapi-e2efrommain, then run the suite)api-e2einscripts/builders/backend.env