You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[APS-22894] Cypress 16: TestHub and Accessibility support files read plugin values via Cypress.expose() (Cypress.env removed) - #1206
Cypress 16.0.0 removed Cypress.env() from the browser. The TestHub (Test Observability) and Accessibility support files this CLI injects read their flags with Cypress.env() inside synchronous Cypress.on(...) / Cypress.Commands.overwrite(...) hooks, so on Cypress 16 every command hits CypressError: Cypress.env() was removed in Cypress version 16.0.0 and TestHub reporting and a11y scans stop. TestHub is the default dashboard for many customers, so this has to work on 16 (Karan, 23 Oct).
This PR makes both features work on Cypress 15.x and 16.x without a version switch:
Plugins (bin/testObservability/plugin/index.js, bin/accessibility-automation/plugin/index.js): every value the support file reads is written to config.expose[key] as well as config.env[key] (exposeToBrowser helper). config.expose is read by Cypress.expose() from Cypress 15.10 onward and is accepted without error by older 15.x.
Support files (bin/testObservability/cypress/index.js, bin/accessibility-automation/cypress/index.js): the 22 Cypress.env(key) reads go through bstackBrowserValue(key), which returns Cypress.expose(key) when that API exists and the value is set, else falls back to Cypress.env(key) inside a try/catch (so the removed API on 16 never throws). Same approach Percy used for percy-cypress in PPLT-6234.
No behaviour change for customers on Cypress ≤ 15.x: config.env is still populated and Cypress.env() still answers. Values exposed are the same ones already placed in env today (flags, extension path, OS, TestHub uuid/jwt/ports, include/exclude tags), so there is no new exposure in the browser; the TestHub JWT is already in env. If the SDK team prefers to keep the JWT out of expose, that one key can be routed via cy.task instead.
Verified against real Cypress
Probe spec (returns config.expose from setupNodeEvents, reads both APIs in the browser):
Cypress
config.expose from setupNodeEvents
Cypress.expose()
Cypress.env()
15.8.2
accepted
undefined (API absent)
works
15.21.1
accepted
works
works (deprecation warning)
16.2.0
accepted
works
throws CypressError: Cypress.env() was removed in Cypress version 16.0.0
Unit tests: 5 new (plugins mirror every key into config.expose; existing customer expose kept) plus the existing accessibility afterEach guard, which loads the modified support file under a stubbed Cypress: 11 passing.
Pending: staging run with testObservability: true and accessibility: true on cypress_version: 16.2.0 once the 16.2.0 binaries exist (APS-22894 Stage 4), and on 15.21.1 as the regression check.
Ownership
Parked with SDK/TestHub for review and release, per Karan. Supersedes the gate in #1205 (which disabled both features on 16); #1205 can be closed once this is approved.
JIRA
APS-22894 — Cypress 16.2.0 release on Automate desktop (Mac + Windows), BAU
Accessibility and test observability settings are now available to browser-based tests, including BrowserStack logging and customer-configured scan limits.
Bug Fixes
Browser-based tests can retrieve settings from either exposed browser configuration or environment configuration, helping preserve existing behavior when a setting is unavailable in one location.
Existing exposed values are preserved when settings are synchronized, and unset scan limits remain unset.
Some coding-guideline sources were skipped or only partly applied; the reason is listed next to each one. Other guidelines were applied as usual. On a public repository, sources from your configuration are listed without their names.
Skipped guideline sources: 1
a configured source — couldn't be fetched
Walkthrough
The accessibility and test observability plugins now expose selected configuration through config.expose. Their Cypress code reads those values through Cypress.expose() when available and falls back to Cypress.env().
Both plugins write selected values to config.env and config.expose. The accessibility plugin forwards scan timeout and circuit-limit values when present, without replacing existing exposed values. Unit tests cover exposure and value preservation.
Accessibility logging, scan selection, timeout, circuit-limit, OS, and TestHub settings now use bstackBrowserValue. Existing defaults and scan-selection rules remain unchanged.
This change keeps TestHub reporting and accessibility scans working on Cypress 16 while retaining the Cypress 15 fallback. The JWT is browser-readable on the base revision too, so this change does not add new exposure. The two test-hygiene notes about unrestored environment variables are optional cleanup. The staging runs on Cypress 16.2.0 and 15.21.1 are still pending.
Security architecture risk: Moderate · up to 62684
The compatibility update carries an existing browser credential-exposure pattern into Cypress 16. The same credential already reached tested pages on earlier versions; broader permissions or cross-tenant exposure have not been demonstrated. Credential scope and lifecycle protections remain uncertain.
Retained concerns
No architecture-level concerns identified.
Security review details
Security Blast Radius
inferred — A script executing in a tested page can observe the token-bearing A11Y_SAVE_RESULTS event when the eligible accessibility hook reaches result saving. This does not require direct access to Cypress.expose. The exposed value is the active TestHub JWT, but its maximum authorized build, project, tenant, or service scope and usable lifetime are not established by the inspected code.
Security Findings and Attack Paths
observed — The retained sensitive-data-exposure finding concerns the TestHub JWT exported at plugin setup and subsequently delivered in the tested window's event payload. Base/head comparison establishes that the token-bearing page event predates this PR on working legacy runtimes. The expose-first path carries that existing condition into the intended Cypress 16 execution path; increased token authority or additional tenant reach has not been demonstrated.
Trust Boundaries and Controls
observed — The local build request is configured with the user's authentication credentials before response metadata is propagated toward the browser. Result saving checks HTTP(S), scanner-element readiness, and accessibility eligibility, but these checks do not authenticate listeners on the tested window. The event contains the JWT without a recipient-specific confidentiality control in this code.
Resilience and Maintainability Implications
inferred — Normal setup preserves the JWT/build-identifier pairing through synchronous assignments. Existing failure handlers replace process-level identity values with string null sentinels, while browser configuration is a setup-time snapshot. No introduced identity-mixing defect was demonstrated, but invalidation of an existing browser snapshot after failure, interruption, concurrent builds, or process/config reuse remains unverified.
Hardening Proposals
proposed — Consider keeping the TestHub credential in Node and using a narrowly scoped task for authenticated result submission, leaving only non-secret identifiers in browser configuration and page events. If the scanner protocol requires browser-side authority, evaluate a short-lived, narrowly scoped capability instead. Either approach requires an explicit scanner-protocol and authorization design rather than merely changing the configuration API.
Pre-merge checks: 4
Check name
Status
Explanation
Description Check
Check skipped - CodeRabbit’s high-level summary is enabled.
Title check
The title names the main change: the TestHub and Accessibility support files now read plugin values through Cypress.expose(), and Cypress.env is removed. The raw summary confirms that bstackBrowserVal…
Linked Issues check
Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check
Check skipped because no linked issues were found for this pull request.
Full details: Title check
Explanation
The title names the main change: the TestHub and Accessibility support files now read plugin values through Cypress.expose(), and Cypress.env is removed. The raw summary confirms that bstackBrowserValue reads from Cypress.expose() with a Cypress.env() fallback, and that the plugins mirror values into config.expose. The title is specific to Cypress 16 support, which is the stated objective. It is long, but it still summarizes the primary change.
Finishing Touches
Generate docstrings
Commit to this branch
Create a new PR
Generate unit tests (beta)
Commit to this branch
Create a new PR
Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
A rabbit checks the settings bright Through exposed values in morning light If none are there, env lends a hand The scans proceed as fields were planned I twitch my nose and hop away With browser values set today
…IRCUIT_LIMIT to config.expose
Customers set these in their Cypress env config. Cypress 16 no longer passes
config.env to the browser, so the support file fell back to the defaults
(25000 ms / 3). Copy them into config.expose when set, without overriding a
value the customer already exposed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.
Restore the process flags set by the plugin.
When these tests invoke the plugin, it sets BROWSERSTACK_LOGS and BROWSERSTACK_ACCESSIBILITY_PLUGIN_LOADED. This afterEach restores neither flag. Later suites can therefore observe a plugin-enabled state that their tests did not establish. Save and restore both flags with the other process environment values.
Unverified finding
Verification ran but could not confirm this finding. It is shown for review, not as a verified issue.
Restore BROWSERSTACK_O11Y_LOGS after the tests.
When this test invokes the plugin, the plugin sets process.env.BROWSERSTACK_O11Y_LOGS to true. The suite never restores its prior value, so later tests can observe observability as enabled. Save the original value in a hook and restore it after each test.
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 6 remain after this review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Summary
Cypress 16.0.0 removed
Cypress.env()from the browser. The TestHub (Test Observability) and Accessibility support files this CLI injects read their flags withCypress.env()inside synchronousCypress.on(...)/Cypress.Commands.overwrite(...)hooks, so on Cypress 16 every command hitsCypressError: Cypress.env() was removed in Cypress version 16.0.0and TestHub reporting and a11y scans stop. TestHub is the default dashboard for many customers, so this has to work on 16 (Karan, 23 Oct).This PR makes both features work on Cypress 15.x and 16.x without a version switch:
bin/testObservability/plugin/index.js,bin/accessibility-automation/plugin/index.js): every value the support file reads is written toconfig.expose[key]as well asconfig.env[key](exposeToBrowserhelper).config.exposeis read byCypress.expose()from Cypress 15.10 onward and is accepted without error by older 15.x.bin/testObservability/cypress/index.js,bin/accessibility-automation/cypress/index.js): the 22Cypress.env(key)reads go throughbstackBrowserValue(key), which returnsCypress.expose(key)when that API exists and the value is set, else falls back toCypress.env(key)inside a try/catch (so the removed API on 16 never throws). Same approach Percy used for percy-cypress in PPLT-6234.No behaviour change for customers on Cypress ≤ 15.x:
config.envis still populated andCypress.env()still answers. Values exposed are the same ones already placed inenvtoday (flags, extension path, OS, TestHub uuid/jwt/ports, include/exclude tags), so there is no new exposure in the browser; the TestHub JWT is already inenv. If the SDK team prefers to keep the JWT out ofexpose, that one key can be routed viacy.taskinstead.Verified against real Cypress
Probe spec (returns
config.exposefromsetupNodeEvents, reads both APIs in the browser):config.exposefrom setupNodeEventsCypress.expose()Cypress.env()CypressError: Cypress.env() was removed in Cypress version 16.0.0Unit tests: 5 new (plugins mirror every key into
config.expose; existing customerexposekept) plus the existing accessibility afterEach guard, which loads the modified support file under a stubbedCypress: 11 passing.Pending: staging run with
testObservability: trueandaccessibility: trueoncypress_version: 16.2.0once the 16.2.0 binaries exist (APS-22894 Stage 4), and on 15.21.1 as the regression check.Ownership
Parked with SDK/TestHub for review and release, per Karan. Supersedes the gate in #1205 (which disabled both features on 16); #1205 can be closed once this is approved.
JIRA
🤖 Generated with Claude Code
Summary by CodeRabbit