Context
On 2026-06-08, the feature-flag-data service experienced a 5-minute outage caused by istio-proxy CPU saturation. During this window, any feature flag check using the enabled_or_raise (a.k.a. *_or_raise) variant raised an exception instead of gracefully falling back to a default value.
This caused cascading 5xx errors across Web, REST API, GraphQL API, and Webhooks — impacting external customers for up to 12 minutes.
~430 feature flags across the monolith were identified as using the raising variant without a safe default. This list was defined based on monolith-side metrics — evaluation of these feature flags was unsuccessful during the time of the incident. Your service owns the flag(s) listed below.
ℹ️ If feature flags listed below were already deleted or the calling methods already use default values, you don't need to do anything — just close this issue.
📓 Default values for feature flag checks guide
🚨 Incident review
What you need to do
For each feature flag listed below, do one of the following:
-
Add a default value — Switch the check from enabled_or_raise / *_or_raise to the default-value variant (enabled? / enabled_with_details?) with an appropriate fallback (typically false). This ensures that when the feature-flag service is unavailable, your code path degrades gracefully instead of raising an exception.
-
Graduate (remove) the flag — If the flag is fully shipped and no longer needed, remove the flag check entirely and hard-code the enabled behavior. Then mark the flag as graduated in DevPortal.
⚠️ Do not leave the raising variant in place. If you cannot add a default value or graduate the flag, please reach out to the #feature-management channel for guidance.
🔎 Can't find a static callsite? Some flags are checked via dynamically-generated names or indirect helpers, so a grep for the literal flag name may not locate the check. Trace where the flag is evaluated (e.g. a name built at runtime, a shared wrapper, or a metaprogrammed helper) and apply a default there.
Code change examples
Before (raising variant — causes errors during outage):
# This raises Vexi::Errors::CircuitBreakerOpenError when feature-flag-data is down
enabled_or_raise(:my_flag, actor: current_user)
After (default-value variant — graceful degradation):
# Returns false (or your chosen default) when feature-flag-data is unavailable
enabled?(:my_flag, actor: current_user, default: false)
⚠️ Choose your default carefully. For most flags, default: false (feature off) is safest. For flags protecting critical paths that should stay enabled, use default: true.
Why this matters
- The
or_raise variant converts a transient infrastructure issue into a customer-facing error.
- During the 2026-06-08 incident, ~75.5M feature flag evaluations raised exceptions, causing ~494K web 5xx errors and ~804K REST API 5xx errors in just 5 minutes.
- Using the default-value variant provides circuit-breaker resilience — your feature gracefully degrades instead of failing catastrophically.
Feature flags to migrate
Acceptance criteria
References
Context
On 2026-06-08, the feature-flag-data service experienced a 5-minute outage caused by istio-proxy CPU saturation. During this window, any feature flag check using the
enabled_or_raise(a.k.a.*_or_raise) variant raised an exception instead of gracefully falling back to a default value.This caused cascading 5xx errors across Web, REST API, GraphQL API, and Webhooks — impacting external customers for up to 12 minutes.
~430 feature flags across the monolith were identified as using the raising variant without a safe default. This list was defined based on monolith-side metrics — evaluation of these feature flags was unsuccessful during the time of the incident. Your service owns the flag(s) listed below.
📓 Default values for feature flag checks guide
🚨 Incident review
What you need to do
For each feature flag listed below, do one of the following:
Add a default value — Switch the check from
enabled_or_raise/*_or_raiseto the default-value variant (enabled?/enabled_with_details?) with an appropriate fallback (typicallyfalse). This ensures that when the feature-flag service is unavailable, your code path degrades gracefully instead of raising an exception.Graduate (remove) the flag — If the flag is fully shipped and no longer needed, remove the flag check entirely and hard-code the enabled behavior. Then mark the flag as graduated in DevPortal.
Code change examples
Before (raising variant — causes errors during outage):
After (default-value variant — graceful degradation):
Why this matters
or_raisevariant converts a transient infrastructure issue into a customer-facing error.Feature flags to migrate
primer_react_select_panel_fullscreen_on_narrowAcceptance criteria
enabled_or_raise/*_or_raisefor the listed flagsReferences