feat(ai): Add ModelMetadata config with context size and utilization

[constantinius]

Closes https://linear.app/getsentry/issue/TET-2220/relay-implement-context-window-usage-per-span

Introduce a new llmModelMetadata global config that extends the existing model cost data with context window size. The new ModelMetadata struct replaces ModelCosts throughout the normalization pipeline, with ModelCosts only retained for backwards-compatible deserialization on GlobalConfig. This config and its schema is introduced is introduced in getsentry/sentry#112656

When ai_model_metadata is present it is used entirely; otherwise ai_model_costs is converted to the new format as a fallback.

For each AI span, if the model has a configured context size, set gen_ai.context.window_size and compute gen_ai.context.utilization as total_tokens / context_window_size. These fields were introduced with getsentry/sentry-conventions#315

Co-Authored-By: Claude [email protected]

[linear-code]

TET-2220 Relay: implement context window usage per span

[sentry]

Bug: The NormalizeSpanConfig is cloned for every span, causing a potentially expensive HashMap clone in a tight loop, which may degrade performance under high load.
_{Severity: MEDIUM}

Suggested Fix

Pass NormalizeSpanConfig by reference instead of by value to the normalization function. This will avoid the expensive HashMap clone for every span, replacing it with a cheap reference copy.

Prompt for AI Agent

Review the code at the location below. A potential bug has been identified by an AI
agent.
Verify if this is a real issue. If it is, propose a fix; if not, explain why it's not
valid.

Location: relay-event-normalization/src/event.rs#L2329

Potential issue: The `NormalizeSpanConfig` struct, which contains an owned
`Option<ModelMetadata>` wrapping a `HashMap`, is cloned for every individual span during
processing. This occurs within a loop that iterates over all spans in an envelope.
Cloning a `HashMap` is an expensive operation that allocates new memory and copies all
its elements. When processing envelopes with a high volume of spans and configured model
metadata, this repeated cloning can lead to significant CPU and memory pressure, causing
performance degradation.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c5f4e71. Configure here.}

[cursor]

Per-span deep clone of ModelMetadata HashMap

Low Severity

The ai_model_metadata field changed from Option<&'a ModelCosts> (a borrowed reference) to Option<ModelMetadata> (an owned value containing a HashMap<Pattern, ModelMetadataEntry>). Since NormalizeSpanConfig is .clone()'d for every span item in the envelope (line 106), the entire HashMap — including heap-allocated Pattern keys with internal Strings — is deep-cloned per span. The previous code only cloned a pointer. For envelopes with many spans this introduces unnecessary allocations.

Additional Locations (1)

• relay-server/src/services/processor/span.rs#L282-L283

 

^{Reviewed by Cursor Bugbot for commit c5f4e71. Configure here.}