Skip to content

Clarifies which LLMs we support for Obs and Sec throughout AI docs#5629

Open
benironside wants to merge 4 commits intomainfrom
internal-731-obs-performance-matrix-applicability
Open

Clarifies which LLMs we support for Obs and Sec throughout AI docs#5629
benironside wants to merge 4 commits intomainfrom
internal-731-obs-performance-matrix-applicability

Conversation

@benironside
Copy link
Contributor

@benironside benironside commented Mar 25, 2026

Fixes /elastic/docs-content-internal/issue/731

The problem was that we weren't always making it clear that certain LLMs are supported for certain use cases, while others aren't, or don't perform well. In other words, we may have oversold the extent to which users can bring any model.

The purpose of this PR is to make it clear to users who are learning about our AI powered features that we have tested models, we vouch for them, and we can only vouch for them.

It:

  • Adds references to the LLM performance matrices into pages that discuss AI Chat
  • Clarifies that the Observability LLM performance matrix applies equally to Agent Builder and to AI Assistant
  • Updates Security LLM performance matrix to match Obs updates
  • Disambiguates the Obs and Sec performance matrix page titles (now "...for Observability" / "...for Security")

Generative AI disclosure

To help us ensure compliance with the Elastic open source and documentation guidelines, please answer the following:

  1. Did you use a generative AI (GenAI) tool to assist in creating this contribution?
  • [x ] Yes
  • No
  1. If you answered "Yes" to the previous question, please specify the tool(s) and model(s) used (e.g., Google Gemini, OpenAI ChatGPT-4, etc.).

Tool(s) and model(s) used: Claude 4.6, Cursor
-->

Updated the AI Assistant section to clarify the integration with large language model (LLM) providers, emphasizing the use of connectors and providing a more specific reference to the LLM performance matrix for observability. This enhances user understanding of available models and their performance ratings.
@benironside benironside self-assigned this Mar 25, 2026
@benironside benironside requested review from a team as code owners March 25, 2026 23:35
@github-actions
Copy link
Contributor

github-actions bot commented Mar 25, 2026

Vale Linting Results

Summary: 1 suggestion found

💡 Suggestions (1)
File Line Rule Message
explore-analyze/ai-features/llm-guides/llm-connectors.md 32 Elastic.WordChoice Consider using 'can, might' instead of 'may', unless the term is in the UI.

The Vale linter checks documentation changes against the Elastic Docs style guide.

To use Vale locally or report issues, refer to Elastic style guide for Vale.

@benironside benironside changed the title Internal 731 obs performance matrix applicability Clarifies which LLMs we support for Obs and Sec throughout AI docs Mar 26, 2026
Copy link
Member

@florent-leborgne florent-leborgne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice additions! How can we clarify the paths to the right performance pages too for the Search solution and the various AI features it includes? For example, is AI Assistant for Obs/Search still relevant for search? If not, is there a dedicated performance page for AB models (there's a page yes, but what is it relevant for?) cc @leemthompo in case you know more there.

I'm thinking that in addition to this, we may benefit from an orientation page if this is a regular concerns for our users like "LLM performance in the Elasticsearch platform" where we could clearly say that different models perform differently based on what feature/solution they're used and then link to the various places. Right now I feel like we have a (too) complex and incomplete web of interlinking which I'm not sure is fixing the underlying issue.

Also, similar to what is being done for EIS, specifying the (minimum) stack version for the models could be a good idea too.

Lastly, support folks seem to have particular needs/interests there, we should include them in reviewing what we produce around this issue IMO to make sure that what we deliver matches their expectations too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants