-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
docs(laravel): Add AI monitoring guide #18512
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
constantinius
wants to merge
2
commits into
master
Choose a base branch
from
constantinius/docs/laravel/ai-monitoring
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+235
−0
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
232 changes: 232 additions & 0 deletions
232
docs/platforms/php/guides/laravel/integrations/laravel-ai.mdx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,232 @@ | ||
| --- | ||
| title: Laravel AI | ||
| description: "Learn about using Sentry for Laravel AI agent monitoring." | ||
| --- | ||
|
|
||
| <Alert> | ||
|
|
||
| Laravel AI support is in beta. Test locally before using it in production. | ||
|
|
||
| </Alert> | ||
|
|
||
| This integration connects Sentry with the [Laravel AI](https://laravel.com/docs/ai) package. | ||
| The integration has been confirmed to work with `sentry/sentry-laravel` version `4.27.0` and above. | ||
|
|
||
| <AgentSkillsCallout skill="sentry-setup-ai-monitoring" platformName="PHP" /> | ||
|
|
||
| Once you've installed this SDK, you can use the [AI Agents Dashboards](https://sentry.io/orgredirect/organizations/:orgslug/dashboards/?filter=onlyPrebuilt&query=agents&sort=mostPopular) to understand what's going on with your AI agents. | ||
|
|
||
| Sentry AI Agents monitoring will automatically collect information about agents, tools, prompts, tokens, and models. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| - Laravel 12.x or later | ||
| - `sentry/sentry-laravel` version `4.27.0` or later | ||
| - An AI provider configured for Laravel AI | ||
| - <PlatformLink to="/tracing/">Tracing</PlatformLink> set up in your Sentry configuration | ||
|
|
||
| ## Install | ||
|
|
||
| Install the `sentry/sentry-laravel` package: | ||
|
|
||
| ```bash | ||
| composer require sentry/sentry-laravel | ||
| ``` | ||
|
|
||
| Install and configure `laravel/ai`: | ||
|
|
||
| ```bash | ||
| composer require laravel/ai | ||
| php artisan vendor:publish --provider="Laravel\Ai\AiServiceProvider" | ||
| php artisan migrate | ||
| ``` | ||
|
|
||
| Add your AI provider API keys to your `.env` file: | ||
|
|
||
| ```shell {filename:.env} | ||
| ANTHROPIC_API_KEY=your-anthropic-key | ||
| OPENAI_API_KEY=your-openai-key | ||
| ... | ||
| ``` | ||
|
|
||
| ## Configure | ||
|
|
||
| The Laravel AI integration is enabled automatically when both `sentry/sentry-laravel` and `laravel/ai` are installed and tracing is active. No additional configuration is needed. | ||
|
|
||
| Make sure tracing is enabled in your `.env`: | ||
|
|
||
| ```shell {filename:.env} | ||
| SENTRY_TRACES_SAMPLE_RATE=1.0 | ||
| ``` | ||
|
|
||
| To include LLM inputs and outputs (prompts, tool arguments, and responses), enable the `send_default_pii` option: | ||
|
|
||
| ```shell {filename:.env} | ||
| SENTRY_SEND_DEFAULT_PII=true | ||
| ``` | ||
|
|
||
| <Alert> | ||
|
|
||
| Sentry considers LLM and tool inputs/outputs as PII and doesn't include them by default. Set `send_default_pii` to `true` to capture this data for debugging. See <PlatformLink to="/data-management/data-collected/">Data Collected</PlatformLink> for more info. | ||
|
|
||
| </Alert> | ||
|
|
||
| ## Verify | ||
|
|
||
| Verify that the integration works by creating an agent with a tool and prompting it. The resulting data should show up in your AI Agents dashboard. | ||
|
|
||
| First, create an agent and a tool: | ||
|
|
||
| ```bash | ||
| php artisan make:agent TimeAgent | ||
| php artisan make:tool GetCurrentTime | ||
| ``` | ||
|
|
||
| Update the tool to return the current time: | ||
|
|
||
| ```php {filename:app/Ai/Tools/GetCurrentTime.php} | ||
| <?php | ||
|
|
||
| namespace App\Ai\Tools; | ||
|
|
||
| use Illuminate\Contracts\JsonSchema\JsonSchema; | ||
| use Laravel\Ai\Contracts\Tool; | ||
| use Laravel\Ai\Tools\Request; | ||
| use Stringable; | ||
|
|
||
| class GetCurrentTime implements Tool | ||
| { | ||
| public function description(): Stringable|string | ||
| { | ||
| return 'Get the current date and time.'; | ||
| } | ||
|
|
||
| public function handle(Request $request): Stringable|string | ||
| { | ||
| return now()->toDateTimeString(); | ||
| } | ||
|
|
||
| public function schema(JsonSchema $schema): array | ||
| { | ||
| return []; | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Register the tool on the agent: | ||
|
|
||
| ```php {filename:app/Ai/Agents/TimeAgent.php} | ||
| <?php | ||
|
|
||
| namespace App\Ai\Agents; | ||
|
|
||
| use App\Ai\Tools\GetCurrentTime; | ||
| use Laravel\Ai\Concerns\RemembersConversations; | ||
| use Laravel\Ai\Contracts\Agent; | ||
| use Laravel\Ai\Contracts\Conversational; | ||
| use Laravel\Ai\Contracts\HasTools; | ||
| use Laravel\Ai\Promptable; | ||
| use Stringable; | ||
|
|
||
| class TimeAgent implements Agent, Conversational, HasTools | ||
| { | ||
| use Promptable, RemembersConversations; | ||
|
|
||
| public function instructions(): Stringable|string | ||
| { | ||
| return 'You are a helpful assistant. Use your tools when asked about the time.'; | ||
| } | ||
|
|
||
| public function messages(): iterable | ||
| { | ||
| return []; | ||
| } | ||
|
|
||
| public function tools(): iterable | ||
| { | ||
| return [ | ||
| new GetCurrentTime, | ||
| ]; | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| Then prompt the agent, for example from a route: | ||
|
|
||
| ```php {filename:routes/web.php} | ||
| use App\Ai\Agents\TimeAgent; | ||
|
|
||
| Route::get('/debug-ai', function () { | ||
| $response = (new TimeAgent)->prompt('What time is it?'); | ||
|
|
||
| return $response->text; | ||
| }); | ||
| ``` | ||
|
|
||
| Visiting this route will trigger an AI agent invocation that will be captured by Sentry. It may take a couple of moments for the data to appear in [sentry.io](https://sentry.io). | ||
|
|
||
| ## Behavior | ||
|
|
||
| The following data is captured automatically: | ||
|
|
||
| - **Agent invocations** — `gen_ai.invoke_agent` spans for each agent prompt | ||
| - **LLM requests** — `gen_ai.chat` spans for each HTTP request to an AI provider | ||
| - **Tool executions** — `gen_ai.execute_tool` spans when agents call tools | ||
| - **Embeddings** — `gen_ai.embeddings` spans for embedding generation | ||
| - **Token usage** — input and output token counts per request | ||
| - **Model and provider info** — which model and provider were used | ||
|
|
||
| Streaming agent responses are also fully supported. The integration captures span data for both `prompt()` and `stream()` calls. | ||
|
|
||
| ### Conversations | ||
|
|
||
| If your agent implements `Conversational` and uses the `RemembersConversations` trait, Sentry automatically tracks the `conversationId` returned by Laravel AI. The integration adds it to AI spans as `gen_ai.conversation.id`, which lets Sentry group multiple prompts into the same conversation in **Explore > Conversations**. | ||
|
|
||
| To continue a conversation, pass the previous `$response->conversationId` back into Laravel AI with `continue()`: | ||
|
|
||
| ```php | ||
| use App\Ai\Agents\TimeAgent; | ||
| use App\Models\User; | ||
|
|
||
| $user = User::firstOrFail(); | ||
| $conversationId = null; | ||
|
|
||
| foreach (['Hello', 'What did I just ask you?'] as $message) { | ||
| $agent = $conversationId | ||
| ? (new TimeAgent)->continue($conversationId, as: $user) | ||
| : (new TimeAgent)->forUser($user); | ||
|
|
||
| $response = $agent->prompt($message); | ||
| $conversationId = $response->conversationId; | ||
| } | ||
| ``` | ||
|
|
||
| Both prompts will appear under the same conversation because Laravel AI reuses the same `conversationId`. For CLI commands and other non-HTTP entry points, create a transaction around each agent call so the AI spans are captured. | ||
|
|
||
| ## Options | ||
|
|
||
| You can selectively disable individual span types in `config/sentry.php` under the `tracing` > `features` key: | ||
|
|
||
| ```php {filename:config/sentry.php} | ||
| 'tracing' => [ | ||
| 'features' => [ | ||
| // Master switch for all AI spans (requires laravel/ai) | ||
| 'gen_ai' => env('SENTRY_TRACE_GEN_AI_ENABLED', true), | ||
|
|
||
| // Individual span types | ||
| 'gen_ai_invoke_agent' => env('SENTRY_TRACE_GEN_AI_INVOKE_AGENT_ENABLED', true), | ||
| 'gen_ai_chat' => env('SENTRY_TRACE_GEN_AI_CHAT_ENABLED', true), | ||
| 'gen_ai_execute_tool' => env('SENTRY_TRACE_GEN_AI_EXECUTE_TOOL_ENABLED', true), | ||
| 'gen_ai_embeddings' => env('SENTRY_TRACE_GEN_AI_EMBEDDINGS_ENABLED', true), | ||
| ], | ||
| ], | ||
| ``` | ||
|
|
||
| Setting `gen_ai` to `false` disables all AI tracing. The individual options let you turn off specific span types while keeping others active. | ||
|
|
||
| ## Supported Versions | ||
|
|
||
| - Laravel: 12.x+ | ||
| - `sentry/sentry-laravel`: 4.27.0+ | ||
| - `laravel/ai`: 0.x+ | ||
| - PHP: 8.2+ | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: The documentation uses
Illuminate\Contracts\JsonSchema\JsonSchema, which is only available in Laravel 13+, but claims support for Laravel 12.x, causing a "class not found" error.Severity: MEDIUM
Suggested Fix
The documentation should be corrected. Either update the prerequisite to require Laravel 13.x or later, or find the correct namespace for the
JsonSchemacontract within thelaravel/ai0.x community package and update the code example accordingly.Prompt for AI Agent
Did we get this right? 👍 / 👎 to inform future reviews.