feat(hosted key): Add exa hosted key#3221
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Greptile SummaryThis PR adds hosted Exa API key support to the Sim platform, allowing users on hosted Sim to run Exa search/content/answer operations without providing their own API key. It introduces a Key changes:
Confidence Score: 3/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant UI as Browser UI
participant Serializer
participant Executor as executeTool()
participant BYOK as getBYOKKey()
participant RateLimiter as HostedKeyRateLimiter
participant DB as Token Bucket DB
participant Exa as Exa API
UI->>Serializer: Serialize workflow (key field hidden when hosted)
Note over Serializer: isSubBlockHiddenByHostedKey skips field
Serializer-->>Executor: params without key
Executor->>Executor: injectHostedKeyIfNeeded()
Executor->>BYOK: getBYOKKey(workspaceId, exa)
alt BYOK key found
BYOK-->>Executor: key result (no billing)
else No BYOK key
BYOK-->>Executor: null
Executor->>RateLimiter: acquireKey(provider, prefix, config, actorId)
RateLimiter->>DB: consumeTokens(actor bucket, 1)
alt Actor rate limited
DB-->>RateLimiter: allowed false
RateLimiter-->>Executor: billingActorRateLimited true
Executor-->>UI: Error 429
else Within limit
DB-->>RateLimiter: allowed true
RateLimiter-->>Executor: success with hosted key
end
end
loop executeWithRetry up to 3 retries on 429
Executor->>Exa: executeToolRequest(params)
alt 429 from Exa
Exa-->>Executor: HTTP 429
Note over Executor: Wait 1s then 2s then 4s
else Success
Exa-->>Executor: Response with cost data
end
end
Executor->>Executor: applyHostedKeyCostToResult()
Executor->>DB: logFixedUsage(userId, cost)
Executor->>Executor: stripInternalFields removes internal fields
Executor-->>UI: output with cost total and timing
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
TheodoreSpeaks
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
|
@cursor review @greptile review |
|
@greptile-apps review |
apps/sim/providers/anthropic/core.ts
Outdated
|
|
||
| const toolCalls = [] | ||
| const toolResults = [] | ||
| const toolResults: any[] = [] |
There was a problem hiding this comment.
we're trying to avoid any annoations in the codebase -- can the function signature be updated with right shared type? if not use unknown
| if (costDollars?.total != null) { | ||
| return { cost: costDollars.total, metadata: { costDollars } } | ||
| } | ||
| // Fallback: $5/1000 (1-25 results) or $25/1000 (26-100 results) |
There was a problem hiding this comment.
we tend to avoid fallbacks like this -- is there a reason they might not give us costDollars? We want to fail loudly and fix the root cause if things don't work since hard to know when it flips to the fallback
| highlights: result.highlights, | ||
| score: result.score || 0, | ||
| })), | ||
| __costDollars: data.costDollars, |
There was a problem hiding this comment.
underscore to hide it from display as an internal field?
There was a problem hiding this comment.
nvm mentioned in the comment below, thanks
There was a problem hiding this comment.
Correct - basically wanted to account for being able to pass costDollars internally without needing it displayed to the customer, which is useful if we want to add a hosted key fee without the user getting confused.
apps/sim/tools/exa/search.ts
Outdated
| return { cost: costDollars.total, metadata: { costDollars } } | ||
| } | ||
|
|
||
| // Fallback: estimate based on search type and result count |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| }, | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Unhandled pricing error crashes successful tool execution
High Severity
applyHostedKeyCostToResult and processHostedKeyCost have no error handling around calculateToolCost. If getCost throws (e.g., Exa API response missing __costDollars), the error propagates unhandled up to executeTool's outer catch block, turning a successful tool execution into a failed result. Since the API call already succeeded and the user already consumed the Exa resource, this silently discards the valid result over a billing calculation failure.
Additional Locations (1)
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
3 participants
Summary
Added hosted api key integration for exa search.
For hosted sim, api sub block no longer appears for exa search. Users can use the BYOK modal to pass an api key if needed. Non-hosted sim behaves unchanged.
Added throttling per billing actor. Requests are throttled proactively, failing fast if they exceed limits (users can switch to BYOK if they need higher limits). Can be either a limit based on requests per minute or a custom limit using the api response. If custom, rate limiting is optimistic and only applies after execution completes.
Moved cost modification behavior to knowledge block directly instead of transforming each block separately.
Hosted key calls retry 3 times with exponential backoff to handle throttles from increased load, with alarms.
Important: API keys need to be stored in prod and staging before merging:
EXA_API_KEY_COUNTnumberEXA_API_KEY_{number}api keys (traffic spread evenly via round robin)Type of Change
Testing
Checklist
Screenshots/Videos