feat(dgw): add a background task API - #2007
Draft
irvingouj@Devolutions (irvingoujAtDevolution) wants to merge 8 commits into
Draft
irvingouj@Devolutions (irvingoujAtDevolution) wants to merge 8 commits into
irvingouj@Devolutions (irvingoujAtDevolution) wants to merge 8 commits into
Conversation
DVLS can start a background task with a TASK token (POST /jet/tasks) and
poll its state with a gateway.tasks.read scope token (GET /jet/tasks/{id}).
Tasks run in memory with a concurrency limit and a timeout, and report
not-started, running, success or failed.
The first kind, ai-log, validates its AI settings and then fails as not
implemented yet. Both endpoints are behind enable_unstable.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds the /jet/tasks endpoints, their schemas, the task_token scheme and the gateway.tasks.read scope to gateway-api.yaml, and regenerates the dotnet and TypeScript clients and the docs. Only the task-related output is kept; the existing version and content-type drift of the committed clients is left as is. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every task started through POST /jet/tasks now has a record in its own
database (provisioner_tasks.db, config key ProvisionerTasksDatabase) that
is never deleted, so GET /jet/tasks/{id} keeps answering after a restart.
Tasks run as job queue jobs and reuse its backoff and attempt limit.
Durable tasks resume after a restart; ephemeral tasks such as ai-log keep
their secrets (the AI API key) in memory only, so after a restart they
fail with "gateway restarted, API key no longer available" instead of
retrying. Neither database ever holds a secret. Transient errors are
retried, permanent ones fail right away. At startup, unfinished tasks
with no job left are marked as failed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Task records are now kept across restarts. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…stable Provisioner tasks now have their own job queue, stored in provisioner_tasks.db next to the task records, with their own runner. The runner claims at most MAX_CONCURRENT_TASKS jobs, so a claimed task never waits for a slot and never holds one of the shared job runner slots; the semaphore is gone. Retry, backoff and max attempts are those of the job queue, and startup still fails tasks whose job is gone. The task system (database, reconcile, runner) starts only when enable_unstable is set; stable installs never open the database. Also: - the task store keeps its schema version in a table, since the job queue in the same file owns PRAGMA user_version; - the ai-log API key is required; - a failure to store the result of a successful run is logged and not returned to the job queue, so the task is not run again; - both task endpoints answer errors with one shape and a typed code; - client errors are logged at debug; - the request body of POST /jet/tasks is a free-form JSON object in OpenAPI, and AiLogParams stays exported. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
StartTask now takes a free-form object; AiLogParams and the new TaskErrorCode are exported as models. The API key of AiLogParams is required, and both task endpoints document the same error body. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`get_or_create_cached_client` now returns an `anyhow::Error`, and HTTP clients need a crypto provider installed first, which `DgwState::mock` now does like the real service. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background task API for DVLS, behind
enable_unstable:POST /jet/taskswith a TASK token starts a task; the kind comes from the token, the body is kind-specific (AiLogParamsforai-log, exported as its own schema)GET /jet/tasks/{id}with agateway.tasks.readscope token returnsnot-started | running | success | failed{ "error": <code> }with a typed codeTasks run on their own job queue in a new
provisioner_tasks.db(ProvisionerTasksDatabase), separate from the recording jobs, 2 at a time, with retries on transient errors. Every task record is kept forever for audit. The whole thing only starts whenenable_unstableis set, so stable installs never touch the DB. Tasks with secrets (the AI key) keep them in memory only, so after a restart they fail instead of retrying; tests check the key never lands in the DB.ai-logis registered but fails as not implemented yet; the next PR fills it in. OpenAPI + dotnet/TS clients regenerated (task parts only).Tested:
tasks11/11,tasks::lib 15/15,dvls_compatibility24/24,config11/11, dotnet client builds.Stacked on #2006.
🤖 Generated with Claude Code