Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,7 @@ export P2P_FILTER_ANNOUNCED_ADDRESSES=
# Each environment references pool resources by id using lightweight refs {id, total?, min?, max?}.
# Dual-gate tracking for fungible resources: per-env ceiling (Gate 1) + engine-wide pool (Gate 2).
# Discrete resources (GPUs) are tracked globally — a GPU in use on envA shows as in-use on envB too.
# export DOCKER_COMPUTE_ENVIRONMENTS='[{"socketPath":"/var/run/docker.sock","resources":[{"id":"disk","total":500},{"id":"gpu0","kind":"discrete","type":"gpu","total":1,"description":"NVIDIA A100","platform":"nvidia","driverVersion":"570.195.03","init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":["GPU-uuid-a"],"Capabilities":[["gpu"]]}}}],"environments":[{"id":"envA","storageExpiry":604800,"maxJobDuration":3600,"minJobDuration":60,"resources":[{"id":"cpu"},{"id":"ram"},{"id":"disk","max":500},{"id":"gpu0"}],"fees":{"1":[{"feeToken":"0x123","prices":[{"id":"cpu","price":1},{"id":"ram","price":0.1},{"id":"disk","price":0.01},{"id":"gpu0","price":5}]}]}}]}]'
# export DOCKER_COMPUTE_ENVIRONMENTS='[{"socketPath":"/var/run/docker.sock","resources":[{"id":"disk","total":500},{"id":"gpu0","kind":"discrete","type":"gpu","total":1,"description":"NVIDIA A100","platform":"nvidia","driverVersion":"570.195.03","init":{"deviceRequests":{"Driver":"nvidia","DeviceIDs":["GPU-uuid-a"],"Capabilities":[["gpu"]]}}}],"serviceOnDemand":{"enabled":true,"nodeHost":"localhost","hostPortRange":[30000,32767],"minDurationSeconds":0,"maxDurationSeconds":86400,"allowImageBuild":false},"environments":[{"id":"envA","storageExpiry":604800,"maxJobDuration":3600,"minJobDuration":60,"minServiceDuration":600,"maxServiceDuration":7200,"resources":[{"id":"cpu"},{"id":"ram"},{"id":"disk","max":500},{"id":"gpu0"}],"fees":{"1":[{"feeToken":"0x123","prices":[{"id":"cpu","price":1},{"id":"ram","price":0.1},{"id":"disk","price":0.01},{"id":"gpu0","price":5}]}]}}]}]'
export DOCKER_COMPUTE_ENVIRONMENTS=


28 changes: 28 additions & 0 deletions docs/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -1672,6 +1672,8 @@ fetch all compute environments
"storageExpiry": 604800,
"maxJobDuration": 3600,
"minJobDuration": 60,
"minServiceDuration": 60,
"maxServiceDuration": 86400,
"resources": [
{ "id": "cpu", "total": 16, "max": 16, "min": 1, "inUse": 0 },
{
Expand All @@ -1698,6 +1700,32 @@ fetch all compute environments
]
```

`maxJobDuration` / `minJobDuration` apply to **compute jobs**. Services have their own pair,
and SERVICE_START rejects a `duration` outside it:

- `maxServiceDuration` — the ceiling. Defaults to the daemon's
`serviceOnDemand.maxDurationSeconds` and may be **lowered** per environment; a larger per-env
value is clamped to the daemon ceiling at startup. SERVICE_EXTEND also caps the resulting
remaining window to it.
- `minServiceDuration` — the floor. SERVICE_START rejects a shorter `duration`, and
SERVICE_EXTEND rejects a shorter `additionalDuration`: the floor is a minimum *purchase*, so a
smaller one is refused rather than silently billed at the floor. Everything accepted is then
priced by its actual duration, rounded up to whole minutes. Defaults to the environment's own
`minJobDuration`, and may be **raised** per environment; a value below the daemon's
`serviceOnDemand.minDurationSeconds` is clamped up at startup.

Environments on the same engine may therefore report different values for both.

Both fields are additive, and an older node omits them. Their fallbacks differ, so treat each
separately:

- A missing `maxServiceDuration` means the 86400 s (24 h) default — **not** `maxJobDuration`,
which is a different limit and is often much larger, so using it would offer windows the node
rejects.
- A missing `minServiceDuration` means the environment's own `minJobDuration` (raised to the
daemon's `serviceOnDemand.minDurationSeconds`, which itself defaults to 0 — no daemon floor).
That is exactly what such a node already bills a service at.

### `HTTP` POST /api/services/freeCompute

### `P2P` command: freeStartCompute
Expand Down
15 changes: 14 additions & 1 deletion docs/env.md
Original file line number Diff line number Diff line change
Expand Up @@ -224,13 +224,24 @@ The config has a two-level structure:
{ "id": "disk", "total": 50 }
],

"serviceOnDemand": {
"enabled": true,
"nodeHost": "localhost",
"hostPortRange": [30000, 32767],
"minDurationSeconds": 0,
"maxDurationSeconds": 86400,
"allowImageBuild": false
},

"environments": [
{
"id": "default",
"description": "CPU compute environment",
"storageExpiry": 604800,
"maxJobDuration": 3600,
"minJobDuration": 60,
"minServiceDuration": 600,
"maxServiceDuration": 7200,
"enableNetwork": false,
"access": {
"addresses": ["0x123", "0x456"],
Expand Down Expand Up @@ -290,7 +301,9 @@ The config has a two-level structure:
- **id** *(optional)*: Stable identifier for the environment. Used to compute the environment hash.
- **description**: Human-readable description.
- **storageExpiry**: Seconds before compute results expire.
- **maxJobDuration** / **minJobDuration**: Maximum/minimum job duration in seconds.
- **maxJobDuration** / **minJobDuration**: Maximum/minimum **compute job** duration in seconds. These do not apply to services.
- **minServiceDuration** *(optional)*: Minimum **service** duration in seconds, for service-on-demand. SERVICE_START rejects a shorter duration and SERVICE_EXTEND rejects a shorter top-up — it is a minimum purchase, not a rounding rule, so anything accepted is billed for its actual duration (rounded up to whole minutes). Must not exceed `maxServiceDuration`, or the node refuses to start. Omit to fall back to this environment's `minJobDuration` — which is what services were already priced at. A value below the daemon's `serviceOnDemand.minDurationSeconds` is raised to it at startup, with a warning. Advertised to clients as `minServiceDuration`.
- **maxServiceDuration** *(optional)*: Maximum **service** duration in seconds, for service-on-demand. Omit to inherit the daemon's `serviceOnDemand.maxDurationSeconds` (default 86400). That daemon value is a hard ceiling — an environment can only lower it, and a larger value is clamped at startup with a warning. Advertised to clients on every environment as `maxServiceDuration`.
Comment thread
coderabbitai[bot] marked this conversation as resolved.
- **maxJobs**: Maximum simultaneous paid jobs.
- **enableNetwork**: Whether algorithm containers can make outbound network connections. Default: `false`
- **access**: Access control for paid jobs.
Expand Down
28 changes: 27 additions & 1 deletion docs/services.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,13 +158,39 @@ Service-on-demand is configured per Docker connection under `serviceOnDemand`:
| `enabled` | Master switch for the feature on this connection. |
| `nodeHost` | Externally reachable host used to build endpoint URLs. |
| `hostPortRange` | `[start, end]` range the node allocates published host ports from. |
| `maxDurationSeconds` | Upper bound on a service's lifetime (default 86400). |
| `minDurationSeconds` | Hard floor under a service's duration for this daemon (default 0 — no floor). An environment may raise it with its own `minServiceDuration`; a smaller per-env value is clamped up to this one at startup, with a warning. |
| `maxDurationSeconds` | Hard ceiling on a service's lifetime for this daemon (default 86400). An environment may lower it with its own `maxServiceDuration`; a larger per-env value is clamped to this one at startup, with a warning. |
| `allowImageBuild` | If true, consumers may submit an inline `dockerfile` to build. |

Whether a given environment accepts services is gated by its `features.services` flag,
and access can be restricted with the environment's `access` allow-list
(`addresses` + on-chain `accessLists`).

Each environment resolves its own service bounds at startup and advertises them in
GET_COMPUTE_ENVIRONMENTS as `minServiceDuration` and `maxServiceDuration`. SERVICE_START
rejects a `duration` outside that range; SERVICE_EXTEND caps the resulting remaining window to
the maximum. Set them per environment to give, say, a cheap CPU env a 1 h limit while a GPU
env keeps the full 24 h:

```json
{ "id": "cpu-small", "minServiceDuration": 600, "maxServiceDuration": 3600, "fees": { "1": [ ... ] } }
```

`minServiceDuration` is a minimum **purchase**, applied to a start and to an extension alike:
SERVICE_START rejects a shorter `duration` and SERVICE_EXTEND rejects a shorter
`additionalDuration`, rather than granting the smaller window and charging for the floor.
Anything accepted is priced by its actual duration, rounded up to whole minutes. Rejecting is
what keeps the two honest — billing a 100 s top-up as 600 s would let ten of them add 1000 s of
runtime while charging for 6000 s. It defaults to the environment's `minJobDuration`, which is
exactly what services were already billed at, so leaving both new fields unset changes nothing.

Both are separate from `minJobDuration` / `maxJobDuration`, which are per-env too but apply
only to compute jobs.

An environment whose resolved floor exceeds its resolved cap is a **fatal config error**: no
duration could satisfy both, so the node logs the offending environment and refuses to start
rather than advertising an environment that can never be booked.

**Templates are not shipped in the image.** The node reads them from a folder the operator
mounts in, so a node without that mount advertises no templates at all. Point
`serviceTemplatesPath` (env var `SERVICE_TEMPLATES_PATH`) at the mount:
Expand Down
10 changes: 10 additions & 0 deletions src/@types/C2D/C2D.ts
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,8 @@ export interface ComputeEnvironmentBaseConfig {
storageExpiry?: number // amount of seconds for storage
minJobDuration?: number // min billable seconds for a paid job
maxJobDuration?: number // max duration in seconds for a paid job
minServiceDuration?: number // min duration in seconds for a paid service
maxServiceDuration?: number // max duration in seconds for a paid service
maxJobs?: number // maximum number of simultaneous paid jobs
fees: ComputeEnvFeesStructure
resources?: ComputeResource[]
Expand Down Expand Up @@ -209,6 +211,14 @@ export interface C2DEnvironmentConfig {
storageExpiry?: number
minJobDuration?: number
maxJobDuration?: number
// Optional per-env service floor. The daemon's serviceOnDemand.minDurationSeconds is a hard
// floor: an env may only raise it, and a smaller value is clamped up (with a warning) at
// startup. Omitted → the env falls back to its own minJobDuration.
minServiceDuration?: number
// Optional per-env service cap. The daemon's serviceOnDemand.maxDurationSeconds is a hard
// ceiling: an env may only lower it, and a larger value is clamped (with a warning) at
// startup. Omitted → the env inherits the daemon cap.
maxServiceDuration?: number
maxJobs?: number
fees?: ComputeEnvFeesStructure
access?: ComputeAccessList
Expand Down
14 changes: 13 additions & 1 deletion src/@types/C2D/ServiceOnDemand.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,11 +104,23 @@ export interface ServiceTemplatePublic extends Omit<ServiceTemplate, 'envVars'>

// ── Operational config (per Docker daemon, not global) ────────────────

// Service duration cap applied when a daemon carries no `serviceOnDemand` block, or one
// that omits `maxDurationSeconds`. Single source of truth for the config schema's default,
// the SERVICE_START / SERVICE_EXTEND checks and the `maxServiceDuration` every compute
// environment advertises — those three must never disagree.
export const DEFAULT_SERVICE_MAX_DURATION_SECONDS = 86400 // 24 h

// Daemon-level service floor when `serviceOnDemand` omits `minDurationSeconds`. Zero means
// "no daemon floor", so an environment's own minServiceDuration (which itself falls back to
// minJobDuration) is what applies — keeping an unconfigured node billing exactly as before.
export const DEFAULT_SERVICE_MIN_DURATION_SECONDS = 0

export interface ServiceOnDemandConfig {
enabled: boolean
nodeHost: string // host (or IP) clients use to reach forwarded service ports; e.g. 'localhost'
hostPortRange?: [number, number] // e.g. [30000, 32767]; specific to this daemon's host
maxDurationSeconds?: number // default: 86400 (24 h)
minDurationSeconds?: number // default: DEFAULT_SERVICE_MIN_DURATION_SECONDS (no daemon floor)
maxDurationSeconds?: number // default: DEFAULT_SERVICE_MAX_DURATION_SECONDS (24 h)
allowImageBuild?: boolean // default: false — gates Dockerfile-based services per daemon
}

Expand Down
45 changes: 42 additions & 3 deletions src/components/c2d/compute_engine_base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,11 @@ import type {
DBComputeJobMetadata,
ComputeEnvFees
} from '../../@types/C2D/C2D.js'
import type { ServiceJob } from '../../@types/C2D/ServiceOnDemand.js'
import {
DEFAULT_SERVICE_MAX_DURATION_SECONDS,
DEFAULT_SERVICE_MIN_DURATION_SECONDS,
type ServiceJob
} from '../../@types/C2D/ServiceOnDemand.js'
import { C2DClusterType, C2DStatusNumber } from '../../@types/C2D/C2D.js'
import { C2DDatabase } from '../database/C2DDatabase.js'
import { Escrow } from '../core/utils/escrow.js'
Expand Down Expand Up @@ -80,6 +84,34 @@ export abstract class C2DEngine {
return this.clusterConfig
}

/**
* Hard cap, in seconds, on how long a service may run — what SERVICE_START validates the
* requested duration against, and what SERVICE_EXTEND caps the resulting remaining window
* to. Per Docker daemon rather than per environment, so every env on this engine reports
* the same value via `maxServiceDuration`. Falls back to the schema default when the
* cluster carries no `serviceOnDemand` block, so what is advertised is exactly what is
* enforced.
*/
getMaxServiceDuration(): number {
return (
this.getC2DConfig().connection?.serviceOnDemand?.maxDurationSeconds ??
DEFAULT_SERVICE_MAX_DURATION_SECONDS
)
}

/**
* Floor, in seconds, that this daemon puts under every service. An environment may raise it
* with its own `minServiceDuration` but never go below it. Defaults to 0 — no daemon floor —
* so an unconfigured node leaves each env's own floor (its minJobDuration) in charge and
* bills exactly as it did before this knob existed.
*/
getMinServiceDuration(): number {
return (
this.getC2DConfig().connection?.serviceOnDemand?.minDurationSeconds ??
DEFAULT_SERVICE_MIN_DURATION_SECONDS
)
}

getC2DType(): C2DClusterType {
/** Returns cluster type */
return this.clusterConfig.type
Expand Down Expand Up @@ -1012,14 +1044,21 @@ export abstract class C2DEngine {
return cost
}

/**
* @param minDurationOverride - billing floor to apply instead of `env.minJobDuration`.
* Services pass their own `minServiceDuration` here so a service is never priced against the
* compute-job floor. Omitted (compute jobs) keeps the original behaviour.
*/
public calculateResourcesCost(
resourcesRequest: ComputeResourceRequest[],
env: ComputeEnvironment,
chainId: number,
token: string,
maxJobDuration: number
maxJobDuration: number,
minDurationOverride?: number
): number | null {
if (maxJobDuration < env.minJobDuration) maxJobDuration = env.minJobDuration
const minDuration = minDurationOverride ?? env.minJobDuration
if (maxJobDuration < minDuration) maxJobDuration = minDuration
const prices = this.getEnvPricesForToken(env, chainId, token)
if (!prices) return null
let cost: number = 0
Expand Down
57 changes: 56 additions & 1 deletion src/components/c2d/compute_engine_docker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -537,6 +537,57 @@ export class C2DEngineDocker extends C2DEngine {
const fees = this.processFeesForEnvironment(envDef.fees, supportedChains)
const envResources = this.resolveEnvironmentResources(envDef, connectionPool)

// Service duration cap. The daemon's serviceOnDemand.maxDurationSeconds is a hard
// ceiling — an env may tighten it but never raise it, so a larger value is clamped
// with a warning (same treatment a resource max above the pool total gets).
const daemonServiceCap = this.getMaxServiceDuration()
const { maxServiceDuration: envServiceCap } = envDef
if (envServiceCap !== undefined && envServiceCap > daemonServiceCap) {
CORE_LOGGER.warn(
`Environment "${envDef.description || envDef.id || 'unknown'}": ` +
`maxServiceDuration (${envServiceCap}) is greater than the daemon's ` +
`serviceOnDemand.maxDurationSeconds (${daemonServiceCap}) — clamping to ` +
`${daemonServiceCap}. An environment can only lower the daemon cap.`
)
}
const maxServiceDuration = Math.min(
envServiceCap ?? daemonServiceCap,
daemonServiceCap
)

// Service floor. Mirror image of the cap: the daemon's serviceOnDemand.minDurationSeconds
// is a hard floor an env may raise but not undercut, so a smaller per-env value is clamped
// up with a warning. Absent, an env falls back to its own minJobDuration, which is what
// services were already billed at — so an unconfigured node is unchanged.
const daemonServiceFloor = this.getMinServiceDuration()
const { minServiceDuration: envServiceFloor } = envDef
if (envServiceFloor !== undefined && envServiceFloor < daemonServiceFloor) {
CORE_LOGGER.warn(
`Environment "${envDef.description || envDef.id || 'unknown'}": ` +
`minServiceDuration (${envServiceFloor}) is below the daemon's ` +
`serviceOnDemand.minDurationSeconds (${daemonServiceFloor}) — raising to ` +
`${daemonServiceFloor}. An environment can only raise the daemon floor.`
)
}
const minServiceDuration = Math.max(
envServiceFloor ?? envDef.minJobDuration ?? 0,
daemonServiceFloor
)
// Fatal, unlike the clamps above: those correct a value into a working range, whereas an
// empty range leaves the env permanently unusable for services — every SERVICE_START would
// 400. Refuse to boot rather than advertise an environment that can never be booked.
if (minServiceDuration > maxServiceDuration) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When minServiceDuration > maxServiceDuration, the env is still built and advertised; every SERVICE_START then 400s. Logged clearly ("Fix your config"), and consistent with how resource-max-over-pool is handled (warn, don't crash). Flagging only so the choice is explicit — a bad config yields a silently-broken env rather than a startup failure.

Proposal:

  • error and crash, do not start node

const envName = envDef.description || envDef.id || 'unknown'
const message =
`Environment "${envName}": minServiceDuration (${minServiceDuration}) exceeds ` +
`maxServiceDuration (${maxServiceDuration}) — no service duration can satisfy both, so ` +
`every SERVICE_START would be rejected. Fix the environment's minServiceDuration / ` +
`maxServiceDuration, or the daemon's serviceOnDemand.minDurationSeconds / ` +
`maxDurationSeconds.`
CORE_LOGGER.error(message)
throw new Error(message)
}

const env: ComputeEnvironment = {
id: '',
runningJobs: 0,
Expand All @@ -555,7 +606,11 @@ export class C2DEngineDocker extends C2DEngine {
features: {
computeJobs: envDef.features?.computeJobs ?? true,
services: envDef.features?.services ?? true
}
},
// Always advertised, even where features.services is false, because they state what
// SERVICE_START would enforce — clients gate on features.services, not on absence.
minServiceDuration,
maxServiceDuration
}

if (envDef.storageExpiry !== undefined) env.storageExpiry = envDef.storageExpiry
Expand Down
Loading
Loading