OpenThreads: Full Implementation Roadmap (meta/17 → main)#34
Open
claude[bot] wants to merge 41 commits intomainfrom
Open
OpenThreads: Full Implementation Roadmap (meta/17 → main)#34claude[bot] wants to merge 41 commits intomainfrom
claude[bot] wants to merge 41 commits intomainfrom
Conversation
Sets up the complete project bootstrap per issue #1: Monorepo: - Bun workspaces with packages: core, storage/mongodb, channels, server, trust - Shared tsconfig.base.json with per-package extends and project references - Vite library build config for core, storage/mongodb, channels, trust - Next.js setup for the server package Quality tooling: - ESLint v9 flat config with @typescript-eslint + prettier integration - Prettier with consistent formatting rules - lint-staged config in package.json for pre-commit hooks - husky prepare script (run `bun install` to initialize hooks) CI/CD: - GitHub Actions workflow: lint, typecheck, test on push/PR Dev environment: - Docker Compose with MongoDB 7.0 - .env.example with documented variables - Seed script at scripts/seed.ts for sample channels, routes, threads Co-authored-by: Gustavo Gondim <ggondim@users.noreply.github.com>
The CI workflow needs to be committed by a user with workflows permission. See .github/workflows/ci.yml in the working tree for the intended content. Co-authored-by: Gustavo Gondim <ggondim@users.noreply.github.com>
Project Bootstrap — Monorepo, CI, Dev Environment
…erfaces Implements issue #2 — all core TypeScript contracts for @openthreads/core: Data model types: - Channel, Recipient, Thread, Turn, Route, Envelope - A2H intent types (INFORM, COLLECT, AUTHORIZE, ESCALATE, RESULT) - Message union type with duck-typing discriminator (intent field = A2H) Utilities: - ID generation for ot_thr_*, ot_turn_*, ot_tk_*, ot_ch_sk_* prefixes - Message classification helpers (isA2HMessage, classifyMessages, etc.) Interfaces (fully abstract, no implementation): - StorageAdapter with CRUD for all entities + token lifecycle - StorageAdapterFactory for pluggable instantiation - ChannelAdapter with register, sendMessage, render, captureResponse, capabilities - ChannelCapabilities type flag set Tests: - Unit tests for ID generation (prefix validation, uniqueness, type guards) - Unit tests for message classification (duck-typing, mixed arrays, helpers) Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Core: Data Model, Types & Interfaces
Add packages/core with full implementation of: - TokenManager: ephemeral token generation/validation/revocation (ot_tk_*) with configurable TTL (default 24h), and channel API key management (ot_ch_sk_*) with channel-scoped validation - ThreadManager: native thread creation with de-duplication on native IDs, virtual thread detection via reply-chain grouping, and main thread get-or-create per (channel, target) pair - TurnManager: inbound/outbound turn logging (ot_turn_*) with chronological listing per thread - StorageAdapter interface for pluggable persistence backends - InMemoryStorageAdapter for testing - Full unit test suite covering token lifecycle, thread resolution, and turn tracking Refs #5 Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Adds the `@openthreads/core` package with:
- **Data model types** (`types.ts`): Channel, Recipient, Route,
RouteCriteria, InboundMessage, Thread, Turn — derived from VISION.md.
- **Route matching engine** (`router.ts`):
- `router(routes, message)` — filters enabled routes whose criteria all
match the inbound message, then returns them sorted by descending
priority (highest-priority first).
- Glob/wildcard support via `globToRegex` / `matchGlob` (`*` = any
chars, `?` = one char; regex metacharacters are escaped so literal
dots in channel/user IDs behave correctly).
- Per-field AND semantics; array criteria use OR (any pattern matches).
- Boolean criteria (`isThread`, `isMention`, `isDM`) accept undefined
as "match everything".
- Disabled routes are always excluded.
- Multiple recipients per route (fan-out) are preserved on each result.
- Input array is never mutated (sort operates on a copy produced by
filter).
- **Unit tests** (`router.test.ts`): 40+ cases covering glob edge-cases,
criterion matching, priority ordering, stable sort, fan-out, overlapping
routes, disabled routes, no-match, and a realistic multi-route scenario.
Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Core: Token & Thread/Turn Management
Core: Router Implementation
- packages/core: define all data types (Channel, Recipient, Thread, Turn, Route, Token) and the StorageAdapter interface - packages/storage/mongodb: full MongoStorageAdapter implementation with connection pooling, graceful shutdown, and all CRUD methods - Indexes: unique threadId/turnId, channelId+nativeThreadId, channelId+targetId, threadId+timestamp, criteria fields for routes, unique token value + TTL on expiresAt - Migration utilities: ensureAllIndexes + idempotent seed script - Integration tests: full coverage against real MongoDB via Docker Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Storage: MongoDB Adapter
Adds the full @openthreads/channels-slack package implementing the
ChannelAdapter interface for Slack using @slack/bolt.
Packages added:
- packages/core — ChannelAdapter interface, A2H types, envelope types
- packages/channels/slack — SlackAdapter class
Features implemented:
- Inbound: message events, app_mention events, /openthreads slash command
- Outbound: text (mrkdwn), Block Kit blocks/buttons/select menus
- Thread support: native Slack thread_ts ↔ OpenThreads threadId (1:1 map)
- A2H AUTHORIZE → Approve/Deny Block Kit buttons (method 1)
- A2H COLLECT with options → static_select menu (method 1)
- A2H COLLECT free-text → thread reply capture (method 2)
- A2H INFORM → plain text message
- Capabilities: { threads, buttons, selectMenus, dms, fileUpload: true, replyMessages: false }
Tests:
- Unit tests for Block Kit builders (blocks.test.ts)
- Unit tests for normalize utilities (normalize.test.ts)
- Integration tests for SlackAdapter with mock App/WebClient (SlackAdapter.test.ts)
- Shared adapter conformance suite (conformance.test.ts)
Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Channel: Slack Adapter
Adds packages/channels/discord — a full ChannelAdapter implementation
for Discord using discord.js:
Inbound:
- message create, @mentions (parseMessage)
- slash commands (parseSlashCommand + REST registration)
Outbound:
- plain text messages
- styled Discord embeds (INFORM / AUTHORIZE / COLLECT / ESCALATE)
- message components: approve/deny buttons (AUTHORIZE), select menus
(COLLECT with closed options)
Threads:
- getThread / createThread supporting both TextChannel threads and
ForumChannel posts (1:1 mapping with OpenThreads thread IDs)
- ensureThreadActive helper to unarchive threads before sending
A2H (method 1 + method 2):
- buildA2HComponents maps A2H intents to Discord action rows
- awaitResponse captures button/select interactions and thread replies
- Timeout configurable via interactionTimeoutSeconds
Capabilities: { threads, buttons, selectMenus, dms, fileUpload: true;
replyMessages: false }
Tests:
- Unit tests for message/command parsing, component builders, embeds
- Conformance tests verifying ChannelAdapter interface contract
Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Adds the Reply Engine component in packages/core — the component that processes recipient inbound envelopes, classifies each message item (Chat SDK vs A2H via duck typing), selects the appropriate reply method via the automatic decision tree, and orchestrates blocking response collection. Key components: - types/index.ts: full data model (ChatSDKMessage, A2HMessage, ChannelAdapter, ReplyEngineConfig, and all supporting types) - reply-engine/normalizer.ts: normalize single object → 1-item array - reply-engine/classifier.ts: duck-typing classification (intent field = A2H) - reply-engine/method-selector.ts: decision tree for methods 1–4 with capture hierarchy for method 2 - reply-engine/response-collector.ts: TimeoutError, withTimeout(), and ResponseRegistry for pending form submissions (methods 3 & 4) - reply-engine/index.ts: ReplyEngine class orchestrating all of the above Unit tests cover: classification, method selection across all intent/capability combinations, normalizer, mixed arrays, method 4 batching, trust layer, timeout handling, escalation handler, and ResponseRegistry. Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Implements the full ChannelAdapter interface for WhatsApp using the
Baileys library (WhatsApp Web protocol).
Key features:
- SessionManager: QR-code auth, multi-file auth-state persistence,
automatic reconnection with exponential backoff, clean shutdown
- Inbound: text, image, video, audio, document, sticker, button/list
responses; virtual thread IDs via quoted-reply chains
- Outbound: text, buttons (max 3), list messages, image/video/audio/
document media
- A2H method 1: AUTHORIZE with ≤3 options → WhatsApp interactive buttons
- A2H method 3: AUTHORIZE >3 options / COLLECT / complex intents → external
form link
- A2H method 2: quoted-reply capture for free-text responses (pending
capture map with configurable timeout)
- Capabilities: { threads: false, buttons: true, selectMenus: false,
replyMessages: true, dms: true, fileUpload: true }
- Conformance test suite covering capabilities, inbound/outbound, A2H
flows, and lifecycle
Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Core: Reply Engine
Adds the @openthreads/channel-telegram package implementing the full
ChannelAdapter interface for Telegram Bot API integration.
- packages/core: ChannelAdapter interface, A2H types, message envelope types
- packages/channels/telegram:
- TelegramApiClient: lightweight Telegram Bot API HTTP client
- TelegramAdapter: full ChannelAdapter implementation
- InMemoryThreadStore: virtual threads via reply chains (reply_to_message_id)
- Inbound parsing: text, commands, callback queries, attachments
- Outbound formatting: plain text, HTML, MarkdownV2, inline/reply keyboards
- A2H renderer: AUTHORIZE → inline keyboard (✅/❌), COLLECT+options → buttons,
COLLECT free-text → reply-capture (method 2)
- Capabilities: { threads: false, buttons: true, selectMenus: false,
replyMessages: true, dms: true, fileUpload: true }
- Comprehensive unit tests for all modules
Closes #8
Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Implements issue #11 — full HTTP surface for OpenThreads server: **Recipient Inbound (core send endpoints)** - POST /send/channel/:channelId/target/:groupOrUser — new thread variant - POST /send/channel/:channelId/target/:groupOrUser/thread/:threadId — existing thread - Dual auth: ephemeral token (?token=) OR channel API key (Authorization: Bearer) - Returns 202 for fire-and-forget, 200 with intent details for blocking A2H **Channel Inbound Webhooks** - POST /webhook/:channelId — generic webhook receiver - Slack signing secret verification (HMAC-SHA256, replay protection) - Telegram webhook secret token verification - Event normalization (Slack, Telegram, generic) - Router integration: fan-out to matched recipients via fetch() **Management CRUD API** - GET/POST /api/channels — list and create channels (auto-generates API key) - GET/PUT/DELETE /api/channels/:id - GET/POST /api/recipients - GET/PUT/DELETE /api/recipients/:id - GET/POST /api/routes — with priority ordering - GET/PUT/DELETE /api/routes/:id - GET /api/threads — filterable by channelId and targetId - GET /api/threads/:threadId - GET /api/threads/:threadId/turns **Auth** - Ephemeral token validation (verifyEphemeralToken) - Channel API key validation (verifyChannelApiKey) - Combined send auth (verifySendAuth) — token first, key fallback - Management API key (MANAGEMENT_API_KEY env var, dev bypass) **Infrastructure** - lib/db.ts — MongoDB singleton with full CRUD + index setup - lib/auth.ts — typed auth middleware helpers - lib/rate-limit.ts — in-memory sliding-window rate limiter - lib/logger.ts — structured request logging helpers - lib/fanout.ts — concurrent recipient delivery via fetch() - GET /health (and /api/health) — database ping + uptime - instrumentation.ts — graceful shutdown (SIGTERM/SIGINT → MongoDB close) - serverExternalPackages: ['mongodb'] in next.config.ts Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Channel: Discord Adapter
Channel: WhatsApp Adapter (Baileys)
Server: API Routes & Infrastructure
Channel: Telegram Adapter
- GET /form/[formKey] — server component that lazily creates a FormRecord from the turn on first access, checks TTL expiry, renders FormClient - FormClient — Ant Design 5 UI with AUTHORIZE (approve/deny + context), COLLECT (free-text, multi-field, select/radio/checkbox), batch layout, and loading/success/error/expired states - POST /api/form/[formKey] — validates responses, marks form submitted in MongoDB, resolves in-process formRegistry promise to unblock Reply Engine - FormResponseRegistry singleton (globalThis) bridges form submission to in-process Reply Engine without cross-module type coupling - FormRecord type + CRUD helpers + TTL index in db.ts - AntdRegistry wrapper in layout.tsx for Next.js SSR support - antd ^5, @ant-design/icons ^5, @ant-design/nextjs-registry ^1 added fixes #12 Co-authored-by: Gustavo Gondim <ggondim@users.noreply.github.com>
Server: Auto-Generated A2H Forms
Implements the full management dashboard UI for the OpenThreads server. ## Dashboard pages (/dashboard/*) - Overview: live stats (channel/route/thread counts) - Channels: list + add wizard (platform → credentials → test → save), edit/delete, masked API key with eye/copy toggle - Routes: ReactFlow visual flow (channel → route → recipient nodes), drag-to-create edges, criteria drawer, priority list, test route matching - Threads: searchable + filterable table, click-to-detail - Thread Detail: chronological turn timeline, A2H intent rendering, raw envelope collapsible - Settings: global TTL/trust-layer config + per-channel overrides ## New API endpoints - POST /api/routes/test — simulate inbound criteria → matching route IDs - GET /api/settings — read global settings from MongoDB - PUT /api/settings — update global settings ## Updated - GET /api/threads — channelId now optional; added search query param - lib/db.ts — listThreads(), AppSettings/ChannelOverride types, getSettings(), updateSettings() - Root page redirects to /dashboard - Root layout wraps with AntdRegistry for Next.js SSR CSS handling - lib/api-client.ts — new typed API client for dashboard pages fixes #13 Co-authored-by: Gustavo Gondim <ggondim@users.noreply.github.com>
Server: Dashboard UI
…y protection Implements the @openthreads/trust package and server integration for Issue 14. packages/trust: - JWS signing via Web Crypto API (ES256/ECDSA P-256) — no external deps signIntent/signResponse bind intent → response into a verifiable evidence chain - Replay protection: ReplayGuard with nonce store + timestamp validation - Audit logging: AuditLogger + InMemoryAuditStorage + AuditStorageAdapter interface - Strong auth: TOTP (RFC 6238 via Web Crypto HMAC-SHA1), WebAuthn challenge gen/verify - AuthChallengeManager: issue + verify challenges (webauthn, totp, sms_otp) - TrustLayerManager: single entry point wiring all subsystems together Auto-generates ES256 key pair; enabled=false means zero overhead packages/server: - GET /api/audit: query audit log (turnId, threadId, eventType, date range filters) - POST /api/form/:key/auth: issue auth challenge before form submission - PUT /api/form/:key/auth: verify challenge (TOTP code or WebAuthn assertion) - Form GET: returns requiresAuth=true + emits intent_rendered audit event - Form POST: requires verified challengeId when TRUST_LAYER_ENABLED=true - TrustLayerManager singleton (globalThis) with MongoDB-backed audit storage - audit_log MongoDB collection with indexes Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Trust Layer — JWS, Auth, Audit
Implements Issue #15 — Integration Testing & Hardening: **Resilience** - `packages/server/src/lib/retry.ts`: Exponential backoff retry utility (`withRetry`, `computeRetryDelay`) with configurable attempts, delays, and per-error `retryable` predicate. - `packages/server/src/lib/fanout.ts`: Extended with `deliverWithRetry` — automatically retries 5xx/network failures; treats 4xx as non-retryable. - `packages/server/src/lib/deduplication.ts`: Idempotent inbound message deduplication via in-memory LRU store with per-entry TTL. Includes platform-specific key builders (Slack, Telegram, WhatsApp, generic). - `packages/server/src/lib/graceful-storage.ts`: `withGracefulStorage` helper catches storage errors and returns a safe fallback; `StorageHealthMonitor` tracks sliding-window error rate for circuit-breaking. - `packages/channels/src/reconnect.ts`: Generic `ReconnectManager` for WebSocket-based adapters (Slack Socket Mode, Discord, etc.) with the same exponential-backoff pattern used by the WhatsApp `SessionManager`. **Adapter Conformance** - `packages/channels/src/conformance-suite.ts`: `runConformanceSuite()` factory generates a standardised Bun test suite for any `ChannelAdapter`. Covers interface shape, capability flags, handler registration, and lifecycle. Handles the different method naming conventions across adapters (Slack/WA/ Discord/Telegram). - `packages/channels/src/mocks/mock-channel-server.ts`: In-process mock servers for Slack, Telegram, and generic HTTP webhooks. Used to simulate platform callbacks in unit and E2E tests. **E2E Test Scenarios (Issue #15 — all 7 scenarios)** - `packages/core/tests/e2e/lifecycle.test.ts`: Full message lifecycle tests exercising all seven E2E scenarios from the issue using `InMemoryStorageAdapter` and core managers (no real HTTP or database). **Unit Tests** - `packages/server/tests/retry.test.ts` - `packages/server/tests/deduplication.test.ts` - `packages/server/tests/graceful-storage.test.ts` - `packages/server/tests/fanout.test.ts` - `packages/channels/tests/reconnect.test.ts` Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Integration Testing & Hardening
Packaging & Deployment: - Dockerfile for OpenThreads server (multi-stage Bun build) - docker-compose.yml production profile (app service, --profile production) - Helm chart scaffold (deploy/helm/openthreads/) with deployment, service, ingress, secret, HPA, serviceaccount templates - create-openthreads scaffold package (bunx create-openthreads <name>) Observability: - Structured JSON logger with configurable LOG_LEVEL + LOG_FORMAT env vars - Prometheus-compatible metrics endpoint (GET /api/metrics) with counters for messages in/out, A2H intents, fanout, HTTP requests, active threads - OpenTelemetry tracing setup in instrumentation.ts (enabled via OTEL_EXPORTER_OTLP_ENDPOINT, gracefully skipped if SDK not installed) A2H Layer 2 preparation: - Stub types for Layer 2 intents: POLICY, REVOKE, DELEGATE, SCOPE - isLayer1Intent() / isLayer2Intent() type guards - IntentHandlerRegistry extension point in reply-engine/intent-handler.ts for registering custom handlers (Layer 2 or overrides) - Export all new types from @openthreads/core Documentation: - docs/self-hosting.md (Docker, env vars, MongoDB, production checklist) - docs/adapter-authoring.md (ChannelAdapter interface guide) - docs/channels/slack.md, docs/channels/telegram.md Examples: - examples/plain-http/ — minimal Bun webhook consumer with cURL snippets - examples/n8n/ — n8n webhook node integration guide - examples/langgraph/ — LangGraph agent with A2H AUTHORIZE flow (Python) Co-authored-by: claude[bot] <claude[bot]@users.noreply.github.com>
Release & Developer Experience
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR merges the complete implementation of the OpenThreads platform, as tracked in meta issue #17. All 16 implementation issues have been completed across 8 waves.
What Was Implemented
Wave 1 — Foundation
Wave 2 — Contracts
Wave 3 — Core Logic + Adapters (parallel)
Wave 4 — Reply Engine
Wave 5 — Server + More Adapters (parallel)
Wave 6 — UI & Forms (parallel)
Wave 7 — Trust & Hardening
Wave 8 — Release
Test plan
meta/17docker compose up)Closes #17
🤖 Generated with Claude Code