feat: add auto-update documentation system #12
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add comprehensive unified baseline adapters supporting Claude, GPT, and Gemini models across multiple evaluation tracks: Provider Abstraction (models/providers/): - BaseAPIProvider ABC with common interface for all providers - AnthropicProvider: Base64 PNG encoding, Messages API - OpenAIProvider: Data URL format, Chat Completions API - GoogleProvider: Native PIL Image support, GenerateContent API - Factory functions: get_provider(), resolve_model_alias() - Error hierarchy: ProviderError, AuthenticationError, RateLimitError Baseline Module (baselines/): - TrackType enum: TRACK_A (coords), TRACK_B (ReAct), TRACK_C (SoM) - TrackConfig dataclass with factory methods for each track - BaselineConfig with model alias resolution and registry - PromptBuilder for track-specific system prompts and user content - UnifiedResponseParser supporting JSON, function-call, PyAutoGUI formats - ElementRegistry for element_id to coordinate conversion Benchmark Integration: - UnifiedBaselineAgent wrapping UnifiedBaselineAdapter for benchmarks - Converts BenchmarkObservation -> adapter format -> BenchmarkAction - Support for all three tracks via --track flag CLI Commands (baselines/cli.py): - run: Single model prediction with track selection - compare: Multi-model comparison on same task - list-models: Show available models and providers All 92 tests pass. Ready for model comparison experiments. Co-Authored-By: Claude Sonnet 4.5 <[email protected]>
Add GitHub Actions workflow and sync script to automatically update docs.openadapt.ai when sub-package READMEs change. Components: - .github/workflows/trigger-docs-update.yml: Triggers docs sync on push - scripts/sync_package_docs.py: Syncs README to MkDocs format - docs/auto-docs-update.md: Setup guide and documentation - docs/workflows/sync-package-docs.yml: Reference workflow for main repo When this is merged and the DOCS_UPDATE_TOKEN secret is configured, pushing to main will trigger the main OpenAdapt repo to sync and deploy updated documentation. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Key changes: - Standalone Dockerfile using vanilla WAA dev mode (Samba share) - Uses dev_win11x64-enterprise-eval.xml unattend (expects \\host.lan\Data) - Injects file copy into samba.sh to populate /tmp/smb/ - Only patches IP addresses (no script path patching needed) - Image size: 1.3GB (vs 45GB official) Verified working: - Windows ISO auto-downloads (VERSION=11e) - Windows installs unattended (no license key) - FirstLogonCommands find scripts at \\host.lan\Data - setup.ps1 installs Python, Git, dependencies - WAA server starts on port 5000 - /probe returns 200 CLI: vm setup-waa builds image, vm run-waa runs benchmark Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Change default emulator IP from 172.30.0.2 to 20.20.20.21 (official WAA) - Export OPENAI_API_KEY before calling run-local.sh - Add automatic results download after benchmark completion (--no-download to disable) - Fix API key passing in both "reuse container" and "fresh start" paths - Update all internal_ip defaults and CLI argument defaults These fixes enable fully unattended vanilla WAA benchmark runs via: uv run python -m openadapt_ml.benchmarks.cli vm run-waa --num-tasks 5 Co-Authored-By: Claude Opus 4.5 <[email protected]>
Member
Author
CLI Fixes for Vanilla WAA AutomationAdded fixes to enable fully unattended vanilla WAA benchmark runs: Changes
Usage# Fully automated benchmark run
uv run python -m openadapt_ml.benchmarks.cli vm run-waa --num-tasks 5
# Results automatically downloaded to benchmark_results/Tested
|
Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Add a GitHub Actions workflow and sync script to automatically update docs.openadapt.ai when sub-package READMEs change.
Components
In this repo (openadapt-ml):
.github/workflows/trigger-docs-update.yml: Workflow that triggers on push to main (when README.md, docs/**, or CHANGELOG.md changes). Sends arepository_dispatchevent to the main OpenAdapt repo.For the main OpenAdapt repo (reference files included):
scripts/sync_package_docs.py: Python script that fetches READMEs from sub-packages and transforms them into MkDocs-compatible docs. This should be copied todocs/_scripts/in the main repo.docs/workflows/sync-package-docs.yml: Reference workflow file that should be copied to.github/workflows/in the main repo. Receives repository_dispatch events and creates PRs with updated docs.Documentation:
docs/auto-docs-update.md: Complete setup guide explaining how the system works and how to configure it.How It Works
Setup Required
After merging this PR:
contents:writepermission on OpenAdaptAI/OpenAdaptDOCS_UPDATE_TOKENto this repo (or as org secret)scripts/sync_package_docs.py->docs/_scripts/sync_package_docs.pydocs/workflows/sync-package-docs.yml->.github/workflows/sync-package-docs.ymlTest Plan
Generated with Claude Code