Drop perception. Update docs.

[Dreamsorcerer]

Problem

Dropping heavyweight perception from base install. With other PRs, this removes torch from base install.

Also dropping bitsandbytes which doesn't appear to be used anywhere at all?

Also updating the docs now to reflect all those changes.

Breaking Changes

Users wanting perception now need to request it.

[paul-nechifor]

bitsandbytes is needed by langchain-ollama or langchain-huggingface. I don't remember which one, but I added it intentionally (should have left a comment at the time).

[greptile-apps]

Greptile Summary

This PR removes perception from the base extra and drops bitsandbytes from agents, with the stated goal of eliminating the PyTorch dependency from a standard dimos install. The documentation is updated to match.

• openai-whisper still in agents: openai-whisper depends on torch, so base (= agents + web + viz) still transitively requires torch — the core goal of removing torch from base is not achieved.
• Undeclared whisper extra: docs/requirements.md documents a new whisper extra (openai-whisper, GPU required), but no such extra exists in pyproject.toml; users requesting it will get an error.
• Docs/code mismatch on agents: The requirements table says agents uses faster-whisper (No GPU), but the code still lists openai-whisper.

Confidence Score: 3/5

Not safe to merge as-is — the primary goal (removing torch from base) is not achieved, and documented extras don't match the package definition.

Multiple P1 findings: openai-whisper (torch dep) is still in agents/base, a documented whisper extra doesn't exist in pyproject.toml, and the agents table entry incorrectly lists faster-whisper. These are actionable defects that contradict the PR's stated goals.

pyproject.toml (agents extra still has openai-whisper) and docs/requirements.md (whisper extra/agents table mismatch)

Important Files Changed

Filename	Overview
pyproject.toml	Removes bitsandbytes from agents and perception from base, but leaves openai-whisper (torch-dependent) in agents, so base still transitively requires torch — the primary goal of the PR is not achieved.
docs/requirements.md	Documents a non-existent whisper extra, incorrectly lists faster-whisper for the agents extra, and marks base as GPU-free — all three claims are inconsistent with the actual pyproject.toml.
README.md	Installation examples updated to drop redundant base prefix for unitree and add a clear perception note; changes look correct.
uv.lock	Lock file faithfully reflects the pyproject.toml changes; openai-whisper correctly remains in base/agents since the pyproject.toml still declares it there.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    base["base extra\n(agents + web + viz)"]
    agents["agents extra\n(langchain, openai,\nopenai-whisper ← torch still here)"]
    web["web extra\n(fastapi, uvicorn, ffmpeg)"]
    viz["visualization extra\n(rerun-sdk, dimos-viewer)"]
    perception["perception extra\n(ultralytics, transformers,\nmoondream) — REMOVED from base ✅"]
    bitsandbytes["bitsandbytes\n— REMOVED from agents ✅"]
    whisper_doc["whisper extra\n(documented in docs,\nbut NOT in pyproject.toml ❌)"]

    base --> agents
    base --> web
    base --> viz
    agents -. "removed" .-> bitsandbytes
    base -. "removed" .-> perception
    whisper_doc -. "missing definition" .-> pyproject["pyproject.toml"]

    style bitsandbytes fill:#d4edda,stroke:#28a745
    style perception fill:#d4edda,stroke:#28a745
    style whisper_doc fill:#f8d7da,stroke:#dc3545
    style agents fill:#fff3cd,stroke:#ffc107

Loading

Comments Outside Diff (1)

1. pyproject.toml, line 172 (link)

   openai-whisper still in agents — torch not actually removed from base

   openai-whisper (line 172) depends on PyTorch, so base = [agents, web, viz] still pulls in torch. The PR description says this change "removes torch from base install," and the updated docs explicitly mark base as "No GPU" and agents as using faster-whisper — but neither is true yet. Additionally, docs/requirements.md documents a new whisper extra (for openai-whisper) that does not exist anywhere in pyproject.toml, so users following the docs will get a package-not-found error when requesting it.

_{Reviews (1): Last reviewed commit: "Merge branch 'dev' into sam/drop-percept..." | Re-trigger Greptile}

[greptile-apps]

Documented whisper extra does not exist

Line 48 documents a whisper extra (openai-whisper, GPU required), but no such extra is defined in pyproject.toml. A user running pip install 'dimos[whisper]' will get an error. Either the whisper extra needs to be added to pyproject.toml (and openai-whisper removed from agents), or this row should be removed from the table.

[greptile-apps]

agents key-packages column says faster-whisper, code says openai-whisper

The table on line 43 lists faster-whisper as a key package for the agents extra and marks it "No GPU", but pyproject.toml still lists openai-whisper (which requires torch) in agents. This mismatch will mislead users who expect a torch-free agent install.