Bug summary
In diffscope v0.5.28, the top-level model: field in .diffscope.yml is silently ignored. The primary review model always falls back to the hardcoded default anthropic/claude-opus-4.5 regardless of what is set in the config file.
Environment
- diffscope version: 0.5.28 (installed via
install.sh)
- OS: macOS Darwin 25.2.0
- Ollama running on
localhost:11434 (reachable, model qwen3.5:latest loaded and verified)
Steps to reproduce
-
Install diffscope: curl -fsSL https://raw.githubusercontent.com/evalops/diffscope/main/install.sh | sh
-
Start ollama locally and pull a model (e.g. ollama pull qwen3.5:latest)
-
Create a .diffscope.yml in any project:
model: ollama:qwen3.5:latest
base_url: http://localhost:11434
-
Run git diff | diffscope review
Expected behavior
The review should use ollama:qwen3.5:latest from the config file.
Actual behavior
The review always uses anthropic/claude-opus-4.5 regardless of model: in YAML. Log line confirms:
INFO diffscope::commands::review::command::review: Starting diff review with generation model: anthropic/claude-opus-4.5
The review hangs trying to reach the Anthropic API. diffscope doctor reports Base URL: http://localhost:11434 (proving base_url: is read), but also reports Model: anthropic/claude-opus-4.5 (proving model: is not read).
What works
--model ollama:qwen3.5:latest CLI flag (overrides the default)
DIFFSCOPE_BASE_URL env var
base_url: in YAML
model_weak:, model_fast:, model_reasoning:, model_embedding: in YAML (suppress role warnings)
- All tuning keys (
temperature, max_tokens, context_window, strictness, max_context_chars, max_diff_chars, context_max_chunks, context_budget_chars)
exclude_patterns in YAML
What does not work
model: in YAML (top-level, with colon format like ollama:qwen3.5:latest)
model: in YAML (with slash format like ollama/qwen3.5:latest)
model: in YAML (with just model name + adapter: ollama)
model: in YAML (quoted)
model_primary:, model_default:, primary_model:, default_model:, review_model:, generation_model:, provider_model:, main_model: (variants tested)
ANTHROPIC_DEFAULT_OPUS_MODEL env var (also ignored)
providers.ollama.models.primary (and other role names under it)
providers.ollama.model:
generation_model_role: primary routing to a configured model: (separate from the model field itself)
Doctor display inconsistency
diffscope doctor also has a display bug: the Configuration: Model: and Primary Provider: lines always show the hardcoded defaults (anthropic/claude-opus-4.5 and anthropic), regardless of the actual config. The Base URL: line in the same section does reflect the YAML base_url:. The "Usage:" line at the bottom of doctor output does reflect the actual configured model.
Workaround
Pass --model on every CLI invocation:
git diff | diffscope review --model ollama:qwen3.5:latest --base-url http://localhost:11434
Or set both in env:
export DIFFSCOPE_BASE_URL=http://localhost:11434
alias ds='diffscope review --model ollama:qwen3.5:latest'
Investigation notes
Confirmed via string extraction from the binary at /usr/local/bin/diffscope:
- The schema (per the embedded web UI) accepts
model, model_weak, model_fast, model_reasoning, model_embedding, adapter, fallback_models at the top level
- Role routing keys
generation_model_role, auditing_model_role, verification_model_role exist
- Provider block accepts
providers.<name>.base_url, providers.<name>.api_key
- Default model literal:
anthropic/claude-opus-4.5 (hardcoded in binary)
- Built-in providers:
openai, openrouter, anthropic, ollama
The parser reads base_url: from YAML successfully (verified via diffscope doctor reporting it). It reads model_weak: etc. (verified via role warnings disappearing). It does NOT read model: (verified via the log line in review output and via the doctor display).
This looks like a deserialization field-name mismatch — the YAML key model may be deserialized into a different struct field, or the field is renamed (model vs primary_model vs something else) and the YAML parser is silently dropping unknown fields. Either way, the user-facing behavior is broken.
Severity
High. This is the most basic field in the config file. Every other config tool works this way (model: at top level), and the README's examples/selfhosted-ollama.yml ships with model: ollama:codellama as the FIRST line. New users following the README will hit this immediately.
Bug summary
In diffscope v0.5.28, the top-level
model:field in.diffscope.ymlis silently ignored. The primary review model always falls back to the hardcoded defaultanthropic/claude-opus-4.5regardless of what is set in the config file.Environment
install.sh)localhost:11434(reachable, modelqwen3.5:latestloaded and verified)Steps to reproduce
Install diffscope:
curl -fsSL https://raw.githubusercontent.com/evalops/diffscope/main/install.sh | shStart ollama locally and pull a model (e.g.
ollama pull qwen3.5:latest)Create a
.diffscope.ymlin any project:Run
git diff | diffscope reviewExpected behavior
The review should use
ollama:qwen3.5:latestfrom the config file.Actual behavior
The review always uses
anthropic/claude-opus-4.5regardless ofmodel:in YAML. Log line confirms:The review hangs trying to reach the Anthropic API.
diffscope doctorreportsBase URL: http://localhost:11434(provingbase_url:is read), but also reportsModel: anthropic/claude-opus-4.5(provingmodel:is not read).What works
--model ollama:qwen3.5:latestCLI flag (overrides the default)DIFFSCOPE_BASE_URLenv varbase_url:in YAMLmodel_weak:,model_fast:,model_reasoning:,model_embedding:in YAML (suppress role warnings)temperature,max_tokens,context_window,strictness,max_context_chars,max_diff_chars,context_max_chunks,context_budget_chars)exclude_patternsin YAMLWhat does not work
model:in YAML (top-level, with colon format likeollama:qwen3.5:latest)model:in YAML (with slash format likeollama/qwen3.5:latest)model:in YAML (with just model name +adapter: ollama)model:in YAML (quoted)model_primary:,model_default:,primary_model:,default_model:,review_model:,generation_model:,provider_model:,main_model:(variants tested)ANTHROPIC_DEFAULT_OPUS_MODELenv var (also ignored)providers.ollama.models.primary(and other role names under it)providers.ollama.model:generation_model_role: primaryrouting to a configuredmodel:(separate from the model field itself)Doctor display inconsistency
diffscope doctoralso has a display bug: theConfiguration: Model:andPrimary Provider:lines always show the hardcoded defaults (anthropic/claude-opus-4.5andanthropic), regardless of the actual config. TheBase URL:line in the same section does reflect the YAMLbase_url:. The "Usage:" line at the bottom of doctor output does reflect the actual configured model.Workaround
Pass
--modelon every CLI invocation:git diff | diffscope review --model ollama:qwen3.5:latest --base-url http://localhost:11434Or set both in env:
Investigation notes
Confirmed via string extraction from the binary at
/usr/local/bin/diffscope:model,model_weak,model_fast,model_reasoning,model_embedding,adapter,fallback_modelsat the top levelgeneration_model_role,auditing_model_role,verification_model_roleexistproviders.<name>.base_url,providers.<name>.api_keyanthropic/claude-opus-4.5(hardcoded in binary)openai,openrouter,anthropic,ollamaThe parser reads
base_url:from YAML successfully (verified viadiffscope doctorreporting it). It readsmodel_weak:etc. (verified via role warnings disappearing). It does NOT readmodel:(verified via the log line in review output and via the doctor display).This looks like a deserialization field-name mismatch — the YAML key
modelmay be deserialized into a different struct field, or the field is renamed (modelvsprimary_modelvs something else) and the YAML parser is silently dropping unknown fields. Either way, the user-facing behavior is broken.Severity
High. This is the most basic field in the config file. Every other config tool works this way (
model:at top level), and the README'sexamples/selfhosted-ollama.ymlships withmodel: ollama:codellamaas the FIRST line. New users following the README will hit this immediately.