Skip to content

diffscope model: field silently ignored in .diffscope.yml #109

Description

@angelxmoreno

Bug summary

In diffscope v0.5.28, the top-level model: field in .diffscope.yml is silently ignored. The primary review model always falls back to the hardcoded default anthropic/claude-opus-4.5 regardless of what is set in the config file.

Environment

  • diffscope version: 0.5.28 (installed via install.sh)
  • OS: macOS Darwin 25.2.0
  • Ollama running on localhost:11434 (reachable, model qwen3.5:latest loaded and verified)

Steps to reproduce

  1. Install diffscope: curl -fsSL https://raw.githubusercontent.com/evalops/diffscope/main/install.sh | sh

  2. Start ollama locally and pull a model (e.g. ollama pull qwen3.5:latest)

  3. Create a .diffscope.yml in any project:

    model: ollama:qwen3.5:latest
    base_url: http://localhost:11434
  4. Run git diff | diffscope review

Expected behavior

The review should use ollama:qwen3.5:latest from the config file.

Actual behavior

The review always uses anthropic/claude-opus-4.5 regardless of model: in YAML. Log line confirms:

INFO diffscope::commands::review::command::review: Starting diff review with generation model: anthropic/claude-opus-4.5

The review hangs trying to reach the Anthropic API. diffscope doctor reports Base URL: http://localhost:11434 (proving base_url: is read), but also reports Model: anthropic/claude-opus-4.5 (proving model: is not read).

What works

  • --model ollama:qwen3.5:latest CLI flag (overrides the default)
  • DIFFSCOPE_BASE_URL env var
  • base_url: in YAML
  • model_weak:, model_fast:, model_reasoning:, model_embedding: in YAML (suppress role warnings)
  • All tuning keys (temperature, max_tokens, context_window, strictness, max_context_chars, max_diff_chars, context_max_chunks, context_budget_chars)
  • exclude_patterns in YAML

What does not work

  • model: in YAML (top-level, with colon format like ollama:qwen3.5:latest)
  • model: in YAML (with slash format like ollama/qwen3.5:latest)
  • model: in YAML (with just model name + adapter: ollama)
  • model: in YAML (quoted)
  • model_primary:, model_default:, primary_model:, default_model:, review_model:, generation_model:, provider_model:, main_model: (variants tested)
  • ANTHROPIC_DEFAULT_OPUS_MODEL env var (also ignored)
  • providers.ollama.models.primary (and other role names under it)
  • providers.ollama.model:
  • generation_model_role: primary routing to a configured model: (separate from the model field itself)

Doctor display inconsistency

diffscope doctor also has a display bug: the Configuration: Model: and Primary Provider: lines always show the hardcoded defaults (anthropic/claude-opus-4.5 and anthropic), regardless of the actual config. The Base URL: line in the same section does reflect the YAML base_url:. The "Usage:" line at the bottom of doctor output does reflect the actual configured model.

Workaround

Pass --model on every CLI invocation:

git diff | diffscope review --model ollama:qwen3.5:latest --base-url http://localhost:11434

Or set both in env:

export DIFFSCOPE_BASE_URL=http://localhost:11434
alias ds='diffscope review --model ollama:qwen3.5:latest'

Investigation notes

Confirmed via string extraction from the binary at /usr/local/bin/diffscope:

  • The schema (per the embedded web UI) accepts model, model_weak, model_fast, model_reasoning, model_embedding, adapter, fallback_models at the top level
  • Role routing keys generation_model_role, auditing_model_role, verification_model_role exist
  • Provider block accepts providers.<name>.base_url, providers.<name>.api_key
  • Default model literal: anthropic/claude-opus-4.5 (hardcoded in binary)
  • Built-in providers: openai, openrouter, anthropic, ollama

The parser reads base_url: from YAML successfully (verified via diffscope doctor reporting it). It reads model_weak: etc. (verified via role warnings disappearing). It does NOT read model: (verified via the log line in review output and via the doctor display).

This looks like a deserialization field-name mismatch — the YAML key model may be deserialized into a different struct field, or the field is renamed (model vs primary_model vs something else) and the YAML parser is silently dropping unknown fields. Either way, the user-facing behavior is broken.

Severity

High. This is the most basic field in the config file. Every other config tool works this way (model: at top level), and the README's examples/selfhosted-ollama.yml ships with model: ollama:codellama as the FIRST line. New users following the README will hit this immediately.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions