Agent Mode - model recommendations

[planetf1]

I'm struggling to get any model (suitable for 32GB macbook m1, served via ollama) working very much at all with agent mode.

Can anyone offer some decent models (for python/k8s/go coding) that work well in agent mode/for tool calling (including mcp) specifically with the continue plugin in vscode?

It can be as simple an example as 'can you list what charts I have' in the context of helm charts for example.

A typical response would be

{"name":"ls","arguments":{"dirPath":"/charts"}}

from the model, but not in the format continue agent mode appears to like

[ShaunaGordon]

That's the same model laptop I have. I've had good results with command-r7b, and the cogito models up to about 14b (8b has been a pretty good balance of speed and accuracy for quicker little stuff). What I like about Command-r7b is that it's designed for RAG, so its whole shtick is analyzing what you've got.

I've also gotten pretty good results with phi-4 (contrary to other people, apparently, but... 🤷‍♀️), but I haven't tried that one in agent mode.

The smaller qwen3.5 models and codestral/devstral are supposed to work well, too, but personally, when I've tried them, they didn't output anything. That was a couple Continue releases back, though, so it might have been fixed, or otherwise YMMV.

[planetf1]

I had some success using LMStudio instead of ollama & mlx (but I did download slightly different variants - which may have been slightly modified like unsloth) and both devstral (a little) and qwen3-coder (better) did work with continue a lot better. But very slow, and increasing the context, even to 32k needed k-v cache optimization. That made the results poor from qwen, so still hunting. I'll try the models you mentioned

[faetschi]

Did you make Tool Calling in devstral and/or qwen3-coder:30b work with Ollama?
I couldn't seem to make the models make more than one Tool Call at once (with any tool-capable model)

Related Issue:
#6913

[ShaunaGordon]

Not in Continue, unfortunately. I've had to switch away from it for Agent stuff (I still use Continue for its autocomplete) after running into the same problem, or the "this model doesn't support tools" error seen in some of the comments in #6913, even though the models very much do and I have tool_use set. I know it's not a model thing, because I'm using Devstral with Kilocode to great effect.

[faetschi]

Update: I found out that the qwen has an inherent "bug"/problem in its qwen3-coder model regarding tool_use according to qwen devs (cant find the exact qwen issue anymore...) but also mentioned here:
anomalyco/opencode#1034 (comment)
QwenLM/Qwen3-Coder#475

Still, devstral and gpt-oss:20b seem to have problems with reliable tool calling in the continue IDE extension. It could be a problem with response streaming (some people seem to have got tool calls working in other agentic CLIs / IDEs when running ollama locally without docker, need this confirmed though...)

Will definitely keep an eye on the development on this matter & use opencode CLI in the meanwhile (seems to be mentioned & fixed there?)
anomalyco/opencode#1034
anomalyco/opencode#3122

[ShaunaGordon]

gpt-oss is a reported bug: #8098

I suspect Devstral, Deepcoder, and other tool-capable models are suffering from a very similar, if not the same, issues at gpt-oss in Continue right now.

[StefanOltmann]

I have the same question. No matter which model I select all have the warning Agent might not work well with this model.

So I like to know which agent actually does work well?
Looking for a 7b and 13b model.

[HaroldStash]

I'm getting very good results w/ qwen3-coder-next on an amd 7900xtx w/ llama.cpp. Even with the yellow triangle it reliably uses the tools and MCP servers provided.

[lordgreg]

hi @StefanOltmann - did you found any good ones?

• qwen2.5-coder 14B,
• gemma4,
• qwen3.5 9B,

none of them make a good agentic coding experience. also looking for something in the 10B direction at most.

Either I got the

read failed because the arguments were invalid, with the following message: Tool read not found

Please try something else or request further instructions.

or UI just showing me a json such as:

[StefanOltmann]

@lordgreg Unfortunately I did not.

OpenCode was able to use gemma4 tools locally, but it's painfully slow.

For now I use OpenCode's free MiniMax 2.5 with decent results.

Not sure if Continue will evolve in something usable for me.

[lordgreg]

I'm struggling to get any model (suitable for 32GB macbook m1, served via ollama) working very much at all with agent mode.

Can anyone offer some decent models (for python/k8s/go coding) that work well in agent mode/for tool calling (including mcp) specifically with the continue plugin in vscode?

It can be as simple an example as 'can you list what charts I have' in the context of helm charts for example.

A typical response would be

{"name":"ls","arguments":{"dirPath":"/charts"}}

from the model, but not in the format continue agent mode appears to like

hi @planetf1 - did you found any good ones?

• qwen2.5-coder 14B,
• gemma4,
• qwen3.5 9B,

none of them make a good agentic coding experience. also looking for something in the 10B direction at most.

Either I got the

read failed because the arguments were invalid, with the following message: Tool read not found

Please try something else or request further instructions.

or UI just showing me a json such as:

[pandego]

A few practical things have mattered more for me than the exact model name:

• In Continue, Agent mode needs a model that really supports tool calling, and if autodetection is wrong you usually need capabilities: [tool_use] in config.yaml.
• MCP only works in Agent mode, so a model that emits raw JSON-ish tool calls but does not follow the provider's actual tool-call format will feel broken even if it looks close.
• On a 32 GB M1, I would bias toward smaller tool-reliable models over larger "sometimes works" ones.

So I would try this order:

1. qwen2.5-coder:7b or another 7B-ish coder model with capabilities: [tool_use]
2. a 14B model only if the 7B is already tool-stable on your machine
3. avoid treating "bigger" as "better" for Agent mode, because tool formatting reliability usually matters more than benchmark IQ here

Minimal config shape:

models:
  - name: Ollama Agent
    provider: ollama
    model: qwen2.5-coder:7b
    roles: [chat, edit, apply]
    capabilities: [tool_use]

If you still get Tool read not found or the UI just prints JSON, I would suspect tool-call compatibility/config before I blame the model size. That symptom usually means the model is not matching the tool schema Continue expects, not just that it is "bad at coding."

[planetf1]

Hi, just to comment as I was OP - not currently using continue sorry. I would say generally that my experience with local models on my m1 32GB hasn't been too positive to get good quality results with low footprint and decent response times. Machine upgrade soon may mean I return to this in future. Thanks for comments above though.

[lordgreg]

@planetf1 which Extension are you using atm if I may ask? :)

[planetf1]

The latest combo I'm currently trying out and the first that seems to do something vaguely useful is:

• omlx 0.3.5
• Qwen3.5-9B-OptiQ-4bit (128k context)
• opencode 1.4.5 (with multiple roles defined & starting in orchestration)