Skip /config/dump pre-flight for md/llm endpoints in playground - #2224
Conversation
…yground md/llm runs died before their request was sent: the pre-flight sends the legacy 'code' field, which 0.9.3 rejects on untrusted requests (fixes #2222).
ntohidi
left a comment
There was a problem hiding this comment.
Diagnosis is right and I'd take this — md/llm were failing on a pre-flight whose result they never used, and dropping the isCrawlEndpoint check is what actually unblocks crawl_stream. Both good.
But this fixes half of #2222, and the other half is worth saying out loud before the issue gets closed.
/config/dump takes {type, params} now. The playground still sends {type, code}, and code is on the untrusted forbidden list, so that call always 400s — for crawl too, not just md/llm. What's left on crawl is the regex fallback, which means:
- Default snippet (
stream=True, cache_mode=CacheMode.BYPASS) → fallback fires, request goes out as{stream: true}. Thecache_modeline is silently dropped. Looks like it worked. - Switch type to BrowserConfig → that template has no
stream=True, so no fallback, hard✖ config errorand the run aborts. The panel is unusable.
So the Advanced Config editor is dead on the one endpoint that still shows it, and on the default path it quietly discards whatever the user typed beyond stream. That's arguably worse than the loud error md was giving.
Not asking you to fix it here — the real fix isn't a playground tweak. The server dropped code on purpose (it was a construction oracle), so a Python-snippet editor can't come back. It needs either a params/JSON editor or the panel removed, and that's a separate PR with a UI decision in it.
Two small things in this one, fine as follow-ups:
pyConfigToJson()is what writes#cfg-status.md/llmno longer call it, so a✖ config errorleft over from a previouscrawlrun stays on screen.- The fallback comment says "so shouldUseStream + server load agree", which reads like the fallback is a normal mode. It's a workaround for a broken pre-flight. A line saying so would save the next person some digging.
Merging this as the one-line unblock. Will link a follow-up issue for the editor so it doesn't get closed with #2222.
|
Follow-up filed as #2260 for the Advanced Config panel. |
Summary
Fixes #2222
The playground validated the Advanced Config editor via
/config/dumpfor every endpoint, includingmdandllmwhere the panel is hidden and its result is never used. That pre-flight sends the legacy{ type, code }protocol, which the server removed in the 0.8.x security fixes and rejects loudly since 0.9.3 (codeis a forbidden field on untrusted requests). The fallback only rescuedcrawl, somd/llm(andcrawl_stream) aborted withfield 'code' is not permitted on CrawlerRunConfig from an untrusted requestbefore their request was ever sent.The fix skips the
/config/dumppre-flight unless the endpoint iscrawlorcrawl_stream. Behavior inside the guard is unchanged.List of files changed and why
deploy/docker/static/playground/index.html- wrap the/config/dumppre-flight inrunCrawl()with an endpoint guard somd/llmskip it; drop the now-redundantisCrawlEndpointcheck inside the catch fallback.How Has This Been Tested?
Manually verified and tested the playground
and
Drove the real playground UI with headless Chromium against a clean
unclecode/crawl4ai:0.9.3Docker container (the affected release):md: fit / raw / bm25+query / llm-filter, both cache modes - all succeed; onlyPOST /mdfires, no/config/dumpcallllmwith a question - succeeds viaGET /llm/{url}?q=...crawlwith the default snippet - behavior unchanged:/config/dump-> fallback ->/crawl, succeedscrawl_streamwith the default snippet - now works (streams from/crawl/stream; on stock 0.9.3 it aborted likemd)md/llmChecklist: