Skip to content

feat: Apify integration#1

Open
JanHranicky wants to merge 11 commits intomainfrom
feat/apify-integration
Open

feat: Apify integration#1
JanHranicky wants to merge 11 commits intomainfrom
feat/apify-integration

Conversation

@JanHranicky
Copy link
Copy Markdown
Collaborator

Overview

Five new tools under deerflow.community.apify:

Tool Name in config Purpose
web_search_tool web_search Drop-in replacement for Tavily/Jina — uses Apify Google Search Scraper
web_fetch_tool web_fetch Drop-in replacement for Jina/Firecrawl — uses Apify Website Content Crawler
apify_actor_discover_tool apify_actor_discover Search Apify Store or fetch an actor's input schema
apify_actor_start_tool apify_actor_start Start an actor run asynchronously, returns run reference immediately
apify_actor_await_tool apify_actor_await Poll a run to completion, stream progress events, return results

The discover/start/await split enables parallel actor runs: start N actors, then await each.

A public skill (skills/public/apify-scraper/) bundles a routing table of known actors (Instagram, TikTok, YouTube, Facebook, Google Maps, e-commerce) with input templates in references/, this is similiar to the one we ship with OpenClaw plugin and the apify-ultimate-scrapper. But has a little different format to fit Deerflow better.


Setup

Follow the repo setup guide, then put your Apify API token in .env under APIFY_API_TOKEN

And enable the tools usage in config.yaml:

tools:
  - name: web_search
    group: web
    use: deerflow.community.apify.tools:web_search_tool
    max_results: 5
    api_key: $APIFY_API_TOKEN

  - name: web_fetch
    group: web
    use: deerflow.community.apify.tools:web_fetch_tool
    crawler_type: cheerio   # or "playwright:firefox" for JS-heavy pages
    api_key: $APIFY_API_TOKEN

  - name: apify_actor_discover
    group: web
    use: deerflow.community.apify.tools:apify_actor_discover_tool
    max_results: 10
    api_key: $APIFY_API_TOKEN

  - name: apify_actor_start
    group: web
    use: deerflow.community.apify.tools:apify_actor_start_tool
    api_key: $APIFY_API_TOKEN

  - name: apify_actor_await
    group: web
    use: deerflow.community.apify.tools:apify_actor_await_tool
    poll_interval_secs: 5
    timeout_secs: 300
    max_items: 50
    api_key: $APIFY_API_TOKEN

@JanHranicky JanHranicky marked this pull request as ready for review April 17, 2026 13:15
@JanHranicky JanHranicky self-assigned this Apr 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants