Web Search Plus

> Unified multi-provider web search with Intelligent Auto-Routing — uses multi-signal analysis to automatically select between Serper, Tavily, Querit, Exa, Perplexity (Sonar Pro), You.com, and SearXNG with confidence scoring.

[](https://clawhub.ai) [](https://clawhub.ai) [](https://github.com/robbyczgw-cla/web-search-plus)

---

🧠 Features (v2.9.0)

```bash python3 scripts/search.py -q "how much does iPhone 16 cost" # → Serper (68% confidence) python3 scripts/search.py -q "how does quantum entanglement work" # → Tavily (86% HIGH) python3 scripts/search.py -q "startups similar to Notion" # → Exa (76% HIGH) python3 scripts/search.py -q "companies like stripe.com" # → Exa (100% HIGH - URL detected) python3 scripts/search.py -q "summarize key points on AI" # → You.com (68% MEDIUM - RAG intent) python3 scripts/search.py -q "search privately without tracking" # → SearXNG (74% HIGH - privacy intent) ```

---

🔍 When to Use Which Provider

Built-in Brave Search (OpenClaw default)

Serper (Google Results)

Tavily (AI-Optimized Research)

Querit (Multilingual AI Search)

Exa (Neural Semantic Search)

Perplexity (Sonar Pro via Kilo Gateway)

You.com (RAG/Real-time)

SearXNG (Privacy-First/Self-Hosted)

---

Table of Contents

---

Quick Start

Option A: Interactive Setup (Recommended)

```bash

Run the setup wizard - it guides you through everything

The wizard explains each provider, collects your API keys, and creates `config.json` automatically.

Option B: Manual Setup

```bash

1. Set up at least one API key (or SearXNG instance)

2. Run a search (auto-routed!)

Run a Search

```bash

Auto-routed to best provider

Or specify a provider explicitly

---

Smart Auto-Routing

How It Works

When you don't specify a provider, the skill analyzes your query and routes it to the best provider:

| Query Contains | Routes To | Example | |---------------|-----------|---------| | "price", "buy", "shop", "cost" | Serper | "iPhone 16 price" | | "near me", "restaurant", "hotel" | Serper | "pizza near me" | | "weather", "news", "latest" | Serper | "weather Berlin" | | "how does", "explain", "what is" | Tavily | "how does TCP work" | | "research", "study", "analyze" | Tavily | "climate research" | | "tutorial", "guide", "learn" | Tavily | "python tutorial" | | multilingual, current status, latest updates | Querit | "latest AI policy updates in Germany" | | "similar to", "companies like" | Exa | "companies like Stripe" | | "startup", "Series A" | Exa | "AI startups Series A" | | "github", "research paper" | Exa | "LLM papers arxiv" | | "private", "anonymous", "no tracking" | SearXNG | "search privately" | | "multiple sources", "aggregate" | SearXNG | "results from all engines" |

Examples

```bash

These are all auto-routed to the optimal provider:

Result Caching (introduced in v2.7.x)

Search results are automatically cached for 1 hour to save API costs:

```bash

First request: fetches from API ($)

Second request: uses cache (FREE!)

Output includes: "cached": true

Bypass cache (force fresh results)

View cache stats

Clear all cached results

Custom TTL (in seconds, default: 3600 = 1 hour)

Debug Auto-Routing

See exactly why a provider was selected:

```bash python3 scripts/search.py --explain-routing -q "best laptop to buy" ```

Output: ```json { "query": "best laptop to buy", "selected_provider": "serper", "reason": "matched_keywords (score=2)", "matched_keywords": ["buy", "best"], "available_providers": ["serper", "tavily", "exa"] } ```

Routing Info in Results

Every search result includes routing information:

```json { "provider": "serper", "query": "iPhone 16 price", "results": [...], "routing": { "auto_routed": true, "selected_provider": "serper", "reason": "matched_keywords (score=1)", "matched_keywords": ["price"] } } ```

---

Configuration Guide

Environment Variables

Create a `.env` file or set these in your shell:

```bash

Required: Set at least one

Config File (config.json)

The `config.json` file lets you customize auto-routing and provider defaults:

```json { "defaults": { "provider": "serper", "max_results": 5 }, "auto_routing": { "enabled": true, "fallback_provider": "serper", "provider_priority": ["serper", "tavily", "exa"], "disabled_providers": [], "keyword_mappings": { "serper": ["price", "buy", "shop", "cost", "deal", "near me", "weather"], "tavily": ["how does", "explain", "research", "what is", "tutorial"], "exa": ["similar to", "companies like", "alternatives", "startup", "github"] } }, "serper": { "country": "us", "language": "en" }, "tavily": { "depth": "basic", "topic": "general" }, "exa": { "type": "neural" } } ```

Configuration Examples

#### Example 1: Disable Exa (Only Use Serper + Tavily)

```json { "auto_routing": { "disabled_providers": ["exa"] } } ```

#### Example 2: Make Tavily the Default

```json { "auto_routing": { "fallback_provider": "tavily" } } ```

#### Example 3: Add Custom Keywords

```json { "auto_routing": { "keyword_mappings": { "serper": [ "price", "buy", "shop", "amazon", "ebay", "walmart", "deal", "discount", "coupon", "sale", "cheap" ], "tavily": [ "how does", "explain", "research", "what is", "coursera", "udemy", "learn", "course", "certification" ], "exa": [ "similar to", "companies like", "competitors", "YC company", "funded startup", "Series A", "Series B" ] } } } ```

#### Example 4: German Locale for Serper

```json { "serper": { "country": "de", "language": "de" } } ```

#### Example 5: Disable Auto-Routing

```json { "auto_routing": { "enabled": false }, "defaults": { "provider": "serper" } } ```

#### Example 6: Research-Heavy Config

```json { "auto_routing": { "fallback_provider": "tavily", "provider_priority": ["tavily", "serper", "exa"] }, "tavily": { "depth": "advanced", "include_raw_content": true } } ```

---

Provider Deep Dives

Serper (Google Search API)

#### Strengths | Strength | Description | |----------|-------------| | 🎯 Accuracy | Google's search quality, knowledge graph, featured snippets | | 🛒 Shopping | Product prices, reviews, shopping results | | 📍 Local | Business listings, maps, places | | 📰 News | Real-time news with Google News integration | | 🖼 Images | Google Images search | | ⚡ Speed | Fastest response times (~200-400ms) |

#### Best Use Cases

#### Getting Your API Key 1. Go to [serper.dev](https://serper.dev) 2. Sign up with email or Google 3. Copy your API key from the dashboard 4. Set `SERPER_API_KEY` environment variable

---

Tavily (Research Search)

#### Strengths | Strength | Description | |----------|-------------| | 📚 Research Quality | Optimized for comprehensive, accurate research | | 💬 AI Answers | Returns synthesized answers, not just links | | 📄 Full Content | Can return complete page content (raw_content) | | 🎯 Domain Filtering | Include/exclude specific domains | | 🔬 Deep Mode | Advanced search for thorough research | | 📰 Topic Modes | Specialized for general vs news content |

#### Best Use Cases

#### Getting Your API Key 1. Go to [tavily.com](https://tavily.com) 2. Sign up and verify email 3. Navigate to API Keys section 4. Generate and copy your key 5. Set `TAVILY_API_KEY` environment variable

---

Exa (Neural Search)

#### Strengths | Strength | Description | |----------|-------------| | 🧠 Semantic Understanding | Finds results by meaning, not keywords | | 🔗 Similar Pages | Find pages similar to a reference URL | | 🏢 Company Discovery | Excellent for finding startups, companies | | 📑 Category Filters | Filter by type (company, paper, tweet, etc.) | | 📅 Date Filtering | Precise date range searches | | 🎓 Academic | Great for research papers and technical content |

#### Best Use Cases

#### Getting Your API Key 1. Go to [exa.ai](https://exa.ai) 2. Sign up with email or Google 3. Navigate to API section in dashboard 4. Copy your API key 5. Set `EXA_API_KEY` environment variable

---

SearXNG (Privacy-First Meta-Search)

#### Strengths | Strength | Description | |----------|-------------| | 🔒 Privacy-First | No tracking, no profiling, no data collection | | 🌐 Multi-Engine | Aggregates Google, Bing, DuckDuckGo, and 70+ more | | 💰 Free | $0 API cost (self-hosted, unlimited queries) | | 🎯 Diverse Results | Get perspectives from multiple search engines | | ⚙ Customizable | Choose which engines to use, SafeSearch, language | | 🏠 Self-Hosted | Full control over your search infrastructure |

#### Best Use Cases

#### Setting Up Your Instance ```bash

Docker (recommended, 5 minutes)

Enable JSON API in settings.yml:

search:

formats: [html, json]

1. See [docs.searxng.org](https://docs.searxng.org/admin/installation.html) 2. Deploy via Docker, pip, or your preferred method 3. Enable JSON format in `settings.yml` 4. Set `SEARXNG_INSTANCE_URL` environment variable

---

Usage Examples

Auto-Routed Searches (Recommended)

```bash

Just search — the skill picks the best provider

Serper Options

```bash

Different search types

With time filter

Include images

Different locale

Tavily Options

```bash

Deep research mode

With full page content

Domain filtering

Exa Options

```bash

Category filtering

Date filtering

Find similar pages

SearXNG Options

```bash

Basic search

Specific engines only

SafeSearch (0=off, 1=moderate, 2=strict)

With time filter

Custom instance URL

---

Workflow Examples

🛒 Product Research Workflow

```bash

Step 1: Get product specs (auto-routed to Serper)

Step 2: Check prices (auto-routed to Serper)

Step 3: In-depth reviews (auto-routed to Tavily)

📚 Academic Research Workflow

```bash

Step 1: Understand the topic (auto-routed to Tavily)

Step 2: Find recent papers (Exa)

Step 3: Find implementations (Exa)

🏢 Competitive Analysis Workflow

```bash

Step 1: Find competitors (auto-routed to Exa)

Step 2: Find similar products (Exa)

Step 3: Deep dive comparison (Tavily)

---

Optimization Tips

Cost Optimization

| Tip | Savings | |-----|---------| | Use SearXNG for routine queries | $0 API cost | | Use auto-routing (defaults to Serper, cheapest paid) | Best value | | Use Tavily `basic` before `advanced` | ~50% cost reduction | | Set appropriate `max_results` | Linear cost savings | | Use Exa only for semantic queries | Avoid waste |

Performance Optimization

| Tip | Impact | |-----|--------| | Serper is fastest (~200ms) | Use for time-sensitive queries | | Tavily `basic` faster than `advanced` | ~2x faster | | Lower `max_results` = faster response | Linear improvement |

---

FAQ & Troubleshooting

General Questions

Auto-Routing Questions

Troubleshooting

Check if key is set

Set it

---

API Reference

Output Format

All providers return unified JSON:

```json { "provider": "serper|tavily|exa", "query": "original search query", "results": [ { "title": "Page Title", "url": "https://example.com/page", "snippet": "Content excerpt...", "score": 0.95, "date": "2024-01-15", "raw_content": "Full page content (Tavily only)" } ], "images": ["url1", "url2"], "answer": "Synthesized answer", "knowledge_graph": { }, "routing": { "auto_routed": true, "selected_provider": "serper", "reason": "matched_keywords (score=1)", "matched_keywords": ["price"] } } ```

CLI Options Reference

| Option | Providers | Description | |--------|-----------|-------------| | `-q, --query` | All | Search query | | `-p, --provider` | All | Provider: auto, serper, tavily, querit, exa, perplexity, you, searxng | | `-n, --max-results` | All | Max results (default: 5) | | `--auto` | All | Force auto-routing | | `--explain-routing` | All | Debug auto-routing | | `--images` | Serper, Tavily | Include images | | `--country` | Serper, You | Country code (default: us) | | `--language` | Serper, SearXNG | Language code (default: en) | | `--type` | Serper | search/news/images/videos/places/shopping | | `--time-range` | Serper, SearXNG | hour/day/week/month/year | | `--depth` | Tavily | basic/advanced | | `--topic` | Tavily | general/news | | `--raw-content` | Tavily | Include full page content | | `--querit-base-url` | Querit | Override Querit API base URL | | `--querit-base-path` | Querit | Override Querit API path | | `--exa-type` | Exa | neural/keyword | | `--category` | Exa | company/research paper/news/pdf/github/tweet | | `--start-date` | Exa | Start date (YYYY-MM-DD) | | `--end-date` | Exa | End date (YYYY-MM-DD) | | `--similar-url` | Exa | Find similar pages | | `--searxng-url` | SearXNG | Instance URL | | `--searxng-safesearch` | SearXNG | 0=off, 1=moderate, 2=strict | | `--engines` | SearXNG | Specific engines (google,bing,duckduckgo) | | `--categories` | SearXNG | Search categories (general,images,news) | | `--include-domains` | Tavily, Exa | Only these domains | | `--exclude-domains` | Tavily, Exa | Exclude these domains | | `--compact` | All | Compact JSON output |

---

License

MIT

---

Links