|
|
173403379f
|
feat(M9): implement WebFetch and WebSearch with domain filtering and pluggable providers
Milestone M9 implementation adds web access tools with security controls.
New crate: crates/tools/web
WebFetch Features:
- HTTP client using reqwest
- Domain allowlist/blocklist filtering
* Empty allowlist = allow all domains (except blocked)
* Non-empty allowlist = only allow specified domains
* Blocklist always takes precedence
- Redirect detection and blocking
* Redirects to unapproved domains are blocked
* Manual redirect policy (no automatic following)
* Returns error message with redirect URL
- Response capture with metadata
* Status code, content, content-type
* Original URL preserved
WebSearch Features:
- Pluggable provider trait using async-trait
- SearchProvider trait for implementing search APIs
- StubSearchProvider for testing
- SearchResult structure with title, URL, snippet
- Provider name identification
Security Features:
- Case-insensitive domain matching
- Host extraction from URLs
- Relative redirect URL resolution
- Domain validation before requests
- Explicit approval required for cross-domain redirects
Tests added (9 new tests):
Unit tests:
1. domain_filtering_allowlist - Verifies allowlist-only mode
2. domain_filtering_blocklist - Verifies blocklist takes precedence
3. domain_filtering_case_insensitive - Verifies case handling
Integration tests with wiremock:
4. webfetch_domain_whitelist_only - Tests allowlist enforcement
5. webfetch_redirect_to_unapproved_domain - Blocks bad redirects
6. webfetch_redirect_to_approved_domain - Detects good redirects
7. webfetch_blocklist_overrides_allowlist - Blocklist priority
8. websearch_pluggable_provider - Provider pattern works
9. webfetch_successful_request - Basic fetch operation
All 84 tests passing (up from 75).
Note: CLI integration deferred - infrastructure is complete and tested.
Future work will add CLI commands for web-fetch and web-search with
domain configuration.
Dependencies: reqwest 0.12, async-trait 0.1, wiremock 0.6 (test)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
|
2025-11-01 20:23:29 +01:00 |
|
