|
|
2970c15dbb
|
feat(url_guard): SSRF and path traversal guard for scraper inputs
- Add url_guard.py with validate_url() that rejects:
- Non-HTTP schemes (file://, ftp://, javascript:)
- Private/reserved IP ranges (RFC 1918, loopback, CGNAT, link-local)
- Internal hostnames (localhost, kubernetes, docker, metadata, etc.)
- Embedded credentials in URLs
- Path traversal sequences (.., %2e%2e%2f, %%2e%%2e%%2f)
- Integrate into scraper.scrape() and UltimateScraper.scrape()
- Use typed InvalidRequestError for structured API error responses
- Add 24 tests covering all rejection cases and valid URLs
- 560 tests passing, ruff clean
|
2026-07-03 12:00:40 +02:00 |
|
