opencode cf376ab3a8 docs(pry): remove hallucinated Zoho CRM from docs + stub crm_sync_zoho.py

AUDIT-2026-Q3.md §Hallucinated features flagged that FEATURES.md claimed
Zoho CRM sync but no code existed. The codebase now ships Salesforce,
HubSpot, Pipedrive, and Close.com in crm_sync.py — no Zoho.

This commit enforces doc-vs-code parity:
  - Remove the Zoho row from FEATURES.md
  - Strip '~~Zoho~~' from USAGE.md and ARCHITECTURE.md
  - Move ROADMAP.md from '✅ Zoho CRM sync' to '⏸ deferred'
  - Add crm_sync_zoho.py as a discoverable stub that raises
    NotImplementedError and points at the audit doc for rationale

PRs welcome: implement Zoho support using the same shape as the other
CRM backends in crm_sync.py.

2026-07-06 20:33:36 +07:00

11 KiB

Raw Permalink Blame History

//: # (Copyright (c) 2026 Rug Munch Media LLC)

title: Pry Feature Catalog status: canonical owner: Rug Munch Media LLC Engineering last_updated: 2026-06-30

Pry Feature Catalog

Scraping & Crawling

Feature	Module	Endpoint	Status
Single URL scrape	`scraper.py`	`POST /v1/scrape`	✅
Multi-page crawl	`scraper.py`	`POST /v1/crawl`	✅
Batch scrape (parallel)	`advanced.py`	`POST /v1/batch`	✅
Batch from file + template	`advanced.py`	`POST /v1/batch-file`	✅
URL discovery (sitemap)	`scraper.py`	`POST /v1/map`	✅
Link analysis	`scraper.py`	`POST /v1/links`	✅

Anti-Detection & Bypass

Feature	Module	Endpoint	Status
10-tier fallback scraper	`ultimate_scraper.py`	Internal	✅
Block detection	`adaptive.py`	`POST /v1/detect-block`	✅
Adaptive strategy rotation	`adaptive.py`	Internal	✅
Browser pool + pre-warming	`browser_pool.py`	Internal	✅
Stealth engine (6 JS scripts)	`stealth_engine.py`	Internal	✅
Stealth scripts	`stealth_scripts/`	Injected	✅
Tor proxy routing (tier documented but disabled)	`network.py`	`POST /v1/config/profile/tor`	⚠️ `aiohttp-socks` missing
SOCKS5 proxy support	`network.py`	Config	✅
User-agent rotation	`scraper.py`	Internal	✅

Document Parsing

Feature	Module	Endpoint	Status
PDF parsing	`parser.py`	`POST /v1/parse`	✅
DOCX parsing	`parser.py`	`POST /v1/parse`	✅
Image OCR	`parser.py`	`POST /v1/parse`	✅
CSV parsing	`parser.py`	`POST /v1/parse`	✅
JSON parsing	`parser.py`	`POST /v1/parse`	✅
Shadow DOM extraction	`shadow_dom.py`	`POST /v1/shadow-dom`	✅
Lazy load / infinite scroll	`lazy_load.py`	`POST /v1/capture/lazy`	✅

Extraction & Structured Data

Feature	Module	Endpoint	Status
JSON schema extraction	`extractor.py`	`POST /v1/extract`	✅
LLM extraction + chunking	`extraction.py`	`POST /v1/extract/llm`	✅
Schema.org / JSON-LD	`extractor.py`	`POST /v1/schema`	✅
Email address extraction	`extractor.py`	`POST /v1/emails`	✅
Structured field extraction	`extraction.py`	`POST /v1/extract/fields`	✅
Field suggestion (AI)	`advanced.py`	`POST /v1/suggest`	✅

AI & Vision

Feature	Module	Endpoint	Status
Vision AI (OpenRouter, 5-model fallback)	`advanced.py`	`POST /v1/vision`	✅
AI summarization (Ollama)	`advanced.py`	`POST /v1/summarize`	✅
AI categorization	`advanced.py`	`POST /v1/categorize`	✅
GPT Action manifest	`ai_plugin.py`	`GET /v1/ai/gpt-manifest`	✅
MCP server config	`ai_plugin.py`	`GET /v1/ai/mcp-config`	✅
~~MCP tool discovery~~	~~`mcp_server.py`~~	~~`GET /mcp/tools`~~	⚠️ not in `api.py`
~~MCP tool execution~~	~~`mcp_server.py`~~	~~`POST /mcp/call`~~	⚠️ not in `api.py`

GET /mcp/tools and POST /mcp/call exist only in mcp_production.py, which is not deployed. Use pry mcp serve (stdio transport) for MCP access — see AUDIT-2026-Q3.md.

Browser Automation

Feature	Module	Endpoint	Status
Step-based automation	`automator.py`	`POST /v1/automate`	✅
Screenshot capture	`automator.py`	`POST /v1/screenshot`	✅
Persistent browser sessions	`sessions.py`	`POST /v1/session/create`	✅
Session list	`sessions.py`	`GET /v1/sessions`	✅
Session save/restore	`sessions.py`	`POST /v1/session/save`	✅
Session close	`sessions.py`	`POST /v1/session/close`	✅
Action recorder	`advanced.py`	`POST /v1/record/start`	✅

CAPTCHA Solving

Feature	Module	Endpoint	Status
CAPTCHA solving (6 providers)	`captcha_solver.py`	`POST /v1/auth/captcha`	✅
Auto-fallback across providers	`captcha_solver.py`	Internal	✅

Authentication

Feature	Module	Endpoint	Status
Credential vault (encrypted)	`auth_connector.py`	`POST /v1/auth/credentials`	✅
SSO login script generation	`auth_connector.py`	`POST /v1/auth/sso`	✅
Session health check	`auth_connector.py`	`POST /v1/auth/session/health`	✅
Signup automation	`signup_automator.py`	Internal	✅
Account pool management	`account_manager.py`	Internal	✅

Monitoring & Change Detection

Feature	Module	Endpoint	Status
Page watch / change detection	`freshness.py`	`POST /v1/watch`	✅
Diff tracking	`advanced.py`	`POST /v1/diff`	✅
Content freshness dashboard	`freshness.py`	`GET /v1/freshness/dashboard`	✅
Scheduled monitors (cron)	`monitor.py`	`POST /v1/monitor`	✅
Monitor run on-demand	`monitor.py`	`POST /v1/monitor/run`	✅
Monitor list / delete	`monitor.py`	`GET /v1/monitors`	✅
Structure monitoring	`structure_monitor.py`	`POST /v1/structure/check`	✅

Alerts & Notifications

Feature	Module	Endpoint	Status
Multi-channel alerts	`alerter.py`	`POST /v1/alert/send`	✅
Supported channel listing	`alerter.py`	`GET /v1/alert/channels`	✅
Webhook support	`destinations.py`	Config	✅
Slack integration	`destinations.py`	Config	✅

Quality & Review

Feature	Module	Endpoint	Status
Content quality scoring	`quality.py`	`POST /v1/quality/check`	✅
Quality history	`quality.py`	`GET /v1/quality/history`	✅
Human review queue	`review.py`	`POST /v1/review/submit`	✅
Review approve/reject	`review.py`	`POST /v1/review/{id}/approve`	✅
Review queue listing	`review.py`	`GET /v1/reviews`	✅

Data Export & Integration

Feature	Module	Endpoint	Status
~~Multi-format export (JSON, CSV, RSS, TXT, SQL)~~	`destinations.py`	`POST /v1/export`	⚠️ `pryfile.py` only does `json`/`csv`; RSS/TXT/SQL not implemented
Webhook delivery	`destinations.py`	Config	✅
S3 upload	`destinations.py`	Config	✅
GCS upload	`destinations.py`	Config	✅
SFTP upload	`destinations.py`	Config	✅
Data transformation	`advanced.py`	`POST /v1/transform`	✅

Commerce & CRM

Feature	Module	Endpoint	Status
WooCommerce product sync	`commerce_sync.py`	`POST /v1/commerce/sync`	✅
Shopify product sync	`commerce_sync.py`	`POST /v1/commerce/sync`	✅
Shopify app backend	`shopify-app/`	External	✅
Salesforce CRM sync	`crm_sync.py`	`POST /v1/crm/sync`	✅
HubSpot CRM sync	`crm_sync.py`	`POST /v1/crm/sync`	✅

Pipelines

Feature	Module	Endpoint	Status
Pipeline definition validation	`pipelines.py`	`POST /v1/pipelines/validate`	✅
Pipeline execution	`pipelines.py`	`POST /v1/pipelines/run`	✅
Pipeline CRUD	`pipelines.py`	CRUD endpoints	✅
Hook registration	`pipeline.py`	`POST /v1/pipeline/hook`	✅
Hook execution	`pipeline.py`	`POST /v1/pipeline/run`	✅
Schema pipeline	`pipelines.py`	`POST /v1/pipelines/schema`	✅

Intelligence & Competitive Analysis

Feature	Module	Endpoint	Status
Competitor snapshot	`intelligence.py`	`POST /v1/intel/snapshot`	✅
Intel comparison (diff)	`intelligence.py`	`POST /v1/intel/compare`	✅
Intel report generation	`intelligence.py`	`GET /v1/intel/report`	✅
Tech stack detection	`enrichment.py`	`POST /v1/enrichment/analyze`	✅

Feature	Module	Endpoint	Status
Compliance check (URL)	`compliance.py`	`POST /v1/compliance/check`	✅
Consent recording	`gdpr.py`	`POST /v1/gdpr/consent`	✅
Consent check / revoke	`gdpr.py`	`GET /v1/gdpr/consent/{id}`	✅
Data deletion request	`gdpr.py`	`POST /v1/gdpr/deletion/request`	✅
Deletion execution	`gdpr.py`	`POST /v1/gdpr/deletion/execute`	✅
Data retention policy	`gdpr.py`	`GET /v1/gdpr/retention`	✅
Data portability (export)	`gdpr.py`	`POST /v1/gdpr/export`	✅
Compliance audit log	`gdpr.py`	`GET /v1/gdpr/audit`	✅

Training Data

Feature	Module	Endpoint	Status
PII/copyright stripping	`training_data.py`	`POST /v1/training/clean`	✅
Dataset generation	`training_data.py`	`POST /v1/training/dataset`	✅
Dataset CRUD	`training_data.py`	CRUD endpoints	✅
License classification	`training_data.py`	Internal	✅

Agency / White-Label

Feature	Module	Endpoint	Status
Agency profile creation	`agency.py`	`POST /v1/agency/create`	✅
Agency branding	`agency.py`	`PUT /v1/agency/{id}/branding`	✅
Client sub-account management	`agency.py`	`POST /v1/agency/{id}/clients`	✅
Usage analytics	`agency.py`	`GET /v1/agency/{id}/analytics`	✅
Client quota enforcement	`agency.py`	`GET /v1/client/{id}/quota`	✅

Reporting

Feature	Module	Endpoint	Status
Report generation	`reports.py`	`POST /v1/reports/generate`	✅
Report list / get	`reports.py`	`GET /v1/reports`	✅

Templates

Feature	Module	Endpoint	Status
Template listing	`template_engine.py`	`GET /v1/templates`	✅
Template detail	`template_engine.py`	`GET /v1/templates/{id}`	✅
~~Template generation~~	~~`template_engine.py`~~	~~`POST /v1/templates/generate`~~	❌ endpoint not registered in `api.py`
Pre-built templates	110 JSON files	`templates/`	⚠️ ~21/110 pass structural validation; ~30–40% work end-to-end (see AUDIT-2026-Q3.md)

SEO

Feature	Module	Endpoint	Status
SEO analysis	`seo_monitor.py`	`POST /v1/seo/analyze`	✅
SEO change tracking	`seo_monitor.py`	`POST /v1/seo/track`	✅
Keyword analysis	`seo_monitor.py`	`POST /v1/seo/keywords`	✅

Infrastructure

Feature	Module	Status
Health probes (liveness/readiness)	`api.py`	✅
Prometheus metrics	`api.py`	✅
Rate limiting (token bucket)	`ratelimit.py`	✅
LRU cache + Redis	`cache.py`	✅
Circuit breaker per-domain	`advanced.py`	✅
WebSocket streaming	`pryextras.py`	✅
Async job queue	`jobqueue.py`	✅
Docker Compose deployment	`docker-compose.yml`	✅
CI/CD (GitHub Actions)	`.github/workflows/ci.yml`	✅
Pre-commit hooks	`.pre-commit-config.yaml`	✅
Browser extension (Chrome)	`browser-extension/`	✅
WordPress plugin	`wordpress-plugin/`	✅

11 KiB Raw Permalink Blame History Unescape Escape

//: # (Copyright (c) 2026 Rug Munch Media LLC)

title: Pry Feature Catalog status: canonical owner: Rug Munch Media LLC Engineering last_updated: 2026-06-30

Pry Feature Catalog

Scraping & Crawling

Anti-Detection & Bypass

Document Parsing

Extraction & Structured Data

AI & Vision

Browser Automation

CAPTCHA Solving

Authentication

Monitoring & Change Detection

Alerts & Notifications

Quality & Review

Data Export & Integration

Commerce & CRM

Pipelines

Intelligence & Competitive Analysis

Compliance & GDPR

Training Data

Agency / White-Label

Reporting

Templates

SEO

Infrastructure

11 KiB

Raw Permalink Blame History