RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pryscraper/PLAN.md
cryptorugmunch bb77eb5f35 chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 
Re-license Pry from full Proprietary to a dual-license model:

- Core engine, extraction, templates (80+), MCP server, x402 payment rail,
  CLI, SDK, browser extension, WordPress plugin, Shopify app, and
  llm_providers: MIT (see LICENSE)
- Anti-detection / stealth subset (15 files): BSL 1.1 with Change Date
  2029-01-01 (see LICENSE-BSL-STEALTH)

BSL files (anti-detection moat):
  ultimate_scraper.py, stealth_engine.py, stealth_scripts/*.js (6),
  camoufox_integration.py, tls_fingerprint.py, cookie_warmer.py,
  behavioral_biometrics.py, adaptive.py, browser_pool.py, network.py,
  captcha_solver.py, shadow_dom.py, lazy_load.py, signup_automator.py,
  auth_connector.py

This enables community contributions to the core engine (templates,
integrations, MCP tools) while protecting the anti-detection techniques
that constitute the actual competitive moat. BSL Additional Use Grant
permits free non-production use; production deployment requires a
commercial license from enterprise@rugmunch.io.

Changes:
- Replace proprietary LICENSE with MIT LICENSE + new LICENSE-BSL-STEALTH
- Add SPDX-License-Identifier headers to 300+ source files
- Add docs/adr/0002-dual-licensing.md (ADR documenting the decision)
- Update README.md: new License section with BSL Additional Use Grant
- Update LICENSING_PRICING_STRATEGY.md: Section 3 (PryScraper) for dual license
- Update AGENTS.md: license line in header + new rule 8 (PRs touching BSL rejected)
- Update pyproject.toml: license = "MIT AND BSL-1.1" + classifiers + license-files
- Update DECISIONS.md index with ADR-0002
- Update STATUS.md (2026-07-03) and PLAN.md sprint goals

Refs: ADR-0002
2026-07-02 19:49:21 +02:00

2.1 KiB
Raw Blame History

//: # (Copyright (c) 2026 Rug Munch Media LLC)

PLAN.md — PryScraper

Current sprint. 21-day cycles per ADR-0012.

Status

current_sprint=2026-Q3-S1 · started=2026-07-01 · ends=2026-07-21

Sprint Goals

  1. DONE Re-license to dual MIT + BSL 1.1 (ADR-0002) - done 2026-07-03
  2. WIP Deploy repo to /srv/pry/ - wire the new api.py (4,668 lines, 190 endpoints) so MCP and x402 actually serve
  3. WIP Wire auth - JWT/API-key middleware on the live deploy so we can monetize

In Progress

  • Re-license the codebase: dual MIT (core) + BSL 1.1 (stealth)
  • Add SPDX headers to all 300+ files
  • Write ADR-0002 documenting the decision
  • Deploy updated api.py with MCP and x402 endpoints to /srv/pry/
  • Set PRY_X402_WALLET and PRY_X402_FACILITATOR env vars on Talos
  • Run templates/validate_templates.py end-to-end, publish honest success rate

Backlog (this sprint)

  • Migrate ~/.pry/* JSON storage to SQLite (concurrency + multi-tenant)
  • Split api.py (4,668 lines) into router modules (cap 500 lines each)
  • Replace 159 broad except Exception with specific types + add ruff BLE001
  • Surface proxy_referrals.py and referrals.py in /v1/referrals API
  • Publish browser extension (Chrome Web Store) and WordPress plugin (wordpress.org)
  • Deploy Cloudflare Worker mcp-worker.js so MCP server is publicly hosted
  • Register on Smithery, Glama, PulseMCP MCP directories
  • Build pry.dev landing page with comparison table (Pry vs Firecrawl vs ScrapingBee)

Risks

  • TODO: anything blocking progress

Done (this sprint)

  • None yet — sprint just started.

Carry-over from previous sprint

  • None.

Definition of Done

  • Code complete
  • Tests written (>80% coverage on changed lines)
  • Pre-commit hooks pass (ruff, mypy, gitleaks, bandit)
  • CI passes on forgejo
  • Conventional commit message
  • PR reviewed + merged
  • Deployed to staging
  • Smoke-tested in production
  • Observability in place (metrics, logs, alerts)
  • Docs updated (AGENTS.md, ARCHITECTURE.md, README.md)
  • STATUS.md updated