RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pryscraper/AGENTS.md
cryptorugmunch bb77eb5f35 chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 
Re-license Pry from full Proprietary to a dual-license model:

- Core engine, extraction, templates (80+), MCP server, x402 payment rail,
  CLI, SDK, browser extension, WordPress plugin, Shopify app, and
  llm_providers: MIT (see LICENSE)
- Anti-detection / stealth subset (15 files): BSL 1.1 with Change Date
  2029-01-01 (see LICENSE-BSL-STEALTH)

BSL files (anti-detection moat):
  ultimate_scraper.py, stealth_engine.py, stealth_scripts/*.js (6),
  camoufox_integration.py, tls_fingerprint.py, cookie_warmer.py,
  behavioral_biometrics.py, adaptive.py, browser_pool.py, network.py,
  captcha_solver.py, shadow_dom.py, lazy_load.py, signup_automator.py,
  auth_connector.py

This enables community contributions to the core engine (templates,
integrations, MCP tools) while protecting the anti-detection techniques
that constitute the actual competitive moat. BSL Additional Use Grant
permits free non-production use; production deployment requires a
commercial license from enterprise@rugmunch.io.

Changes:
- Replace proprietary LICENSE with MIT LICENSE + new LICENSE-BSL-STEALTH
- Add SPDX-License-Identifier headers to 300+ source files
- Add docs/adr/0002-dual-licensing.md (ADR documenting the decision)
- Update README.md: new License section with BSL Additional Use Grant
- Update LICENSING_PRICING_STRATEGY.md: Section 3 (PryScraper) for dual license
- Update AGENTS.md: license line in header + new rule 8 (PRs touching BSL rejected)
- Update pyproject.toml: license = "MIT AND BSL-1.1" + classifiers + license-files
- Update DECISIONS.md index with ADR-0002
- Update STATUS.md (2026-07-03) and PLAN.md sprint goals

Refs: ADR-0002
2026-07-02 19:49:21 +02:00

2.9 KiB
Raw Blame History

//: # (Copyright (c) 2026 Rug Munch Media LLC)

AGENTS.md — PryScraper

AI agent contract. Read this before touching anything in this repo.

Status

canonical · owner=crmuncher · last_updated=2026-07-02

What This Repo Is

Multi-source crypto intelligence crawler (munchcrawl). Scraper, extractor, parser, automator, job queue. CLI + MCP server + SDK. License: Dual -- MIT (core) + BSL 1.1 (stealth/anti-detection subset). See LICENSE and LICENSE-BSL-STEALTH.

Type

backend · language=Python 3.12 + FastAPI

Where It Lives

  • Source: https://git.rugmunch.io/RugMunchMedia/pryscraper
  • Deployed: Docker on Talos (/srv/pry/)
  • Port: 8005

Built With

Components

  • /srv/pry/*.py: 83 Python modules (api.py, extractor.py, parser.py, etc.)
  • browser-extension/: Chrome/Firefox extension

Dependencies

  • postgresql (job storage)
  • redis (cache)
  • flaresolverr (Cloudflare bypass, on Hydra)
  • playwright (browser automation)

🚨 CRITICAL RULES

  1. No nested repos. Don't commit other complete project trees inside this one.
  2. No secrets. Never commit .secrets/, .env, *.pem, *.key, *.crt. Use gopass.
  3. No data blobs. Don't commit *.zip, *.parquet, model weights, sqlite files.
  4. No broken files. Shell heredoc accidents must be caught before commit.
  5. No duplicate scaffolds. If a backend/ or frontend/ subdir exists at root, it's either THE app or bloat.
  6. Update STATUS.md before committing. Builders must know where we are.
  7. Read DECISIONS.md before architectural changes. Don't repeat rejected designs.
  8. Respect dual licensing. When contributing, the core/ files are MIT (any PR welcome). The stealth/ files (listed in LICENSE-BSL-STEALTH) are BSL 1.1 -- only Rug Munch Media LLC can modify them. PRs touching BSL files will be rejected.

Commands

make install       # install deps
make dev           # dev server
make test          # tests
make lint          # ruff + format check
make typecheck     # mypy strict
make security      # bandit + gitleaks
make ci            # full pipeline
make commit        # conventional commit
make clean         # remove build artifacts

Architecture

See ARCHITECTURE.md. Update it when components change.

Plan

PLAN.md shows current sprint. Update weekly.

Status

STATUS.md shows where we are RIGHT NOW. Update before each commit.

Owners

  • Primary: crmuncher
  • Backup: TBD

Related Repos

See standards/PRODUCTS.md.