RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pryscraper/DEVELOPMENT.md
cryptorugmunch bb77eb5f35 chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 
Re-license Pry from full Proprietary to a dual-license model:

- Core engine, extraction, templates (80+), MCP server, x402 payment rail,
  CLI, SDK, browser extension, WordPress plugin, Shopify app, and
  llm_providers: MIT (see LICENSE)
- Anti-detection / stealth subset (15 files): BSL 1.1 with Change Date
  2029-01-01 (see LICENSE-BSL-STEALTH)

BSL files (anti-detection moat):
  ultimate_scraper.py, stealth_engine.py, stealth_scripts/*.js (6),
  camoufox_integration.py, tls_fingerprint.py, cookie_warmer.py,
  behavioral_biometrics.py, adaptive.py, browser_pool.py, network.py,
  captcha_solver.py, shadow_dom.py, lazy_load.py, signup_automator.py,
  auth_connector.py

This enables community contributions to the core engine (templates,
integrations, MCP tools) while protecting the anti-detection techniques
that constitute the actual competitive moat. BSL Additional Use Grant
permits free non-production use; production deployment requires a
commercial license from enterprise@rugmunch.io.

Changes:
- Replace proprietary LICENSE with MIT LICENSE + new LICENSE-BSL-STEALTH
- Add SPDX-License-Identifier headers to 300+ source files
- Add docs/adr/0002-dual-licensing.md (ADR documenting the decision)
- Update README.md: new License section with BSL Additional Use Grant
- Update LICENSING_PRICING_STRATEGY.md: Section 3 (PryScraper) for dual license
- Update AGENTS.md: license line in header + new rule 8 (PRs touching BSL rejected)
- Update pyproject.toml: license = "MIT AND BSL-1.1" + classifiers + license-files
- Update DECISIONS.md index with ADR-0002
- Update STATUS.md (2026-07-03) and PLAN.md sprint goals

Refs: ADR-0002
2026-07-02 19:49:21 +02:00

1.7 KiB
Raw Blame History

//: # (Copyright (c) 2026 Rug Munch Media LLC)

DEVELOPMENT.md — PryScraper

Dev workflow. Install, code, test, commit, PR.

Setup

Prerequisites

  • Python 3.12+ / Node 20+
  • gopass for secrets
  • mise for tool version mgmt (or manual)
  • pre-commit for hooks

Install

git clone https://git.rugmunch.io/RugMunchMedia/pryscraper.git
cd pryscraper
make install
pre-commit install

Environment

# Required env vars (loaded from gopass on deploy, .env locally)
# See .env.example
cp .env.example .env
$EDITOR .env  # fill in test values

Workflow

1. Create a branch

git checkout -b feat/my-feature
# or fix/my-bug, docs/my-doc, chore/my-chore

2. Make changes

  • Write code
  • Add tests
  • Update docs (AGENTS.md, ARCHITECTURE.md, STATUS.md)

3. Run pre-commit locally

make lint
make test
make typecheck
make security

4. Commit (conventional)

make commit   # interactive
# or:
git commit -m "feat(scope): add new feature"

Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore, ops, security

5. Push + PR

git push -u origin feat/my-feature
# Open PR on forgejo: https://git.rugmunch.io/RugMunchMedia/pryscraper/pulls/new

6. Wait for CI

  • All checks must pass
  • Review by CODEOWNERS
  • Squash-merge to main
  • Auto-deploys to Talos (via forgejo webhook)

Daily End-of-Day

make status    # show what's changed
fleet-commit   # commit helper with checklist

Code Style

  • Python: ruff (lint + format), mypy strict
  • TypeScript: eslint + prettier
  • Shell: shellcheck
  • Markdown: vale

See standards/CONVENTIONS.md.