RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pryscraper/TESTING.md
cryptorugmunch bb77eb5f35 chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 
Re-license Pry from full Proprietary to a dual-license model:

- Core engine, extraction, templates (80+), MCP server, x402 payment rail,
  CLI, SDK, browser extension, WordPress plugin, Shopify app, and
  llm_providers: MIT (see LICENSE)
- Anti-detection / stealth subset (15 files): BSL 1.1 with Change Date
  2029-01-01 (see LICENSE-BSL-STEALTH)

BSL files (anti-detection moat):
  ultimate_scraper.py, stealth_engine.py, stealth_scripts/*.js (6),
  camoufox_integration.py, tls_fingerprint.py, cookie_warmer.py,
  behavioral_biometrics.py, adaptive.py, browser_pool.py, network.py,
  captcha_solver.py, shadow_dom.py, lazy_load.py, signup_automator.py,
  auth_connector.py

This enables community contributions to the core engine (templates,
integrations, MCP tools) while protecting the anti-detection techniques
that constitute the actual competitive moat. BSL Additional Use Grant
permits free non-production use; production deployment requires a
commercial license from enterprise@rugmunch.io.

Changes:
- Replace proprietary LICENSE with MIT LICENSE + new LICENSE-BSL-STEALTH
- Add SPDX-License-Identifier headers to 300+ source files
- Add docs/adr/0002-dual-licensing.md (ADR documenting the decision)
- Update README.md: new License section with BSL Additional Use Grant
- Update LICENSING_PRICING_STRATEGY.md: Section 3 (PryScraper) for dual license
- Update AGENTS.md: license line in header + new rule 8 (PRs touching BSL rejected)
- Update pyproject.toml: license = "MIT AND BSL-1.1" + classifiers + license-files
- Update DECISIONS.md index with ADR-0002
- Update STATUS.md (2026-07-03) and PLAN.md sprint goals

Refs: ADR-0002
2026-07-02 19:49:21 +02:00

1.9 KiB
Raw Blame History

//: # (Copyright (c) 2026 Rug Munch Media LLC)

TESTING.md — PryScraper

Test strategy. Unit, integration, e2e, fixtures.

Test Pyramid

       /\
      /  \      E2E (slow, few)
     /----\
    /      \    Integration (medium, more)
   /--------\
  /          \  Unit (fast, many)
 /____________\
  • Unit tests: test individual functions/classes in isolation
  • Integration tests: test components together (DB, API, auth)
  • E2E tests: test full user flows (slow, run before deploy)

Structure

tests/
├── unit/          # fast, no external deps
├── integration/   # uses test DB, mocks external APIs
├── e2e/           # full stack, runs against staging
├── fixtures/      # test data (mainnet guard: no real addresses/keys)
└── conftest.py    # pytest config

Running

make test                  # all tests
make test-unit            # only unit
make test-integration     # only integration
make test-cov             # with coverage report
pytest tests/unit/test_X.py -v   # single file

Fixtures

Use pytest fixtures. Examples:

  • clean_db — fresh DB per test
  • mock_helius — mock Helius API responses
  • auth_headers — Bearer token for test user
  • sample_token — fake token (test address, NOT mainnet)

NEVER use mainnet data in tests. Pre-commit hook blocks this.

Coverage

  • Target: >80% on changed lines
  • Enforced in CI via --cov-fail-under=80

Mocking

  • httpx for HTTP mocking (use respx or pytest-httpx)
  • unittest.mock for general mocking
  • VCR.py for recording/replaying API calls

Performance Tests

  • locust for load testing (run against staging)
  • Profile with py-spy or cProfile

CI

Every PR runs:

  1. Lint (ruff)
  2. Type check (mypy strict)
  3. Unit tests + coverage
  4. Integration tests (test DB)
  5. Security scan (bandit, gitleaks)
  6. Build Docker image (verify it compiles)

Merge blocked if any fails.