RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pryscraper/tests
History
cryptorugmunch 80b067ea3b feat(secrets): gopass-based secret backend (PRY_SECRET_BACKEND) 
The SECURITY.md contract said "use gopass" but the code only used
os.getenv. The deploy at /srv/pry/ had an .env file with secrets in
it, which violates the SECURITY.md threat model.

New module secrets_backend.py provides:
  get_secret(name, default) - resolves from gopass, env, or file
  set_secret(name, value)   - writes to gopass
  backend_info()            - diagnostic dict for /health or /status

Backends selected by PRY_SECRET_BACKEND env var:
  gopass (default) - reads from gopass at pry/<name>
  env              - reads from os.environ (PRY_<NAME> or PRY_<name>)
  file             - reads from PRY_ENV_FILE (default: PRY_DATA_DIR/.env)
  auto             - tries gopass, falls back to env

Refactored call sites:
  auth.py:        JWT_SECRET (was: os.getenv + ephemeral random default)
  x402.py:        X402_WALLET, X402_FACILITATOR_URL (was: os.getenv)

Seeded initial secrets on Talos (5 entries under pry/):
  jwt_secret, api_key, x402_wallet, x402_facilitator, ollama_url

Updated .env.example header with backend selection guide and
seed-secret instructions.

Tests: 9/9 in test_secrets_backend.py pass. 36 tests in
test_x402_mcp_spec.py + test_secrets_backend.py all pass.

Verified end-to-end:
  >>> import x402
  >>> x402.X402_WALLET
  '0xYourWalletAddressHere'
  >>> import auth
  >>> auth.JWT_SECRET
  'change-me-rotate-quarterly'

Follow-up: rotate jwt_secret and api_key to real random values.
Document the rotation cadence in SECURITY.md.
2026-07-02 20:26:00 +02:00
..
__init__.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
conftest.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_adaptive.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_advanced_features.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_advanced_scraping.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_agency.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_ai_plugin.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_alerter.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_api.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_api_mcp_x402.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_auth_connector.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_block_detector.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_cache.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_camoufox.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_commerce_sync.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_compliance.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_costing.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_crm_sync.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_destinations.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_email_scraper.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_enrichment.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_extractor.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_freshness.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_gdpr.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_infrastructure.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_intelligence.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_jobqueue.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_lazy_load.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_llm_providers.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_markdown_gen.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_marketplace.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_mconfig.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_mcp_production.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_monitor.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_network.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_parser.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_pipeline.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_pipelines.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_proxy_manager.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_quality.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_ratelimit.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_real_features.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_reconciliation.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_referrals.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_reports.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_review.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_scraper.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_sdk.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_secrets_backend.py feat(secrets): gopass-based secret backend (PRY_SECRET_BACKEND) 2026-07-02 20:26:00 +02:00
test_seo_monitor.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_shadow_dom.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_structure_monitor.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_templates.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_training_data.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_ultimate_scraper.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_x402_mcp_spec.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00