RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pryscraper/templates
History
cryptorugmunch 0200bf3e16 refactor(exceptions): add ruff BLE001; convert 103 broad except Exception 
Per CONVENTIONS.md Part 2 ("Never bare except") and CONVENTIONS.md
Part 7 (pre-commit hooks: ruff), blind `except Exception` is now a
lint failure. Pre-existing sites are marked `# noqa: BLE001` for
later manual review; new code must use specific exception types.

Changes:
- pyproject.toml: added "BLE" to ruff lint select. BLE001 is now enforced
- 103 of 166 `except Exception` sites were auto-converted to specific
  types based on context (httpx, json, OSError, subprocess, etc.)
- 62 remaining sites marked with `# noqa: BLE001` for later review
  (mostly generic try/except wrappers that legitimately need broad catch
  for graceful degradation: e.g. compliance LLM fallback must catch
  any error to preserve the regex result)
- 1 manual fix: reverted compliance.py LLM fallback to broad except
  with explicit "must catch all errors" comment + noqa
- 2 files (commerce_sync.py, crm_sync.py) needed `import httpx` added
  so the auto-converted exception references would resolve
- 5 source files (agency, monitor, pipelines, auth_connector,
  llm_providers/registry) renamed "name" -> "<scope>_name" in
  extra={...} dicts because "name" is a reserved LogRecord field

Test impact:
- 14 failing tests -> 1 (the SSE subprocess test is a sandbox limitation,
  pre-existing and unrelated)
- New `test_ble_temp.py` verifies BLE001 catches new violations

Follow-up:
- Each `# noqa: BLE001` site should be reviewed and replaced with a
  specific exception type where possible. The most common legitimate
  broad-catch case is the LLM fallback path; everything else probably
  can be narrowed.
2026-07-02 21:04:53 +02:00
..
abcnews-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
adidas-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
airbnb-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
aliexpress-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
aljazeera-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
amazon-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
amazon-search.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
arstechnica-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
asos-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
associated-press-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
axios-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
bbc-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
bestbuy-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
bhphotovideo-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
bloomberg-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
booking-hotel.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
businessinsider-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
cbsnews-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
chewy-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
cnet-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
cnn-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
coinmarketcap-coin.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
costco-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
coverage_report.html chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
craigslist-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
crate-and-barrel-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
crunchbase-company.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
cvs-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
dailymail-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
dicks-sporting-goods.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
dw-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
ebay-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
economist-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
espn-sports.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
etsy-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
facebook-marketplace.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
financialtimes-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
forbes-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
foxnews-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
france24-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
g2-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
gap-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
generic-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
github-repo.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
glassdoor-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
google-maps-place.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
google-patent.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
h-and-m-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
hackernews-post.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
homedepot-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
ikea-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
imdb-movie.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
indeed-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
jcpenney-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
kohls-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
linkedin-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
linkedin-profile.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
lowes-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
lululemon-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
macys-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
medium-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
monster-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
nbcnews-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
newegg-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
nike-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
nordstrom-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
nytimes-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
officedepot-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
old-navy-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
petsmart-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
politico-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
pottery-barn-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
producthunt-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
realtor-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
reddit-post.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
rei-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
reuters-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
samsclub-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
scmp-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
sec-edgar-filing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
sephora-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
staples-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
substack-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
target-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
techcrunch-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_live.py refactor(exceptions): add ruff BLE001; convert 103 broad except Exception 2026-07-02 21:04:53 +02:00
theguardian-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
theverge-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
tiktok-profile.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
time-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
tripadvisor-attraction.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
trustpilot-reviews.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
twitter-profile.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
ulta-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
usatoday-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
uspto-patent.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
validate_templates.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
walgreens-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
walmart-grocery-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
walmart-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
washingtonpost-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
wayfair-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
west-elm-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
wikipedia-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
wired-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
wsj-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
yahoo-finance-quote.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
yelp-business.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
youtube-video.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
zara-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
zerohedge-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
zillow-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
ziprecruiter-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00