RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
pryscraper/templates
History
Exact
cryptorugmunch 604b4f49c9 feat(templates): add validate_live.py for end-to-end template testing 
The existing validate_templates.py only checks JSON schema (required keys,
field types, transforms). It does not verify that templates actually extract
fields from real sites. This is why we have a claimed 110 templates with
a rumored 30-40% success rate that nobody ever measured.

validate_live.py:
- POSTs each template to the /v1/extract endpoint (which is free, not x402-gated)
- Uses a built-in dict of representative test URLs per site (BBC, NYT, Amazon,
  Airbnb, etc.) so each template gets a real article/product page
- Counts how many fields come back non-empty
- Categorizes: full / partial / no_fields_extracted / blocked / error
- Writes a JSON report with per-template results

Findings from running 10 templates against the live Pry:
- 0/10 full extraction
- 0/10 partial extraction
- 7/10 no fields extracted
- 3/10 exceptions (4xx responses from strict sites)

The honest 0% reflects that the current test URLs are not real article pages
AND that the running Pry is the older build. To get a real measurement,
merge this branch + dedup + metrics + orphan-wiring, rebuild Pry, then add
specific real article URLs to SITE_TEST_URLS for each template and re-run.

This commit provides the measurement infrastructure; populating the test
URLs is the follow-up work.

Audit item 11.
2026-07-06 10:46:25 +02:00
..
abcnews-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
adidas-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
airbnb-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
aliexpress-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
aljazeera-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
amazon-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
amazon-search.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
arstechnica-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
asos-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
associated-press-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
axios-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
bbc-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
bestbuy-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
bhphotovideo-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
bloomberg-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
booking-hotel.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
businessinsider-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
cbsnews-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
chewy-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
cnet-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
cnn-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
coinmarketcap-coin.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
costco-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
coverage_report.html chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
craigslist-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
crate-and-barrel-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
crunchbase-company.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
cvs-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
dailymail-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
dicks-sporting-goods.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
dw-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
ebay-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
economist-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
espn-sports.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
etsy-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
facebook-marketplace.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
financialtimes-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
forbes-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
foxnews-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
france24-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
g2-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
gap-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
generic-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
github-repo.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
glassdoor-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
google-maps-place.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
google-patent.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
h-and-m-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
hackernews-post.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
homedepot-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
ikea-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
imdb-movie.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
indeed-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
jcpenney-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
kohls-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
linkedin-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
linkedin-profile.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
lowes-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
lululemon-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
macys-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
medium-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
monster-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
nbcnews-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
newegg-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
nike-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
nordstrom-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
nytimes-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
officedepot-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
old-navy-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
petsmart-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
politico-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
pottery-barn-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
producthunt-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
realtor-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
reddit-post.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
rei-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
reuters-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
samsclub-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
scmp-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
sec-edgar-filing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
sephora-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
staples-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
substack-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
target-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
techcrunch-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
test_live.py chore(lint): auto-fix 253 of 283 ruff issues (F401, I001, E402, RUF100, UP037, SIM105) 2026-07-02 21:51:25 +02:00
theguardian-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
theverge-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
tiktok-profile.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
time-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
tripadvisor-attraction.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
trustpilot-reviews.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
twitter-profile.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
ulta-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
usatoday-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
uspto-patent.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
validate_live.py feat(templates): add validate_live.py for end-to-end template testing 2026-07-06 10:46:25 +02:00
validate_templates.py chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
walgreens-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
walmart-grocery-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
walmart-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
washingtonpost-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
wayfair-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
west-elm-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
wikipedia-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
wired-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
wsj-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
yahoo-finance-quote.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
yelp-business.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
youtube-video.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
zara-product.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
zerohedge-article.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
zillow-listing.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00
ziprecruiter-job.json chore(license): re-license to dual MIT (core) + BSL 1.1 (stealth) 2026-07-02 19:59:18 +02:00