chore(lint): auto-fix 253 of 283 ruff issues (F401, I001, E402, RUF100, UP037, SIM105)
Some checks failed
CI / lint (push) Failing after 2s
CI / typecheck (push) Failing after 2s
CI / test (push) Failing after 2s
CI / Secret scan (gitleaks) (push) Failing after 1s
CI / Security audit (bandit) (push) Failing after 2s

Mass ruff auto-fix:
  - ruff check --fix: 109 issues fixed (F401 unused imports,
    I001 unsorted imports, UP037 quoted annotations, SIM105
    suppressible exception, RUF100 unused-noqa)
  - ruff check --fix --unsafe-fixes: 22 additional issues
  - ruff format: 70 files reformatted
  - Manual pass: fix 16 misplaced import httpx lines
  - Manual pass: fix remaining E402 (import-after-docstring)

Result: 283 errors -> 30 errors.

The remaining 30 are real issues that need manual review:
  5 F401 unused-import (likely auto-generated stubs)
  5 F821 undefined-name (real bugs in code that references
    redis/pydantic/LLMRegistry without imports)
  3 BLE001 (the compliance LLM fallback is intentional; the
    other two are real)
  3 RUF012 mutable-class-default
  3 SIM105, 3 SIM117, 2 E722, 2 E741
  1 B007, 1 B025, 1 E402, 1 RUF200 (pyproject.toml issue)

Tests: 436/437 pass (1 pre-existing SSE sandbox failure).
format check + import sort: now clean.
make ci: still gated on the 30 remaining real issues.
Follow-up: triage the 30 issues file-by-file.
This commit is contained in:
Crypto Rug Munch 2026-07-02 21:51:25 +02:00
parent e60a62a07a
commit a7c30b12cd
85 changed files with 2374 additions and 1071 deletions

View file

@ -217,9 +217,12 @@ class PryScraper:
if not self.playwright_available:
try:
import subprocess
r = subprocess.run(
["python3", "-m", "playwright", "install", "--dry-run", "chromium"],
capture_output=True, text=True, timeout=10
capture_output=True,
text=True,
timeout=10,
)
if "chromium" in r.stdout:
self.playwright_available = True
@ -258,13 +261,11 @@ class PryScraper:
if indicator.lower() in html_lower:
return True
# Check for very short / challenge-looking pages
if (
return bool(
len(html) < 2000
and "document" in html_lower
and ("var " in html_lower or "function" in html_lower)
):
return True
return False
)
def _quality_score(self, content: str) -> int:
"""Score content quality 0-100. Low score triggers retry with better tier."""
@ -288,6 +289,7 @@ class PryScraper:
# Use UltimateScraper for the 10-tier fetching
if self._ultimate is None:
from ultimate_scraper import UltimateScraper
self._ultimate = UltimateScraper()
result = await self._ultimate.scrape(url, opts)
@ -563,5 +565,3 @@ class PryScraper:
if urlparse(link).netloc == base_domain:
links.add(link.split("#")[0].rstrip("/"))
return sorted(links)[: opts.get("limit", 50)]