refactor(exceptions): add ruff BLE001; convert 103 broad except Exception

Per CONVENTIONS.md Part 2 ("Never bare except") and CONVENTIONS.md
Part 7 (pre-commit hooks: ruff), blind `except Exception` is now a
lint failure. Pre-existing sites are marked `# noqa: BLE001` for
later manual review; new code must use specific exception types.

Changes:
- pyproject.toml: added "BLE" to ruff lint select. BLE001 is now enforced
- 103 of 166 `except Exception` sites were auto-converted to specific
  types based on context (httpx, json, OSError, subprocess, etc.)
- 62 remaining sites marked with `# noqa: BLE001` for later review
  (mostly generic try/except wrappers that legitimately need broad catch
  for graceful degradation: e.g. compliance LLM fallback must catch
  any error to preserve the regex result)
- 1 manual fix: reverted compliance.py LLM fallback to broad except
  with explicit "must catch all errors" comment + noqa
- 2 files (commerce_sync.py, crm_sync.py) needed `import httpx` added
  so the auto-converted exception references would resolve
- 5 source files (agency, monitor, pipelines, auth_connector,
  llm_providers/registry) renamed "name" -> "<scope>_name" in
  extra={...} dicts because "name" is a reserved LogRecord field

Test impact:
- 14 failing tests -> 1 (the SSE subprocess test is a sandbox limitation,
  pre-existing and unrelated)
- New `test_ble_temp.py` verifies BLE001 catches new violations

Follow-up:
- Each `# noqa: BLE001` site should be reviewed and replaced with a
  specific exception type where possible. The most common legitimate
  broad-catch case is the LLM fallback path; everything else probably
  can be narrowed.
This commit is contained in:
Crypto Rug Munch 2026-07-02 21:04:53 +02:00
parent 117001006f
commit 0200bf3e16
50 changed files with 172 additions and 166 deletions

View file

@ -223,9 +223,9 @@ class PryScraper:
)
if "chromium" in r.stdout:
self.playwright_available = True
except Exception:
except (subprocess.SubprocessError, OSError):
pass
except Exception:
except (subprocess.SubprocessError, OSError):
self.playwright_available = False
def _rotate_ua(self) -> str:
@ -325,7 +325,7 @@ class PryScraper:
result["method"] = "direct"
return result
errors.append(f"direct: thin content ({len(result.get('content', ''))} chars)")
except Exception as e:
except Exception as e: # noqa: BLE001
errors.append(f"direct: {str(e)[:80]}")
# Tier 2: FlareSolverr (bypass Cloudflare/WAF)
@ -344,7 +344,7 @@ class PryScraper:
errors.append(
f"flaresolverr: thin content ({len(result.get('content', ''))} chars)"
)
except Exception as e:
except Exception as e: # noqa: BLE001
errors.append(f"flaresolverr: {str(e)[:80]}")
# Tier 3: Playwright stealth (full browser with fingerprint)
@ -358,7 +358,7 @@ class PryScraper:
result["method"] = "playwright"
return result
errors.append("playwright: thin content")
except Exception as e:
except Exception as e: # noqa: BLE001
errors.append(f"playwright: {str(e)[:80]}")
# Tier 4: Raw retry with different UA + longer timeout
@ -373,7 +373,7 @@ class PryScraper:
result["method"] = "googlebot"
return result
errors.append("googlebot: thin content")
except Exception as e:
except Exception as e: # noqa: BLE001
errors.append(f"googlebot: {str(e)[:80]}")
return {"status": "error", "url": url, "error": "; ".join(errors), "content": ""}
@ -532,14 +532,14 @@ class PryScraper:
current_url, {"limit": max_pages - len(visited)}
)
to_visit.extend((l, depth + 1) for l in links if l not in visited)
except Exception:
except Exception: # noqa: BLE001
continue
return results
async def map_urls(self, url: str, options: dict | None = None) -> list[str]:
try:
return await self._extract_links(url, options or {})
except Exception:
except Exception: # noqa: BLE001
return []
async def _extract_links(self, url: str, options: dict | None = None) -> list[str]:
@ -548,10 +548,10 @@ class PryScraper:
html = None
try:
html = await self._fetch_direct(url, self._build_headers(url), 15)
except Exception:
except Exception: # noqa: BLE001
try:
html = await self._fetch_via_flaresolverr(url, 30)
except Exception:
except Exception: # noqa: BLE001
return []
links = set()
for m in re.finditer(r'href=["\'](https?://[^"\']+)["\']', html):