RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(logging): add structlog + JSON logging (CONVENTIONS.md Part 5)

Browse source 
Pry logs are now JSON objects with the required fields (timestamp,
level, service, event, plus key-value pairs). This is the standard
required by CONVENTIONS.md Part 5 and is what makes the service
operable in production (Loki, ELK, etc. can index the structured
records).

New module logging_config.py:
  setup_logging(level, fmt) - configure once at process startup
  get_logger(name)         - get a structlog logger; falls back to stdlib
  is_configured()          - diagnostic for /health

Configuration via env vars:
  PRY_LOG_FORMAT=json|console   (default json)
  PRY_LOG_LEVEL=DEBUG|INFO|...  (default INFO)
  PRY_LOG_STRICT_EXTRAS=1       (default unset = lenient)

Backward compatibility:
  - stdlib logging.getLogger(__name__) calls still work
  - setup_logging bridges stdlib through structlog's formatter
  - In lenient mode, extra={...} keys that collide with reserved
    LogRecord names (e.g. 'name') are moved to an `extra` sub-dict
    so existing code doesn't crash

Wired in:
  api.py: setup_logging() at module import time; lifespan log uses
          structlog style (logger.info("event", key="value") without
          the `extra={...}` wrapper)
  pyproject.toml: structlog>=24.0.0 dep added

Fixed source files that used reserved LogRecord keys in extra={...}:
  agency.py:        "name" -> "agency_name"
  auth_connector.py: "name" -> "credential_name"
  monitor.py:       "name" -> "monitor_name"
  pipelines.py:     "name" -> "pipeline_name"
  llm_providers/registry.py: "name" -> "provider_name"
These would have crashed with KeyError "Attempt to overwrite 'name' in
LogRecord" the moment a real log handler was attached.

Tests: 8/8 in test_logging_config.py pass. Full test suite went from
14 failures -> 2 (one is the SSE subprocess test that doesn't work in
this sandbox; one was the openapi title test that I also fixed in
this commit).

Documentation: DEVELOPMENT.md now has a full "Logging" section with
quick-start, config, and the reserved-key gotcha.
This commit is contained in:
Crypto Rug Munch 2026-07-02 20:55:41 +02:00
parent 17b16c8666
commit 117001006f
12 changed files with 463 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

14
api.py
View file

@ -60,13 +60,25 @@ from x402_middleware import X402Middleware
config = PryConfig()
logger = logging.getLogger(__name__)
# Configure structured (JSON) logging for the whole process. Per
# CONVENTIONS.md Part 5, all Pry logs must be JSON with timestamp, level,
# service, and event. setup_logging() bridges stdlib logging through
# structlog so that any logger.info("event", k=v) becomes a JSON record.
try:
from logging_config import setup_logging, get_logger as _get_logger
setup_logging()
logger = _get_logger(__name__)
except ImportError:
# logging_config not available (e.g., minimal install); use stdlib
pass
# ── Init ──
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
"""Startup: validate deps. Shutdown: cleanup clients."""
logger.info("pry_startup", extra={"version": "3.0.0"})
logger.info("pry_startup", version="3.0.0")
get_pipeline() # Initialize pipeline
yield
logger.info("pry_shutdown")