RugMunchMedia/pryscraper
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(pry): Phase 0 infra hardening — metrics, redis/postgres, pinned deps, observability #8

Closed
cryptorugmunch wants to merge 5 commits from feat/phase0-rebased into main
pull from: feat/phase0-rebased
merge into: RugMunchMedia:main
Conversation 0 Commits 5 Files changed 11 +172 -28
cryptorugmunch commented 2026-07-03 03:05:14 +02:00
Owner
Copy link

Reconciles the Phase 0 infra-hardening commits on top of the latest main (post Alembic/SQLAlchemy migration PRs).

Changes:

  • Adds /metrics endpoint + Prometheus counters/histograms for requests and scrapes
  • Wires REQUEST_COUNT and REQUEST_LATENCY into PryHttpMiddleware
  • Tracks scrape metrics in routers/scraping.py (direct, lazy, crawl, map, batch)
  • Adds Redis + Postgres services to docker-compose.yml
  • Fixes Ollama URL default to Talos (100.104.130.92:11434)
  • Adds LLMRegistry.complete_or_empty() graceful fallback for API boundaries
  • Regenerates requirements.txt; adds pinned requirements.lock
  • Updates STATUS.md

Verification:

  • ruff check . clean
  • pytest tests/test_api_scraping.py tests/test_api_templates.py tests/test_api.py tests/test_apify_schema.py — 23 passed
Reconciles the Phase 0 infra-hardening commits on top of the latest main (post Alembic/SQLAlchemy migration PRs). Changes: - Adds `/metrics` endpoint + Prometheus counters/histograms for requests and scrapes - Wires `REQUEST_COUNT` and `REQUEST_LATENCY` into `PryHttpMiddleware` - Tracks scrape metrics in `routers/scraping.py` (`direct`, `lazy`, `crawl`, `map`, `batch`) - Adds Redis + Postgres services to `docker-compose.yml` - Fixes Ollama URL default to Talos (`100.104.130.92:11434`) - Adds `LLMRegistry.complete_or_empty()` graceful fallback for API boundaries - Regenerates `requirements.txt`; adds pinned `requirements.lock` - Updates `STATUS.md` Verification: - `ruff check .` clean - `pytest tests/test_api_scraping.py tests/test_api_templates.py tests/test_api.py tests/test_apify_schema.py` — 23 passed
cryptorugmunch added 5 commits 2026-07-03 03:05:15 +02:00
docs(status): update STATUS.md with phase0 infra hardening progress
Some checks failed
CI / lint (pull_request) Failing after 49s
Details
CI / typecheck (pull_request) Successful in 54s
Details
CI / Secret scan (gitleaks) (pull_request) Successful in 32s
Details
CI / Security audit (bandit) (pull_request) Successful in 34s
Details
CI / test (pull_request) Successful in 1m19s
Details
b6e5f6a51c
cryptorugmunch closed this pull request 2026-07-06 10:07:02 +02:00
Some checks are pending
CI / lint (pull_request) Failing after 49s
Details
CI / typecheck (pull_request) Successful in 54s
Details
CI / Secret scan (gitleaks) (pull_request) Successful in 32s
Details
CI / Security audit (bandit) (pull_request) Successful in 34s
Details
CI / test (pull_request) Successful in 1m19s
Details
lint
Required
test
Required
typecheck
Required

Pull request closed

This pull request cannot be reopened because the branch was deleted.
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: RugMunchMedia/pryscraper#8
No description provided.
Delete branch "feat/phase0-rebased"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?