docs(state): update STATUS.md and ARCHITECTURE.md for Phase 4 progress
This commit is contained in:
parent
4686cb3cfd
commit
757eefd227
2 changed files with 61 additions and 22 deletions
|
|
@ -1,10 +1,11 @@
|
|||
# RMI Backend — Architecture
|
||||
|
||||
**Status:** canonical
|
||||
**Last updated:** 2026-07-03
|
||||
**Last updated:** 2026-07-07
|
||||
|
||||
## Overview
|
||||
FastAPI monolith serving crypto risk intelligence via HTTP, Telegram, and MCP.
|
||||
Domains are consolidated under `app/domains/` per AUDIT-2026-Q3 Phase 4.
|
||||
|
||||
## High-level flow
|
||||
```
|
||||
|
|
@ -12,9 +13,9 @@ Client (Web / Telegram / AI Agent)
|
|||
↓
|
||||
nginx → rmi-backend container (:8000)
|
||||
↓
|
||||
FastAPI routers
|
||||
FastAPI routers (mounted by app/mount.py)
|
||||
↓
|
||||
Services / scanners / DataBus providers
|
||||
app/domains/<domain>/service.py → app/infra/ + app/domains/<domain>/repository.py
|
||||
↓
|
||||
Postgres / Redis / Qdrant / Neo4j / ClickHouse
|
||||
```
|
||||
|
|
@ -22,35 +23,42 @@ Postgres / Redis / Qdrant / Neo4j / ClickHouse
|
|||
## Components
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `app/routers/` | HTTP route handlers (thin) |
|
||||
| `app/scanners/` | Token risk scanners |
|
||||
| `app/databus/` | 30+ third-party data providers |
|
||||
| `app/telegram_bot/` | Telegram bot command handlers |
|
||||
| `app/factory.py` | FastAPI app factory (lifespan, middleware, error handlers, routers) |
|
||||
| `app/mount.py` | Single source of truth for router mounting |
|
||||
| `app/core/` | Cross-cutting concerns (logging, errors, redis, http, auth, config) |
|
||||
| `app/api/v1/` | Thin HTTP transport layer (public, auth, admin, x402, mcp) |
|
||||
| `app/domains/` | Business logic domains: auth, billing, databus, news, reports, scanner, scanners, telegram, threat, token, tokens, wallet, x402 |
|
||||
| `app/domains/scanners/` | Token risk scanners (IP — will move to rmi-ip in Phase 6) |
|
||||
| `app/domains/databus/` | 30+ third-party data provider integrations |
|
||||
| `app/domains/telegram/rugmunchbot/` | Telegram bot command handlers |
|
||||
| `app/mcp/` | MCP manifest and tool manager |
|
||||
| `app/facilitators/` | x402 payment verification |
|
||||
| `app/wallet_memory/` | Wallet label + clustering subsystem |
|
||||
| `app/infra/` | External integrations (ollama, langfuse, vector stores, chain clients) |
|
||||
| `app/wallet_memory/` | Wallet label + clustering subsystem (IP — will move to rmi-ip in Phase 6) |
|
||||
| `app/routers/` | Legacy HTTP route handlers (being migrated to app/domains/ or app/api/v1/) |
|
||||
|
||||
## Storage
|
||||
| Store | Use | Container / Host |
|
||||
|---|---|---|
|
||||
| Postgres | Primary OLTP, Ponder indexer data | `rmi-postgres` |
|
||||
| Redis | Cache, Celery queue | `rmi-redis` |
|
||||
| Qdrant | Vector embeddings | `rmi-qdrant` (down) |
|
||||
| Qdrant | Vector embeddings | `rmi-qdrant` |
|
||||
| Neo4j | Wallet graph / labels | `rmi-neo4j` |
|
||||
| ClickHouse | Analytics, 39M address labels | `rmi-clickhouse` |
|
||||
|
||||
## Interfaces
|
||||
| Endpoint | Purpose |
|
||||
|---|---|
|
||||
| `GET /health` | Health check (currently unhealthy due to Redis) |
|
||||
| `GET /health` | Health check (degraded — ETH RPC endpoint missing) |
|
||||
| `GET /metrics` | Prometheus metrics |
|
||||
| `GET /docs` | Swagger/OpenAPI docs |
|
||||
| `POST /scanner/scan` | Live token scan |
|
||||
| `POST /mcp/...` | MCP tool endpoints |
|
||||
|
||||
## Current issues
|
||||
- `x402_tools.py` (212 KB) and `x402_enforcement.py` (108 KB) violate the 500-line limit and should become separate services or split by domain.
|
||||
- `app/routers/x402_tools.py` (212 KB) and `app/routers/x402_enforcement.py` (108 KB) violate the 500-line limit and should become separate services or split by domain.
|
||||
- `sqlite3` is used in 5 files for local state; should migrate to Postgres/aiosqlite.
|
||||
- F-string logs and `print()` statements prevent structured log aggregation.
|
||||
- `mypy` has a structural issue with `app/domains/auth/` that needs resolution.
|
||||
|
||||
## Deployment
|
||||
```bash
|
||||
|
|
|
|||
51
STATUS.md
51
STATUS.md
|
|
@ -1,7 +1,7 @@
|
|||
# STATUS -- rmi-backend
|
||||
|
||||
**Owner:** Rug Munch Media LLC Engineering
|
||||
**Last updated:** 2026-07-06 -- Phase 1 of [AUDIT-2026-Q3.md](AUDIT-2026-Q3.md) complete.
|
||||
**Last updated:** 2026-07-07 -- Phase 4 domain consolidation in progress; P4.1-P4.8 + P5.1 complete.
|
||||
|
||||
## What This Repo Is
|
||||
|
||||
|
|
@ -13,7 +13,7 @@ Stack: Python 3.12 / FastAPI / SQLAlchemy / async / Playwright / Pydantic v2 / R
|
|||
|
||||
## Current Status
|
||||
|
||||
**Phase 1 (development hygiene) complete.** Phase 2 (delete dead code) starts next per [AUDIT-2026-Q3.md](AUDIT-2026-Q3.md).
|
||||
**Phase 4 (domain consolidation) is active.** Foundation (Phase 1) is solid, dead-router wiring (Phase 2) is partial, god-file splitting (Phase 3) is deferred, and standardization (Phase 5) has begun with CI trustworthiness and mypy config cleanup.
|
||||
|
||||
### Phase 1 -- DONE
|
||||
|
||||
|
|
@ -30,16 +30,44 @@ Committed to `main`:
|
|||
- `92a01ff` + `5294983` -- `AppError` hierarchy in `app/core/errors.py`, wired through `error_handlers.py`
|
||||
- `cd02714` + `2fb571e` -- author amend (canonical identity: `cryptorugmunch <admin@rugmunch.io>`)
|
||||
|
||||
### What Phase 1 DID NOT do (deferred)
|
||||
### Phase 2 -- IN PROGRESS (partial)
|
||||
|
||||
Rather than deleting 148 routers, we validated and mounted the useful ones in `app/mount.py` Wave 1-3 (2026-07-07). Dead routers remain in tree and will be archived in a later pass.
|
||||
|
||||
### Phase 3 -- NOT STARTED
|
||||
|
||||
God-files (`x402_tools.py`, `x402_enforcement.py`, `auth.py`, `databus/providers.py`, `telegram_bot/bot.py`, `wallet_manager_v2.py`) still await splitting.
|
||||
|
||||
### Phase 4 -- IN PROGRESS
|
||||
|
||||
Domain consolidation (AUDIT-2026-Q3 P4.x) — committed to `main`:
|
||||
- `dca458e` -- P4.1 billing (`app/billing/` + `app/facilitators/` → `app/domains/billing/`)
|
||||
- `7109a16` -- P4.2 wallet (`app/wallet/` → `app/domains/wallet/`)
|
||||
- `948e41c` -- P4.3 auth (`app/auth/` → `app/domains/auth/`)
|
||||
- `cab9740` -- P4.4 tokens (`app/tokens/` → `app/domains/tokens/`)
|
||||
- `ed5b830` -- P4.5 databus (`providers/` + `_generated/` → `app/domains/databus/`)
|
||||
- `56075cf` -- P4.6 telegram (`rugmunchbot/` → `app/domains/telegram/rugmunchbot/`)
|
||||
- `3b7ef42` -- P4.7 rename `app/domain/` → `app/domains/` + consolidate
|
||||
- `7cced4e` + `4686cb3` -- P4.8 scanners (`app/scanners/` → `app/domains/scanners/`)
|
||||
|
||||
Remaining Phase 4:
|
||||
- Finish databus root migration (`app/databus/` engine → `app/domains/databus/`)
|
||||
- Finish telegram migration (`app/telegram_bot/` → `app/domains/telegram/`)
|
||||
- Clean up deprecated shim dirs (`app/auth/`, `app/wallet/`, `app/billing/`, `app/facilitators/`, `app/scanners/`, `app/domain/`)
|
||||
- Consolidate bulletin, intelligence, markets, admin, referral, MCP domains
|
||||
|
||||
### Phase 5 -- STARTED
|
||||
|
||||
- `d666ad2` -- P5.1 `HEALTH_CHECK_DURATION` + `test_factory_has_minimum_routes`
|
||||
- `4686cb3` -- mypy config fixes (`mypy.ini` regex, `disallow_any_explicit` typo, `app.domains.*` module pattern)
|
||||
- Still open: alembic, real coverage gate, structured logging, Sentry wiring, Prometheus middleware
|
||||
|
||||
### What Phase 1-4 DID NOT do (deferred)
|
||||
|
||||
- ~2K ruff warnings in legacy code (Phase 5)
|
||||
- Real coverage 8.17% -> 80% gate (Phase 5, slowest part)
|
||||
- `mypy.ini` line 8 parse error + `disallow_any_express_imports` typo (Phase 5)
|
||||
- Real coverage 8.17% -> 80% gate (Phase 5)
|
||||
- Alembic migrations (still MISSING; Phase 5)
|
||||
- 285 dead modules (Phase 2)
|
||||
- 148 unmounted routers (Phase 2)
|
||||
- God-files split (Phase 3)
|
||||
- Domain consolidation (Phase 4)
|
||||
|
||||
### Open Issues
|
||||
|
||||
|
|
@ -68,8 +96,11 @@ e404e90 feat(rmi-backend,audit): add app/main.py entrypoint delegating to factor
|
|||
- `gitleaks detect` reports 28 findings -- all **public ERC-20/Solana token contract addresses** (USDC, WETH, MATIC, BONK, DAI, plus cryptoscam-address dataset). NOT secrets. Suppressed via `.gitleaksignore` + `.gitleaks.toml` `[allowlist]`. No rotation needed.
|
||||
- All secrets live in gopass under `rmi/contacts/admin_recovery` etc.
|
||||
|
||||
## Next: Phase 2 (week 2)
|
||||
## Next: Phase 4 completion
|
||||
|
||||
Archive **148 unmounted routers** + **~140 dead `app/*.py` files** to `app/_archive/legacy_2026_07/`. Goal: ~30% LOC reduction without changing any current behavior.
|
||||
1. Finish `app/databus/` engine migration to `app/domains/databus/`.
|
||||
2. Finish `app/telegram_bot/` migration to `app/domains/telegram/rugmunchbot/`.
|
||||
3. Remove deprecated shim directories (`app/auth/`, `app/wallet/`, `app/billing/`, `app/facilitators/`, `app/scanners/`, `app/domain/`).
|
||||
4. Consolidate bulletin, intelligence, markets, admin, referral, MCP into `app/domains/`.
|
||||
|
||||
See [AUDIT-2026-Q3.md](AUDIT-2026-Q3.md) for the full 6-phase plan.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue