From bbb9a9291d7a6e578c250972cd5a8e3ac6a5e0c6 Mon Sep 17 00:00:00 2001 From: cryptorugmunch Date: Fri, 3 Jul 2026 15:56:26 +0200 Subject: [PATCH] docs: rewrite README and add PRODUCT, ARCHITECTURE, ROADMAP, TODO per governance framework --- ARCHITECTURE.md | 61 +++++++++++++++++++++++ PRODUCT.md | 54 ++++++++++++++++++++ README.md | 130 ++++++++++++++++-------------------------------- ROADMAP.md | 33 ++++++++++++ TODO.md | 21 ++++++++ 5 files changed, 212 insertions(+), 87 deletions(-) create mode 100644 ARCHITECTURE.md create mode 100644 PRODUCT.md create mode 100644 ROADMAP.md create mode 100644 TODO.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..be1609b --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,61 @@ +# RMI Backend — Architecture + +**Status:** canonical +**Last updated:** 2026-07-03 + +## Overview +FastAPI monolith serving crypto risk intelligence via HTTP, Telegram, and MCP. + +## High-level flow +``` +Client (Web / Telegram / AI Agent) + ↓ +nginx → rmi-backend container (:8000) + ↓ +FastAPI routers + ↓ +Services / scanners / DataBus providers + ↓ +Postgres / Redis / Qdrant / Neo4j / ClickHouse +``` + +## Components +| Path | Purpose | +|---|---| +| `app/routers/` | HTTP route handlers (thin) | +| `app/scanners/` | Token risk scanners | +| `app/databus/` | 30+ third-party data providers | +| `app/telegram_bot/` | Telegram bot command handlers | +| `app/mcp/` | MCP manifest and tool manager | +| `app/facilitators/` | x402 payment verification | +| `app/wallet_memory/` | Wallet label + clustering subsystem | + +## Storage +| Store | Use | Container / Host | +|---|---|---| +| Postgres | Primary OLTP, Ponder indexer data | `rmi-postgres` | +| Redis | Cache, Celery queue | `rmi-redis` | +| Qdrant | Vector embeddings | `rmi-qdrant` (down) | +| Neo4j | Wallet graph / labels | `rmi-neo4j` | +| ClickHouse | Analytics, 39M address labels | `rmi-clickhouse` | + +## Interfaces +| Endpoint | Purpose | +|---|---| +| `GET /health` | Health check (currently unhealthy due to Redis) | +| `GET /docs` | Swagger/OpenAPI docs | +| `POST /scanner/scan` | Live token scan | +| `POST /mcp/...` | MCP tool endpoints | + +## Current issues +- `x402_tools.py` (212 KB) and `x402_enforcement.py` (108 KB) violate the 500-line limit and should become separate services or split by domain. +- `sqlite3` is used in 5 files for local state; should migrate to Postgres/aiosqlite. +- F-string logs and `print()` statements prevent structured log aggregation. + +## Deployment +```bash +ssh netcup +cd /root/backend/ +docker compose pull +docker compose up -d rmi-backend +``` diff --git a/PRODUCT.md b/PRODUCT.md new file mode 100644 index 0000000..c9fa24d --- /dev/null +++ b/PRODUCT.md @@ -0,0 +1,54 @@ +# RMI Backend — Product Charter + +**Product:** Rug Munch Intelligence (RMI) Backend +**Repo:** `git.rugmunch.io/RugMunchMedia/rmi-backend` +**Deploy path:** `/root/backend/` on Talos +**Container:** `rmi-backend` +**Health:** `http://127.0.0.1:8000/health` +**Status:** Production / degraded + +## Problem +Crypto users and platforms need a single, fast, machine-readable source of token risk, wallet labels, and on-chain intelligence. Existing tools are fragmented, expensive, or closed. + +## User +- Telegram users via `@rugmunchbot` +- RMI web frontend at `https://rugmunch.io` +- AI agents via MCP +- API consumers via x402 pay-per-call + +## Scope +- FastAPI backend with token scanner, wallet forensics, market data, alerts +- DataBus: unified data plane across 30+ providers +- Telegram bot commands: `/scan`, `/portfolio`, `/simulate`, `/chart`, `/watch` +- MCP server for agent tool calling +- x402 payment gating for premium endpoints + +## Anti-scope +- No frontend code (that lives in `rmi-frontend`) +- No wallet generation (that lives in `walletpress`) +- No web scraping engine (that lives in `pryscraper`) +- No social protocol clients (that lives in `degenfeed-web`) + +## Success criteria +- `/health` returns 200 +- CI gates are authoritative +- Test coverage >60% +- All MCP tools have Pydantic input/output schemas +- No production file >500 lines + +## Current reality +- 990 routes across modular routers +- Health is **unhealthy** because `rmi-redis:6379` does not resolve +- 8.61% test coverage +- 17 `time.sleep()`, 1,713 f-string logs, 399 `print()` calls +- Largest file `app/routers/x402_tools.py` is 212 KB + +## Fleet dependencies +- Postgres `rmi-postgres:5432` +- Redis `rmi-redis:6379` (currently failing) +- Qdrant `127.0.0.1:6333` (currently down due to stale IP bind) +- Neo4j `127.0.0.1:7687` +- ClickHouse `127.0.0.1:8123` + +## Standards +This repo follows [Rug Munch Media standards](https://git.rugmunch.io/RugMunchMedia/standards). diff --git a/README.md b/README.md index 309ed8f..a3e6963 100644 --- a/README.md +++ b/README.md @@ -1,97 +1,53 @@ -# RMI — Rug Munch Intelligence +# RMI Backend -> **The first open-source crypto intelligence platform.** -> Bloomberg-grade security and market intelligence for the on-chain world. +**Repo:** `git.rugmunch.io/RugMunchMedia/rmi-backend` +**Deploy:** `/root/backend/` on Talos +**Container:** `rmi-backend` +**Health:** `http://127.0.0.1:8000/health` (currently unhealthy — Redis host not resolving) ---- +## What it is +FastAPI backend for the RMI platform. Serves token risk scans, wallet forensics, market data, Telegram bot commands, MCP agent tools, and x402-gated premium endpoints. -## About Rug Munch Media LLC +## Status +Production, but degraded. The container runs and responds, but `/health` reports unhealthy because the Redis hostname `rmi-redis:6379` is not resolving inside the container network. -We are **Rug Munch Media LLC**, a Wyoming-based software company building open-source tools that make the crypto economy safer — for consumers, builders, and institutions. - -Crypto moves fast, and bad actors move faster. Phishing sites clone trusted brands within hours of a certificate being issued. Token launches rug in seconds. Wallets drain before a user knows what happened. The existing security stack is fragmented, expensive, and gated behind paywalls. **We are building the open alternative.** - -RMI (Rug Munch Intelligence) is the **first open-source crypto intelligence platform** — a unified, AI-powered command center for token risk analysis, wallet forensics, multi-chain market data, phishing detection, and regulatory-grade intelligence. Our ambition is to become the **Bloomberg terminal of crypto**: one trusted surface where the entire industry can see what is happening, what is risky, and what is real. - -**Who we serve:** -- **Consumers** — real-time protection against rug pulls, phishing, and wallet drainers -- **Industry** — exchanges, custodians, compliance teams, and researchers who need defensible on-chain intelligence -- **AI agents** — first-class Model Context Protocol (MCP) integration and x402 micropayments for autonomous tooling - -We are an open-source company. Our code, our data, and our threat intelligence are public. Our business model is delivering the managed product, the API, and the institutional tier — not locking away the safety net. - ---- - -## What RMI does - -### Threat detection -- **Token scanning** — real-time rug-pull, honeypot, and scam detection across 18+ EVM chains + Solana -- **Wallet forensics** — track wallet behavior and transaction patterns, flag known bad actors -- **Phishing detection** — Certificate Transparency monitoring catches phishing domains within hours of registration, before they go live -- **Bayesian reputation** — statistically grounded deployer trust scoring, not vibes - -### Market intelligence -- **2,500+ assets** — full coverage across major chains -- **News aggregation** — 1,800+ sources with clustering, dedup, and sentiment -- **Whale alerts** — large wallet movements and trading patterns in real time -- **Multi-chain market data** — OHLCV, liquidity, and volume from a single API - -### AI-native platform -- **RAG-grounded reports** — every claim cited, no hallucination -- **MCP server** — drop RMI into any AI agent or LLM workflow -- **x402 micropayments** — pay-per-call pricing for autonomous agents -- **Open weights and open data** — no vendor lock-in - ---- - -## Architecture - -RMI is a single FastAPI backend with modular domain services: - -``` -Backend (FastAPI + Python) -├── DataBus — single data plane, 18 chains, 4-layer defense (labels → scanner → RAG → prices) -├── SENTINEL — multi-chain token security scanner -├── Wallet Bank — 39M+ labeled wallets, federated from open sources -├── RAG Engine — 3-pillar hybrid retrieval (dense + sparse + entity) -├── x402 Gateway — AI-agent micropayments -├── MCP Server — 221 tools for AI agents -└── Observability — Prometheus, GlitchTip, OpenTelemetry +## Quick start (local) +```bash +python -m venv .venv +source .venv/bin/activate +pip install -r requirements.txt +python -m pytest tests/unit/ -x -q ``` -Storage: PostgreSQL, Redis, Neo4j, Qdrant, ClickHouse, DuckDB. +## Deploy +```bash +ssh netcup +cd /root/backend/ +docker compose pull +docker compose up -d rmi-backend +``` ---- +## Fleet dependencies +| Service | Address | Status | +|---|---|---| +| Postgres | `rmi-postgres:5432` | Up | +| Redis | `rmi-redis:6379` | Failing to resolve | +| Qdrant | `127.0.0.1:6333` | Down (stale IP bind) | +| Neo4j | `127.0.0.1:7687` | Up | +| ClickHouse | `127.0.0.1:8123` | Up | -## Repositories +## Current issues +- 8.61% test coverage +- 17 `time.sleep()`, 1,713 f-string logs, 399 `print()` calls +- Largest file `app/routers/x402_tools.py` is 212 KB +- CI gates use `continue-on-error: true` and `|| true` +- 5 Telegram tokens leaked in git history -| Platform | URL | -|----------|-----| -| 🐙 GitHub (canonical) | https://github.com/Rug-Munch-Media-LLC/rugmuncher-backend | -| 🦊 GitLab (mirror) | https://gitlab.com/cryptorugmuncher/rugmuncher-backend | -| 🤗 HuggingFace (canonical) | https://huggingface.co/cryptorugmuncher/rugmuncher-backend | +## Docs +- `PRODUCT.md` — charter and scope +- `ARCHITECTURE.md` — components and data flow +- `ROADMAP.md` — next 90 days +- `TODO.md` — current tasks ---- - -## Links - -- 🌐 **Website**: https://rugmunch.io -- 📧 **Contact**: info@rugmunch.io -- 💬 **Telegram**: https://t.me/cryptorugmuncher -- 🐦 **X / Twitter**: https://x.com/cryptorugmunch -- 🐙 **GitHub org**: https://github.com/Rug-Munch-Media-LLC -- 🦊 **GitLab**: https://gitlab.com/cryptorugmuncher -- 🤗 **HuggingFace org**: https://huggingface.co/cryptorugmuncher - ---- - -## License - -**MIT** — see `LICENSE.md`. - -RMI is open source because the safety of the crypto economy cannot depend on a single vendor. We compete on the quality of the product, not on who is allowed to see the threat. - ---- - -**Built by Rug Munch Media LLC · Wyoming, USA** -*Open-source crypto intelligence. AI-native. Built to last.* +## Standards +This repo follows [Rug Munch Media standards](https://git.rugmunch.io/RugMunchMedia/standards). diff --git a/ROADMAP.md b/ROADMAP.md new file mode 100644 index 0000000..6fbc4bd --- /dev/null +++ b/ROADMAP.md @@ -0,0 +1,33 @@ +# RMI Backend — Roadmap + +**Next 90 days only.** See `TODO.md` for current session tasks. + +## Week 1 — Stabilize +- Fix Redis host resolution for `rmi-backend` health +- Fix Qdrant stale Tailscale IP bind +- Rotate leaked secrets (Telegram tokens, API keys) +- Harden public service binds + +## Week 2 — Gates +- Make CI authoritative (remove `|| true` and `continue-on-error`) +- Add `web3-safety` and `hallucination-check` pre-commit hooks +- Get `ruff check` and `ruff format --check` green + +## Week 3-4 — Standards Autofix +- Replace all `time.sleep()` with `asyncio.sleep()` +- Convert f-string logs to structured logs +- Remove/replace `print()` statements +- Migrate `sqlite3` to async Postgres +- Add `# noqa: ANY` or replace `Any` annotations + +## Week 5-6 — Modularize +- Split `x402_tools.py` into domain modules +- Split `x402_enforcement.py` +- Split `databus/providers.py` +- Begin extracting x402/MCP into standalone product boundaries + +## Acceptance +- `/health` green +- CI fails on lint/secrets +- Coverage >60% +- No file >500 lines diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..f2aad78 --- /dev/null +++ b/TODO.md @@ -0,0 +1,21 @@ +# RMI Backend — TODO + +## Active +- [ ] Fix `rmi-redis:6379` name resolution so `/health` returns healthy +- [ ] Fix Qdrant stale IP bind (`100.100.18.18:6333` → `127.0.0.1:6333`) +- [ ] Rotate 5 Telegram tokens leaked in `data/cryptoscamdb_urls.yaml` history +- [ ] Rotate Solana Tracker, Arkham WS, Stripe, Helius webhook secrets +- [ ] Remove `continue-on-error: true` and `|| true` from `.github/workflows/ci.yml` + +## Next +- [ ] Replace 17 `time.sleep()` calls with `asyncio.sleep()` +- [ ] Convert top 10 f-string log files to structured logging +- [ ] Delete or migrate 5 `sqlite3` usages +- [ ] Split `app/routers/x402_tools.py` (>4,000 lines) into modules + +## Blocked +- GitHub external mirror pushes until org account flag is lifted + +## Done +- HuggingFace backups consolidated to single bucket +- External mirrors active on GitLab + Codeberg