Lexora
Asynchronous document processing and retrieval backend.
- Python
- FastAPI
- PostgreSQL
- Redis
- Celery
- FAISS
- Docker
- 35 tests
- CI
- source
Problem
Small teams need grounded Q&A over their own PDFs, text, Markdown, and Word files, without building retrieval, auth, and background processing from scratch. "RAG" is one component of that system; the backend around it is most of the work.
Constraints
- Uploads must be validated (type, size) before any processing starts.
- Document processing must not block chat: same pipeline, inline or background.
- One user's vectors must never leak into another user's answers.
Architecture
Client ↓ FastAPI (/api/v1: auth, documents, chat) ↓ PostgreSQL (users, documents, conversations, messages) ↓ Redis / Background Worker (retrieval cache, token blacklist, document jobs) ↓ Document Processing (validate → extract → chunk → embed → per-user FAISS) ↓ Retrieval (query embed → top-k → rank → context) → LLM (chat / stream)
Implementation
JWT register/login/refresh/logout with a Redis-backed token blacklist for logout and rotation. Uploads go through type/size validation, then extraction (pypdf, python-docx), chunking, embedding, and per-user FAISS indexing, all through the same process_document path in both inline and Celery modes. Chat answers through retrieval + LLM with SSE streaming and conversation history. Retrieval results cache for 1h under user-scoped keys. Non-streaming LLM calls retry 3× with exponential backoff; API routes carry request IDs and enforced per-IP rate limits (429 + Retry-After). Prometheus /metrics, structured JSON logs, /health + /ready (DB + Redis checks), Docker Compose for the full stack.
Engineering decisions
- PostgreSQL for relational history with async SQLAlchemy pooling; SQLite only for tests.
- Redis for two jobs: retrieval cache and JWT revocation, not a general dumping ground.
- Per-user FAISS isolation for simplicity, rebuilt on delete. Documented trade-off; pgvector is the planned replacement.
- Dead dependencies removed (Sentry SDK never wired, pymupdf never imported, Alembic with no migrations) rather than carried for keywords.
Challenges
The README's own limitation list was the work queue: rate limiting configured but not enforced, no request IDs, LLM calls without retries. Each was closed with a mechanism plus a test. The reliability test file covers the limiter windows, the 429 contract, and a transient-failure-then-recovery LLM sequence. What remains open (streaming retries, idempotent uploads, FAISS→pgvector) stays listed as limitations, not silent gaps.
Benchmarks
Measured: 35 unit tests passing, ~49% coverage, and a deterministic retrieval recall probe (scripts/eval_retrieval.py) scoring 3/3 at top-3, a keyword-recall floor over chunking that needs no API keys. Latency and throughput numbers (upload to processable, retrieval, chat, first token) require PG + Redis + an OpenAI key and are not yet measured, so they are not cited.
Trade-offs
- FAISS per-user indexes are simple and isolated but rebuild on delete and don't scale like pgvector.
- LangChain wrappers remain (direct OpenAI calls planned); abstraction stays only where it already works.
- Rate limiting is single-replica in-memory; Redis sliding windows come with multi-replica deployment.
Lessons
AI is a fraction of an AI product. Auth, validation, background processing, caching, revocation, rate limits, metrics, and tests are the system; retrieval quality is one input to it. An eval probe you can run offline beats a demo you can only run on stage.