Ibán López
Back to logs

Open Sourcing the Zero-Cost RAG Architecture

2026-08-01

Open Sourcing the Zero-Cost RAG Architecture

When building the AI assistant for my portfolio, I realized a fundamental flaw in modern AI architecture: we are paying twice for vector embeddings. You pay when you ingest the data, and you pay again for every single query your users make.

Over the long term, scaling cloud embeddings for high-traffic applications becomes a significant operational expense for a feature that essentially just does semantic routing.

To solve this, I architected a Hybrid RAG Pipeline that brings the embedding layer entirely locally, reducing embedding costs to exactly $0.00.

The Architecture

The pipeline consists of three core components:

  1. Local Vectorization via Transformers.js: Instead of calling OpenAI's text-embedding-ada-002, the server runs Xenova's all-MiniLM-L6-v2 directly in the Node.js runtime. This model is small enough to be loaded instantly but powerful enough to generate high-quality 384-dimensional embeddings.
  2. Supabase pgvector: The generated embeddings are stored in a standard PostgreSQL database using the pgvector extension, allowing for incredibly fast cosine similarity searches.
  3. DeepSeek Generation: Only the final step—synthesizing the retrieved context into a natural language response—is sent to a cloud LLM. We utilize DeepSeek for this due to its exceptional cost-to-performance ratio.

To ensure the endpoint isn't abused, the architecture is wrapped in an Upstash Redis sliding window rate limiter, completely neutralizing token-drain attacks.

The Open Source Template

Because this pattern is so effective, I've extracted the core intelligence, cleaned up the dependencies, and open-sourced the entire architecture as a Next.js template.

It comes pre-configured with a premium Stepper UI for securely capturing environment variables, the SQL migrations for Supabase, and a gorgeous RAG chat interface.

You can view, fork, and use the template directly on my GitHub:

View the Zero-Cost RAG Template Repository

By utilizing this architecture, independent developers and startups can deploy highly intelligent context-aware applications without the fear of runaway embedding costs.