# Letting an application send RAG requests without ever holding a provider key

> A self-hosted Go gateway between your application and the providers: one OpenAI-compatible API, with RAG context injected into every request.

- What it does today: Lets an application talk to one OpenAI-compatible endpoint while a project's configuration decides which model the request reaches and, when a document store is linked, which documents it draws context from. Anyone who does not want provider keys spread across applications, and wants spend capped per project, can run it on their own server.
- Form: Open source, Web app
- State: Became a product
- Focus: AI-assisted development
- Started: 2026-08-19
- Left Labs: 2026-09-18
- Technologies: Go, PostgreSQL, pgvector, Docker, OpenAI, Anthropic Claude, Google Gemini, DeepSeek, Ollama, Prometheus, OpenTelemetry
- Tags: #rag, #gateway, #llm
- Source code: https://github.com/ragmux/ragmux
- Website: https://ragmux.com
- This one became a product: Ragmux (https://www.muhammetsafak.com.tr/en/portfolio/ragmux/)
- Source: https://www.muhammetsafak.com.tr/en/labs/ragmux-lab/
- Language: en-US
- Author: Muhammet Şafak

---
In an LLM feature that runs on your own documents, deciding to use RAG is the
easy half. The hard half is what carries that retrieval in production.

## What is left after the decision

The documents are embedded, the chunks are in the database, search works. Three
questions remain: where the provider key sits, which application uses which
model, and where spend gets cut off. None of the three is part of retrieval, and
solving all three inside every application means copying the same code once per
application.

Ragmux moves them out of the application and into the gateway between it and the
providers. The application talks to one OpenAI-compatible endpoint; which
provider the request reaches, which system prompt it carries and which document
store it draws context from is decided by the project's connection.

## The model name the client sends selects nothing

The `model` field does no routing. Which model the request reaches is decided by
the project's connection; the client's value is echoed back unchanged in the
response and in every stream chunk, so SDKs that compare that field keep working.

What that buys: the provider credential stays on the server, encrypted, and is
never read back through any interface. All the application holds is the
project's own key. The cost is plain too — the client can no longer choose a
model; model choice stops being a deployment decision and becomes a
configuration one.

## Budgets are capped in tokens, not in money

Every request's estimated cost is calculated and shown on the dashboard, but
**cost is informational**: no request is ever refused because a monetary
threshold was reached. The way to cap spend is to cap tokens — requests and
tokens per minute per project, daily and monthly token budgets.

There is a limit here that is not hidden: token counts are only final once the
provider has answered, so a single request can overshoot a budget by its own
size, and it is the next one that gets refused.

## One PostgreSQL, no second store

Users, connections, projects, documents, chunks, vectors, request logs and usage
counters all live in one PostgreSQL with pgvector. No Redis, no separate vector
database. Because a counter is a row rather than a process variable, several
replicas pointed at the same database share one budget.

## The review that found the product breaking its own rule

Three of the five recorded decisions came out of a pre-release review, two of
them from places where code and documentation disagreed: the role matrix
described `viewer` as read-only while the code let a viewer mint their own
gateway key; a price-table row reported a connection pointed at a paid endpoint
as "$0.00".

The fifth decision was softened within a day of being taken. A hard per-tenant
share had been put on the image-download ceiling; measurement showed that on a
single-project install — the common case — it left half the ceiling permanently
idle. The share was loosened the same day: when nobody is waiting, one tenant
may use the whole ceiling.

The product's own entry is in the portfolio.