Skip to content

Writing

Building a Low-Cost Go RAG Chatbot with pgvector, Semantic Caching, and Payload CMS

How showmic-bot combines Go stdlib net/http, pgx, Neon pgvector, multi-tier semantic caching, and Payload CMS syncs to stream grounded answers for pennies.

By Sifat RabbiReviewed November 22, 20263 min readDatabases & Cloud Architecture

Most portfolio or documentation chatbots fail in one of two ways: either they burn $50/month answering off-topic troll prompts with frontier LLMs, or they hallucinate skills and metrics that the author never shipped. When I built showmic-bot (apps/bot) for showmic96.dev, I designed it as a small, deterministic Go + PostgreSQL `pgvector` service using only Go's standard net/http router and pgx.

1. End-to-End Request Pipeline: Cut 70% of LLM Calls Before Retrieval

Every incoming POST /api/chat request passes through a layered funnel ordered strictly from cheapest (0ms, $0) to most expensive (streaming LLM generation):

  1. Normalization (`0ms`): Lowercases text, strips punctuation, and canonicalizes aliases (showmic -> sifat) so variations share one cache key.
  2. Exact SQL Cache Lookup (`~2ms`): Checks common_questions.normalized and refused_questions.normalized before calling any embedding API.
  3. Deterministic Rule Gate (`0ms`): Regex/pattern rules handle greetings and immediately refuse arithmetic, weather, sports scores, or generic code-debugging prompts with zero API spend.
  4. Single Embedding + Dual Semantic Cache (`~60ms`): Computes one text-embedding-3-small vector, checks refused_questions (cosine >= 0.93) for known off-topic misses, then checks grounded FAQs (cosine >= 0.90).
  5. Top-5 `pgvector` Retrieval + Relevance Gate: Queries synced Payload CMS chunks (1 - (embedding <=> query)). Off-topic queries below 0.38 similarity receive a deterministic contact-email fallback instead of guessing.
  6. Multi-Provider SSE Streaming + Hygiene Store: Streams tokens via Server-Sent Events (data: {"token":"..."}) and stores high-confidence (top_sim >= 0.45) answers for future cache hits.

2. Why Go (net/http + pgx) Beat Python/FastAPI for This Workload

MetricGo (net/http + pgx)Typical Python + LangChain Container
Container RAM Idle~18–25 MB180–350 MB
Cold Boot Time< 80 ms1.5–4.0 seconds
SSE Stream ConcurrencyGoroutines (kilobytes per stream)Asyncio + worker tuning
Dependency ChurnStdlib + pgx onlyFrequent breaking SDK updates

3. Multi-Provider Failover Without Duplicate Stream Tokens

Single-vendor LLM APIs suffer intermittent rate limits and regional brownouts. In llm/providers.go, our streamer walks an ordered provider chain: Google Gemini API direct (`gemini-3.5-flash-lite`) -> OpenRouter (`google/gemini-3.5-flash-lite`) -> OpenRouter (`deepseek/deepseek-v4.1-flash`). Crucially, failover only triggers if a provider errors or closes empty before emitting its first token, guaranteeing the browser never receives half of one answer stitched onto the start of another.

Top-5 grounded chunk query with pgvector:
-- pgvector cosine similarity retrieval in apps/bot
SELECT id, doc_type, title, content,
1 - (embedding <=> $1::vector) AS similarity
FROM chunks
ORDER BY embedding <=> $1::vector
LIMIT 5;

4. Syncing Payload CMS Content Daily and on Boot

Rather than scraping HTML pages, the bot's sync/ package pulls structured JSON directly from Payload CMS (projects, contents, experiences, skills), strips Lexical AST nodes into clean plain-text chunks with --- type: title --- headers, hashes document content so unchanged records skip re-embedding, and seeds 100 grounded FAQ pairs automatically.

Frequently Asked Questions

How much does it cost per month to run this Go + pgvector RAG chatbot?

Because rule gates, exact SQL lookups, negative semantic caching (>= 0.93), and FAQ semantic caching (>= 0.90) intercept repetitive and off-topic traffic, monthly LLM + text-embedding-3-small spend on portfolio traffic stays under a few dollars alongside a lightweight ~25MB Go process on Railway and shared Neon Postgres.

How do you prevent a portfolio RAG chatbot from hallucinating fake experience?

Three layers work together: a relevance floor (0.38 cosine similarity) that blocks ungrounded off-topic queries before calling the LLM, a strict system prompt treating context chunks as the sole source of truth with a fixed email fallback, and post-generation hygiene filters that refuse to cache low-evidence answers.

Why use Go without LangChain or LlamaIndex for RAG?

A RAG pipeline is fundamentally string normalization, an HTTP POST to an embedding endpoint, a 6-line pgvector SQL query, and an SSE HTTP stream. Writing those 400 lines directly in Go with pgx gives complete visibility into thresholds, retries, and latency without framework abstractions.

Production AI & RAG

Putting a grounded RAG assistant or LLM pipeline in production?

I build low-latency RAG backends in Go and FastAPI with hybrid retrieval, pgvector, and streaming SSE. Email me if you want grounded answers without runaway token costs.

Open to full-time and part-time roles (remote or onsite)

Discuss an AI systemGitHubLinkedIn

Also on Toptal. Email is the best way to start a conversation.