Blog

Vibe coding is incredible.

You can ship a full SaaS product in a weekend. Features that used to take a senior dev three days now take three hours. The speed is real.

But here is what nobody tells you when they post their “I built this in 2 hours” thread on X.

Speed without security is just a faster way to get hacked.

Let me give you some real numbers. A December 2025 study tested five of the most popular vibe coding tools including Cursor, Claude Code, Replit, and Devin across 15 applications. The output contained 69 total vulnerabilities. Around half a dozen were rated critical. A separate Veracode study found that 45% of AI-generated code still contains classic vulnerabilities from the OWASP Top-10 list, with little improvement over two years. And just last week, a Lovable-built app leaked over 18,000 users’ data because the AI implemented the access control logic completely backwards. Authenticated users were blocked. Unauthenticated users got full access.

A human reviewer would have caught that in seconds.

The problem is not vibe coding. The problem is shipping vibe coded apps without understanding what the AI actually built.

I have been building and shipping software for years. Here are the 30 security rules I follow on every single project. No exceptions.

Authentication and Sessions

Rule 1: Set session expiration properly

JWT tokens should have a maximum life of 7 days combined with refresh token rotation. Never issue tokens that live forever.

const token = jwt.sign(
  { userId: user.id },
  process.env.JWT_SECRET,
  { expiresIn: '7d' }
);

Pair this with refresh token rotation so that every time a refresh happens, the old token is invalidated. One leaked token should not last forever.

Rule 2: Never use AI-built auth

This is non-negotiable.

Authentication is the most security-critical part of your entire stack. AI generates plausible-looking auth code that has subtle logic flaws. The Lovable breach mentioned above? Classic AI auth logic inversion.

Use Clerk, Supabase Auth, or Auth0. These are battle-tested, maintained by security teams, and handle the edge cases AI will miss every single time.

Rule 3: Never paste API keys into AI chats

When you paste a key into an AI chat to get help with a bug, you have no idea where that key goes. Use environment variables always.

// Never do this
const client = new OpenAI({ apiKey: "sk-abc123yourrealkeyhere" });

// Always do this
const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

Add your .env file to .gitignore before you write a single line of code. Which brings us to the next rule.

Project Setup

Rule 4: .gitignore is your first file, not your last

Before you scaffold the project. Before you install packages. Before you do anything.

Create .gitignore.

Add .env, node_modules, .DS_Store, and any local config files before your first commit. One accidental push of a .env file to a public repo and your keys are compromised within minutes. GitHub scanners and credential harvesters run constantly.

Rule 5: Rotate secrets every 90 days minimum

Set a calendar reminder. Every 90 days, rotate your API keys, database credentials, and webhook secrets. If you suspect a breach at any point, rotate immediately.

This is not paranoia. This is hygiene.

Rule 6: Verify every package the AI suggests actually exists

This one is genuinely scary and not enough people talk about it.

AI models sometimes suggest packages that do not exist. Attackers monitor for this and register those package names with malicious code inside. It is called slopsquatting, and it is a growing threat vector in 2026.

Before you run npm install on any package the AI recommends, check npmjs.com or pypi.org. Make sure the package exists, has real downloads, and has recent maintenance activity.

Rule 7: Always ask for newer, more secure package versions

When asking AI to scaffold your project, add this to your prompt: “Use the latest stable and most secure version of every package. Flag any deprecated dependencies.”

Old packages have known CVEs. AI models trained on older data will suggest older package versions by default unless you explicitly ask for newer ones.

Rule 8: Run npm audit fix right after building

Make this a habit you cannot break.

npm audit fix

Run it after every major scaffolding session. Review the output. If there are high or critical vulnerabilities that cannot be auto-fixed, address them manually before you ship anything.

Input, Data, and Queries

Rule 9: Sanitize every input. Use parameterized queries always.

SQL injection is still the most exploited vulnerability in web applications in 2026. AI-generated code frequently skips this.

// This will get you hacked
const query = `SELECT * FROM users WHERE email = '${email}'`;

// This is how you do it
const query = 'SELECT * FROM users WHERE email = $1';
const result = await db.query(query, [email]);

Never interpolate user input directly into a query. Ever. Not even once to test something quickly.

Rule 10: Enable Row-Level Security from day one

If you are using Supabase or PostgreSQL, turn on Row-Level Security before you write your first query. Not after. Before.

ALTER TABLE documents ENABLE ROW LEVEL SECURITY;

CREATE POLICY "Users can only access their own documents"
ON documents
FOR ALL
USING (auth.uid() = user_id);

The Moltbook breach in February 2026 exposed 1.5 million API keys and 35,000 email addresses from a misconfigured Supabase database. The entire thing was vibe coded. The database had no proper access controls. Row-Level Security would have prevented it.

Rule 11: Remove all console.log statements before shipping

AI loves adding console.log for debugging. It will log user objects, request bodies, tokens, and internal error details.

Every one of those is a potential data leak in your server logs.

Before you ship, search your entire codebase for console.log and remove or replace with a proper logging library that has log-level controls.

# Quick way to find them all
grep -r "console.log" ./src

API and Endpoint Security

Rule 12: CORS should only allow your production domain

Never use a wildcard CORS policy in production.

// This is dangerous
app.use(cors({ origin: '*' }));

// This is correct
app.use(cors({
  origin: process.env.ALLOWED_ORIGIN, // 'https://yourdomain.com'
  methods: ['GET', 'POST', 'PUT', 'DELETE'],
  credentials: true
}));

A wildcard means any website on the internet can make requests to your API from a user’s browser. That is not an API. That is an open door.

Rule 13: Validate all redirect URLs against an allow-list

Open redirect vulnerabilities are commonly missed in AI-generated auth flows.

const ALLOWED_REDIRECTS = [
  'https://yourdomain.com/dashboard',
  'https://yourdomain.com/onboarding',
  'https://yourdomain.com/settings'
];

function safeRedirect(url) {
  if (ALLOWED_REDIRECTS.includes(url)) {
    return url;
  }
  return '/dashboard'; // safe default
}

If you do not validate, attackers will craft phishing links using your domain as a trusted relay.

Rule 14: Apply auth and rate limits to every endpoint including mobile APIs

AI-generated backends often protect the web routes and forget the mobile API routes entirely.

Every endpoint that touches user data needs authentication. Every endpoint that accepts input needs rate limiting. No exceptions for mobile, internal, or admin routes.

Rule 15: Rate limit everything from day one

100 requests per hour per IP is a reasonable starting point. Adjust based on your use case.

import rateLimit from 'express-rate-limit';

const limiter = rateLimit({
  windowMs: 60 * 60 * 1000, // 1 hour
  max: 100,
  message: 'Too many requests from this IP. Please try again later.',
  standardHeaders: true,
  legacyHeaders: false
});

app.use('/api/', limiter);

Without rate limiting, a single attacker can enumerate your users, brute force passwords, or burn through your AI API budget in minutes.

Rule 16: Password reset routes get their own strict limit

Your general rate limit is not enough for password reset flows. These are high-value attack targets.

const passwordResetLimiter = rateLimit({
  windowMs: 60 * 60 * 1000, // 1 hour
  max: 3, // only 3 reset attempts per email per hour
  keyGenerator: (req) => req.body.email, // limit per email, not per IP
  message: 'Too many reset attempts. Please try again in an hour.'
});

app.post('/auth/reset-password', passwordResetLimiter, resetHandler);

Infrastructure and Cost Controls

Rule 17: Cap AI API costs in your dashboard AND in your code

Do both. Not one or the other.

Set a hard spend limit in your OpenAI or Anthropic dashboard. Then add a check in your code that tracks spend and returns a graceful error when the limit is hit. A single runaway loop or prompt injection attack can burn through thousands of dollars before you wake up.

Rule 18: Add DDoS protection via Cloudflare or Vercel Edge Config

Put your app behind Cloudflare on day one. It is free at the base tier and gives you DDoS protection, bot filtering, and rate limiting at the edge before traffic even hits your server.

If you are on Vercel, use Edge Config for geographic blocking and bot protection rules. This is not optional for any app with real users.

Rule 19: Lock down storage buckets

Users should only be able to access their own files. Not each other’s. Not all files in a folder. Only their own.

// Supabase storage policy example
CREATE POLICY "Users access only their own files"
ON storage.objects
FOR ALL
USING (auth.uid()::text = (storage.foldername(name))[1]);

Default storage bucket settings in Supabase are public. You have to explicitly lock them down. AI-generated code will not do this for you unless you ask.

Rule 20: Limit upload sizes and validate file type by signature

Extension validation is useless. A malicious file named payload.jpg is still a malicious file.

import fileType from 'file-type';

async function validateUpload(buffer, maxSizeMB = 10) {
  // Check size
  if (buffer.length > maxSizeMB * 1024 * 1024) {
    throw new Error('File too large');
  }

  // Check actual file signature, not extension
  const type = await fileType.fromBuffer(buffer);
  const allowed = ['image/jpeg', 'image/png', 'image/webp', 'application/pdf'];

  if (!type || !allowed.includes(type.mime)) {
    throw new Error('File type not allowed');
  }

  return type;
}

Payments, Email, and Webhooks

Rule 21: Verify webhook signatures before processing any payment data

A webhook without signature verification means anyone on the internet can send your server fake payment events.

// Stripe webhook verification
import Stripe from 'stripe';

const stripe = new Stripe(process.env.STRIPE_SECRET_KEY);

app.post('/webhooks/stripe', express.raw({ type: 'application/json' }), (req, res) => {
  const sig = req.headers['stripe-signature'];

  let event;
  try {
    event = stripe.webhooks.constructEvent(
      req.body,
      sig,
      process.env.STRIPE_WEBHOOK_SECRET
    );
  } catch (err) {
    return res.status(400).send(`Webhook signature verification failed: ${err.message}`);
  }

  // Now it is safe to process
  handleStripeEvent(event);
  res.json({ received: true });
});

Rule 22: Use Resend or SendGrid with proper SPF/DKIM records

Do not send email from a raw SMTP connection or an unverified domain. Set up SPF, DKIM, and DMARC records for your sending domain. Without these, your transactional emails go to spam and your domain reputation gets destroyed.

Resend makes this setup genuinely easy. Do it before your first email goes out.

Permissions, Logs, and Compliance

Rule 23: Check permissions server-side. UI-level checks are not security.

This is one of the most common mistakes in AI-generated code.

Hiding a button in the UI does not prevent anyone from calling the API endpoint directly. Every permission check must happen on the server.

// This is not security
if (user.role === 'admin') {
  showDeleteButton();
}

// This is security
app.delete('/api/users/:id', authenticate, async (req, res) => {
  if (req.user.role !== 'admin') {
    return res.status(403).json({ error: 'Forbidden' });
  }
  // proceed with deletion
});

Rule 24: Ask the AI to act as a security engineer and review your code

After building any feature, do this before you commit.

Paste your code and say: “Act as a senior security engineer. Review this code for vulnerabilities including injection attacks, broken authentication, insecure direct object references, missing authorization, and data exposure. List every issue with severity and a fix.”

You will be surprised what it finds.

Rule 25: Ask the AI to try and hack your app

This one sounds aggressive. It is also one of the most useful things you can do.

Say: “Act as a malicious hacker. I am going to describe my app’s architecture. Try to find ways to exploit it. Be specific about attack vectors.”

It will surface things a standard code review will miss.

Rule 26: Log critical actions

Deletions, role changes, payment events, data exports, and admin actions all need to be logged with timestamp, user ID, IP address, and what changed.

async function logCriticalAction(userId, action, metadata) {
  await db.query(
    'INSERT INTO audit_log (user_id, action, metadata, ip, created_at) VALUES ($1, $2, $3, $4, NOW())',
    [userId, action, JSON.stringify(metadata), getClientIP()]
  );
}

// Use it everywhere that matters
await logCriticalAction(user.id, 'ACCOUNT_DELETED', { email: user.email });
await logCriticalAction(user.id, 'ROLE_CHANGED', { from: 'member', to: 'admin' });
await logCriticalAction(user.id, 'EXPORT_TRIGGERED', { recordCount: rows.length });

Rule 27: Build a real account deletion flow

GDPR fines are not theoretical. Build a proper account deletion flow that removes personal data from your database, revokes all active sessions, cancels active subscriptions, and sends a confirmation email.

AI will not build this correctly unless you explicitly ask for it with every requirement spelled out.

Rule 28: Automate backups and test restoration

An untested backup is not a backup. It is a false sense of security.

Automate daily database backups. Once a month, actually restore one to a test environment and verify the data is intact and the app works. Document the restoration process so anyone on your team can do it, not just you.

Rule 29: Keep test and production environments completely separate

Separate databases. Separate API keys. Separate environment variables. Separate Stripe accounts in test mode vs live mode.

Never let test data touch production infrastructure. Never let production credentials exist in your local development environment.

Rule 30: Never let test webhooks touch real systems

Use Stripe test mode webhooks for local development and staging. Use Stripe live webhooks for production only. Use Stripe’s webhook CLI tool to forward events in development.

stripe listen --forward-to localhost:3000/webhooks/stripe

One misconfigured environment variable pointing your test server at the live Stripe webhook endpoint has already cost founders real money.

Ship Fast. Ship Secure.

Here is the reality of vibe coding in 2026.

The tools are extraordinary. The speed is real. The ability to ship a full product in a weekend is genuinely possible and genuinely impressive.

But the AI does not know your threat model. It does not know which of your users are high-value targets. It does not know that your storage bucket is wide open or that your webhook has no signature verification. It will generate code that works perfectly in a demo and has critical vulnerabilities in production.

Your job is not to write every line. Your job is to review, validate, and own everything that ships.

Thirty rules. None of them optional. All of them faster to implement upfront than to fix after a breach.

Ship fast. But ship secure.

You built a RAG pipeline. Tested it with five questions. It worked perfectly.

Then you shipped it.

And everything broke.

Users asked ambiguous questions. The vector database pulled irrelevant chunks. The model hallucinated confidently. Leadership lost trust in three days.

Sound familiar?

Here’s the thing. That’s not a model problem. That’s not even a data problem. That’s an architecture problem.

I keep seeing the same pattern repeat in 2026. Something ships quickly, the demo looks fine, leadership is satisfied. Then real users start asking real questions. The answers are vague. Sometimes wrong. Occasionally confident and completely nonsensical. Trust disappears fast, and once users decide a system can’t be trusted, they simply stop using it. They won’t give it a second chance.

Building a bad RAG system is worse than no RAG at all.

The good news? The failure modes are completely predictable. And they all trace back to four layers that most teams either skip or underbuild. This post breaks down exactly what each layer needs, with real Python code you can use today.

Let’s get into it.

Why Your RAG Demo Works But Your Production System Doesn’t

The naive pipeline everyone starts with looks like this:

# What most teams build (and regret)
def naive_rag(query: str) -> str:
    embedding = embed(query)
    chunks = vector_db.search(embedding, top_k=5)
    context = "\n".join(chunks)
    return llm.generate(f"Context: {context}\n\nQuestion: {query}")

This works on demos. It fails in production because it makes four dangerous assumptions:

1. Every question is semantic (it isn’t)
2. Retrieval results are always good enough to generate from (they aren’t)
3. Naive chunking preserves meaning (it doesn’t)
4. If the model can’t find good context, it will admit it (it won’t)

The math here is brutal. A system that retrieves the wrong document, reranks poorly, and generates a hallucination didn’t fail once. It failed four or five times in sequence. When each failure compounds, your 95% accuracy per layer becomes an 81% reliable system overall. That means your system fails one in five times.

Here’s how to fix all four failure points.

Layer 1: Hybrid Retrieval (Vector Search Is Not Enough)

Embeddings are fantastic for meaning. They are awful for exact identity.

When a user asks “explain our refund policy,” vector search works great. When they ask “show me contract A-1023,” vector search will return semantically similar contracts, not the exact one. When they ask “what was our Q3 revenue,” you need SQL, not cosine similarity.

A user searching for “ISO 27001 compliance requirements” is a perfect example. Pure vector search might return documents about “security best practices” and “compliance frameworks,” which are semantically similar but miss the specific standard. The one document that explicitly mentions ISO 27001 by name gets buried because it doesn’t have the richest semantic context. BM25 catches the exact keyword match that vector search glossed over.

Hybrid approaches can improve recall accuracy by 1% to 9% compared to vector search alone, depending on implementation. That gap matters massively at scale.

Here is a production hybrid retriever combining BM25, vector search, and a cross-encoder reranker:

from langchain.retrievers import BM25Retriever, EnsembleRetriever
from langchain.vectorstores import Pinecone
from langchain.embeddings import OpenAIEmbeddings
from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import CrossEncoderReranker
from langchain_community.cross_encoders import HuggingFaceCrossEncoder

# Step 1: Set up both retrievers
vector_store = Pinecone.from_existing_index("your-index", OpenAIEmbeddings())
vector_retriever = vector_store.as_retriever(search_kwargs={"k": 20})

bm25_retriever = BM25Retriever.from_documents(documents)
bm25_retriever.k = 20

# Step 2: Combine with Reciprocal Rank Fusion
# Tune weights based on your query distribution
# Higher BM25 weight for keyword-heavy domains (legal, medical)
# Higher vector weight for conversational/exploratory queries
ensemble_retriever = EnsembleRetriever(
    retrievers=[bm25_retriever, vector_retriever],
    weights=[0.4, 0.6]
)

# Step 3: Rerank the merged results using a cross-encoder
# Cross-encoders score query+chunk pairs together, much more accurate than embeddings alone
reranker_model = HuggingFaceCrossEncoder(model_name="BAAI/bge-reranker-large")
compressor = CrossEncoderReranker(model=reranker_model, top_n=5)

final_retriever = ContextualCompressionRetriever(
    base_compressor=compressor,
    base_retriever=ensemble_retriever
)

Add a SQL path for structured data questions:

from langchain import SQLDatabase
from langchain.chains import create_sql_query_chain

db = SQLDatabase.from_uri("postgresql://user:pass@localhost/mydb")
sql_chain = create_sql_query_chain(llm, db)

def retrieve_by_type(query: str, query_type: str) -> list:
    if query_type == "structured":
        sql = sql_chain.invoke({"question": query})
        return db.run(sql)
    elif query_type == "exact":
        return bm25_retriever.get_relevant_documents(query)
    else:
        return final_retriever.get_relevant_documents(query)

Also important in 2026: track embedding drift. You embed your knowledge base once. Six months later, your domain language evolves with new regulations or product launches, but your vectors are stale. Retrieval quality degrades silently. Users don’t notice until your competitor’s RAG answers better. The fix is to embed incrementally, monitor embedding drift via cosine similarity distribution changes, and re-embed cold data quarterly. Track embedding model versions like source code versions.

Layer 2: Intelligent Query Routing

This is the layer almost nobody builds. And it removes roughly 80% of bad answers before retrieval even runs.

Before fetching anything, your system needs to make three decisions:

• Is this semantic or exact or structured?
• Is this a single-hop or multi-hop question?
• Which data source should answer this?

Modern production systems now add intent classification as a first step: an LLM analyzes query complexity and determines retrieval strategy, distinguishing simple lookup from multi-hop reasoning. Query transformation then rewrites vague queries into specific, retrievable forms before any retrieval happens.

Here is a full query router with Pydantic output parsing:

from pydantic import BaseModel
from enum import Enum
from langchain.output_parsers import PydanticOutputParser

class QueryType(str, Enum):
    SEMANTIC = "semantic"       # "explain our refund policy"
    EXACT = "exact"             # "find contract A-1023"
    STRUCTURED = "structured"  # "what was Q3 revenue"
    MULTI_HOP = "multi_hop"    # "compare our policy to competitors"

class QueryRoute(BaseModel):
    query_type: QueryType
    data_source: str            # "vector_db", "sql", "graph", "hybrid"
    sub_queries: list[str]      # for multi-hop, break into steps
    rewritten_query: str        # cleaned-up version of the original
    reasoning: str

parser = PydanticOutputParser(pydantic_object=QueryRoute)

ROUTING_PROMPT = """
Analyze this query and determine the best retrieval strategy.

Query: {query}

Consider:
- Is it asking for a concept or explanation (semantic) or a specific named item (exact)?
- Does it need joining information from multiple sources (multi-hop)?
- Does it reference numbers, dates, or IDs that suggest structured data?
- Can you rewrite it more precisely without changing the meaning?

{format_instructions}
"""

def route_query(query: str) -> QueryRoute:
    prompt = ROUTING_PROMPT.format(
        query=query,
        format_instructions=parser.get_format_instructions()
    )
    response = llm.invoke(prompt)
    return parser.parse(response.content)

For multi-hop queries, use the previous retrieval result to inform the next:

def multi_hop_retrieve(route: QueryRoute) -> list:
    all_context = []

    for sub_query in route.sub_queries:
        sub_route = route_query(sub_query)
        results = retrieve_by_type(sub_query, sub_route.query_type)
        all_context.extend(results)

        # Use what we just found to refine the next sub-query
        if all_context:
            enriched = f"{sub_query}\nContext so far: {all_context[-1]}"
            results = retrieve_by_type(enriched, sub_route.query_type)

    return all_context

Layer 3: Advanced Indexing (Chunking Is Not Enough)

80% of RAG failures trace back to chunking decisions. Not retrieval. Not generation. Chunking.

Fixed window chunking splits by length with optional overlap. It is easy to implement but can break semantic units and degrade answer grounding. Title-based splitting preserves author intent and improves attribution when users ask about a specific policy or procedure. Similarity-based splitting detects semantic shifts using embeddings and reduces topic mixing. Tables deserve special handling because they contain dense facts with strong row and column semantics.

Here is a semantic chunker with hierarchical parent-child indexing:

from langchain.text_splitter import SemanticChunker
from langchain.embeddings import OpenAIEmbeddings
from langchain.schema import Document

# Semantic chunking splits on meaning, not token count
semantic_splitter = SemanticChunker(
    embeddings=OpenAIEmbeddings(),
    breakpoint_threshold_type="percentile",
    breakpoint_threshold_amount=95
)

def create_hierarchical_index(documents: list[Document]) -> dict:
    indexed = {}

    for doc in documents:
        # Level 1: document-level summary for broad questions
        summary = llm.invoke(
            f"Summarize this document in 2 sentences, focusing on its main topic and key facts:\n{doc.page_content}"
        )

        # Level 2: semantic chunks for specific questions
        chunks = semantic_splitter.create_documents([doc.page_content])

        # Attach parent reference and summary to each chunk
        # This allows retrieval of the child but return of the full parent context
        for i, chunk in enumerate(chunks):
            chunk.metadata.update({
                "parent_doc_id": doc.metadata["id"],
                "chunk_index": i,
                "total_chunks": len(chunks),
                "doc_summary": summary.content,
                "source": doc.metadata.get("source", "unknown")
            })

        indexed[doc.metadata["id"]] = {
            "summary": summary.content,
            "chunks": chunks,
            "original": doc
        }

    return indexed

# Retrieve the child chunk, return the full parent section for more context
def retrieve_with_parent_context(query: str, top_k: int = 5) -> list:
    child_results = vector_retriever.get_relevant_documents(query)

    parent_context = []
    seen_parents = set()

    for chunk in child_results:
        parent_id = chunk.metadata.get("parent_doc_id")

        if parent_id and parent_id not in seen_parents:
            parent = get_parent_document(parent_id)
            parent_context.append(parent)
            seen_parents.add(parent_id)
        else:
            parent_context.append(chunk)

    return parent_context[:top_k]

Handle PDFs with mixed tables and text using structure-aware parsing:

from unstructured.partition.pdf import partition_pdf
import pandas as pd

def process_mixed_document(file_path: str) -> list[Document]:
    elements = partition_pdf(file_path, strategy="hi_res")
    processed = []

    for element in elements:
        if element.category == "Table":
            # Store both markdown representation and a plain-text description
            # Markdown helps with exact retrieval, description helps with semantic retrieval
            processed.append(Document(
                page_content=f"TABLE:\n{element.metadata.text_as_html}\n\nDescription: {element.text}",
                metadata={"type": "table", "source": file_path}
            ))
        elif element.category == "Title":
            processed.append(Document(
                page_content=element.text,
                metadata={"type": "title", "source": file_path}
            ))
        else:
            processed.append(Document(
                page_content=element.text,
                metadata={"type": "text", "source": file_path}
            ))

    return processed

Also critical in 2026: frequent index refresh cycles are now standard. Daily for dynamic content like product catalogs and compliance docs. Hourly for real-time use cases like customer support and news feeds. Stale indexes are a silent killer.

Layer 4: Evaluation Loop (Non-Negotiable)

If you can’t measure it, you can’t fix it. And in RAG, what you can’t fix will silently get worse.

Most evaluations start with a simple “vibe check” where you test domain-specific questions and see if the application answers sensibly. But once you have a baseline, you need systematic evaluation of both retrieval and generation separately. Teams often rely on manual validation by subject matter experts, but this leads to a slower development cycle and can be subjective.

Open-source frameworks like Ragas and DeepEval provide standardized approaches for generating test datasets, defining custom metrics, and monitoring in production. However, they have limitations: scores can be inconsistent between runs for the same inputs, and biased results have been reported when the same LLM that generates answers also judges them. Knowing this, use them as directional signals, not gospel.

Here is a full eval setup with a pre-deploy gate:

from ragas import evaluate
from ragas.metrics import (
    faithfulness,
    answer_relevancy,
    context_precision,
    context_recall
)
from datasets import Dataset
import json

def evaluate_rag_pipeline(test_cases: list[dict]) -> dict:
    """
    test_cases format:
    [{"question": "...", "ground_truth": "...", "answer": "...", "contexts": [...]}]
    """
    dataset = Dataset.from_list(test_cases)

    results = evaluate(
        dataset,
        metrics=[
            faithfulness,       # Is the answer grounded in retrieved context?
            answer_relevancy,   # Does the answer address the actual question?
            context_precision,  # Are retrieved chunks relevant?
            context_recall      # Did retrieval find everything needed?
        ]
    )

    return results

def pre_deploy_eval(pipeline, eval_set_path: str) -> bool:
    with open(eval_set_path) as f:
        test_cases = json.load(f)

    results = []
    for case in test_cases:
        answer, contexts = pipeline.run(case["question"])
        results.append({
            "question": case["question"],
            "ground_truth": case["ground_truth"],
            "answer": answer,
            "contexts": contexts
        })

    scores = evaluate_rag_pipeline(results)

    # Block deployment if scores drop below thresholds
    THRESHOLDS = {
        "faithfulness": 0.85,
        "answer_relevancy": 0.80,
        "context_precision": 0.75,
        "context_recall": 0.70
    }

    failed = []
    for metric, threshold in THRESHOLDS.items():
        if scores[metric] < threshold:
            failed.append(f"{metric}: {scores[metric]:.2f} < {threshold}")

    if failed:
        print(f"DEPLOYMENT BLOCKED: {failed}")
        return False

    print("All metrics passed. Safe to deploy.")
    return True

Add a confidence gate so the system admits when it doesn’t know instead of hallucinating:

def rag_with_confidence_gate(query: str) -> dict:
    route = route_query(query)
    chunks = retrieve_by_type(query, route.query_type)

    if not chunks:
        return {
            "answer": "I don't have relevant information to answer this question.",
            "confidence": 0.0,
            "chunks_used": []
        }

    # Score each chunk against the query before generating
    relevance_scores = [
        cross_encoder.predict([(query, chunk.page_content)])[0]
        for chunk in chunks
    ]

    max_relevance = max(relevance_scores)

    # Below threshold, admit ignorance rather than hallucinate
    if max_relevance < 0.5:
        return {
            "answer": "I couldn't find information relevant enough to answer this confidently.",
            "confidence": max_relevance,
            "chunks_used": []
        }

    context_with_sources = [
        f"[Source {i+1}]: {chunk.page_content}"
        for i, chunk in enumerate(chunks)
    ]

    answer = llm.invoke(
        f"Answer using only the provided sources. Cite [Source N] for each claim.\n\n"
        f"{''.join(context_with_sources)}\n\nQuestion: {query}"
    )

    return {
        "answer": answer.content,
        "confidence": max_relevance,
        "chunks_used": [c.metadata for c in chunks]
    }

Add continuous production monitoring that alerts before users complain:

import logging
from datetime import datetime, timedelta

class RAGMonitor:
    def __init__(self):
        self.logger = logging.getLogger("rag_monitor")

    def log_query(self, query: str, result: dict, latency_ms: float):
        self.logger.info({
            "timestamp": datetime.utcnow().isoformat(),
            "query_hash": hash(query),  # Don't log raw PII queries
            "confidence": result["confidence"],
            "chunks_retrieved": len(result["chunks_used"]),
            "latency_ms": latency_ms,
            "answered": result["confidence"] > 0.5
        })

    def check_health(self, window_minutes: int = 60):
        recent = self.get_recent_logs(window_minutes)
        if not recent:
            return

        answer_rate = sum(1 for l in recent if l["answered"]) / len(recent)
        avg_confidence = sum(l["confidence"] for l in recent) / len(recent)
        avg_latency = sum(l["latency_ms"] for l in recent) / len(recent)

        # 2026 standard: p90 TTFT should stay under 2 seconds
        if avg_latency > 2000:
            self.send_alert(f"Avg latency {avg_latency:.0f}ms exceeds 2s SLA")
        if answer_rate < 0.70:
            self.send_alert(f"Answer rate dropped to {answer_rate:.0%}")
        if avg_confidence < 0.60:
            self.send_alert(f"Avg confidence dropped to {avg_confidence:.2f}")

Putting It All Together

Here is the complete production pipeline with all four layers wired up:

import time

class ProductionRAG:
    def __init__(self):
        self.router = QueryRouter()
        self.retriever = HybridRetriever()
        self.reranker = CrossEncoderReranker()
        self.generator = LLMGenerator()
        self.monitor = RAGMonitor()

    def run(self, query: str) -> dict:
        start = time.time()

        # Layer 2: Route before you retrieve
        route = self.router.route(query)

        # Layer 1: Hybrid retrieval based on route type
        if route.query_type == "multi_hop":
            chunks = multi_hop_retrieve(route)
        else:
            chunks = self.retriever.retrieve(route.rewritten_query, route)

        # Layer 3: Rerank with cross-encoder
        chunks = self.reranker.rerank(route.rewritten_query, chunks, top_n=5)

        # Confidence gate before generation
        if not self.has_sufficient_confidence(route.rewritten_query, chunks):
            return {
                "answer": "I don't have enough relevant context to answer confidently.",
                "confidence": 0.0,
                "chunks_used": []
            }

        # Generate with citations
        result = self.generator.generate(route.rewritten_query, chunks)

        # Layer 4: Log for monitoring and eval
        latency = (time.time() - start) * 1000
        self.monitor.log_query(query, result, latency)

        return result

    def has_sufficient_confidence(self, query: str, chunks: list) -> bool:
        if not chunks:
            return False
        scores = [cross_encoder.predict([(query, c.page_content)])[0] for c in chunks]
        return max(scores) >= 0.5

One Cost Optimization Worth Knowing

Before you ship at scale, add semantic caching. Semantic caching cuts LLM costs by up to 68.8% in typical production workloads by returning cached answers for semantically similar queries rather than hitting the LLM every time.

from langchain.cache import InMemoryCache
from langchain.globals import set_llm_cache
import numpy as np

class SemanticCache:
    def __init__(self, similarity_threshold: float = 0.95):
        self.cache = {}
        self.threshold = similarity_threshold

    def get(self, query: str) -> str | None:
        query_embedding = embed(query)

        for cached_query, (cached_embedding, cached_answer) in self.cache.items():
            similarity = np.dot(query_embedding, cached_embedding)
            if similarity >= self.threshold:
                return cached_answer

        return None

    def set(self, query: str, answer: str):
        self.cache[query] = (embed(query), answer)

# Wrap your RAG pipeline with the cache
semantic_cache = SemanticCache(similarity_threshold=0.95)

def cached_rag(query: str) -> dict:
    cached = semantic_cache.get(query)
    if cached:
        return {"answer": cached, "source": "cache"}

    result = production_rag.run(query)
    semantic_cache.set(query, result["answer"])
    return result

The Hard Truth About RAG in 2026

In 2026, if your knowledge base is small enough to fit in context windows, you may not even need RAG at all. For knowledge bases under roughly 200,000 tokens, full-context prompting plus prompt caching can be faster and cheaper than building retrieval infrastructure. Know when to use the tool and when not to.

But for anything larger, the gap between demo RAG and production RAG is these four layers.

Most teams treat RAG as a feature. Connect an LLM to a vector database. Run a demo. Ship it. Then spend the next six months firefighting.

The teams shipping reliable AI products in 2026 are not the ones with the best models. They’re the ones who treated retrieval like feature engineering, built evaluation into their deployment pipeline, and monitor production like an actual system.

Build systems. Not toys.

I’ve been in the software business for over 15 years.

When GPT-3.5 launched in November 2022, I started using it to fix and optimize code. Nothing crazy – just a productivity boost. But when tools like Cursor and GitHub Copilot came along, everything changed. I went from using AI occasionally to being completely dependent on vibe coding tools for almost everything I build.

And the results have been insane.

Code that used to take days now gets written in minutes – and honestly, in a better way than I would have written it manually.

But here’s what I keep hearing from friends and fellow builders. They’re frustrated. They say vibe coding doesn’t work for them. They’re getting broken outputs, half-finished features, and code they can’t understand. And every time I dig into what’s going wrong, it’s the same answer.

They’re prompting it wrong.

The only skill you need to get dramatically better results from vibe coding is learning how to write better prompts. Full stop. That’s the unlock.

Before I get into how, let me address something that’s been bothering me.

I keep seeing developers on social media saying vibe coding isn’t going to take their jobs. That it “can’t really code.” They’ve heard the buzzword, downloaded one of the tools, tried it as a demo, got imperfect results, and breathed a sigh of relief. “I’m safe,” they think. “This thing is overrated.”

Let me be very direct: you didn’t actually do vibe coding. You gave it a bad prompt.

There is nothing that vibe coding can’t do. I built a complete portfolio website for my wife – from buying the domain on GoDaddy to going live on Vercel – in just 20 minutes. A website that would have taken days a few years ago. Tools like Cursor, Lovable, Bolt, Replit, GitHub Copilot, v0 by Vercel, Windsurf, and Claude aren’t toys. They are professional-grade development environments that are replacing entire workflows.

I’ve been doing business since 2010. I know how markets shift. Within the next one to two years, when companies start making decisions based on output speed rather than headcount, the perception will change fast. My goal isn’t to create panic. It’s to help you grow and get ahead of the curve.

The best way to do that right now is to learn how to prompt well.

Here’s exactly how.

Why Most Vibe Coding Prompts Fail

Vague prompts produce vague code.

When you tell an AI “build a project management app,” you’re handing the wheel to a model that will make dozens of architectural decisions on your behalf – most of which you won’t like once you see them.

The result? Code that technically runs but falls apart the moment you try to scale it, modify it, or hand it to someone else.

Think of your AI as a brilliant but overeager junior developer. Left unsupervised, they’ll build a skyscraper on a foundation of sand. Managed well, they’ll ship faster than any team you’ve ever worked with.

Step 1: Set the Role Before You Write a Single Line

The first thing you should write in any vibe coding session isn’t a task. It’s a persona.

Tell the AI who it is.

This one change alone will transform the quality of your output. A system message that establishes the AI’s role – for example, “You are a senior Python developer who adheres to PEP8 style and security best practices” – directly influences the tone and correctness of everything that follows.

Try these role-setting prompts:

• “You are a senior full-stack engineer specializing in production-grade Next.js applications. You prioritize security, scalability, and clean architecture above all else.”
• “You are a backend Python developer with 10 years of experience building multi-tenant SaaS products on AWS. You write defensive code and always handle edge cases.”
• “You are a senior React developer. You write clean, accessible, and performant components. You never use inline styles and always follow component separation principles.”

Don’t skip this step. It takes 30 seconds and changes everything that follows.

Step 2: Write a Mini PRD Before You Prompt

Here’s what separates builders who ship from builders who spin.

Before you ask the AI to write code, write down what you’re building. A short Product Requirements Document – even just a paragraph – gives the AI the full picture before it writes a single line.

Your mini PRD needs three things:

1. What you’re building – e.g., “A client dashboard where users can track their subscription invoices”
2. Who it’s for – e.g., “Small business owners, non-technical, accessing on mobile”
3. How it works – e.g., “Reads from a Stripe API, displays in a sortable table, exports to CSV”

Paste this context at the start of your session. Your AI now has the full picture. It will make better decisions, ask better clarifying questions, and produce code that actually fits your use case.

Step 3: Break Big Prompts Into Small, Goal-Driven Steps

This is the mistake I see everywhere.

People write one massive prompt – “build the entire app” – and then get frustrated when the output is a mess.

Here’s the thing. Instead of one big prompt, break it down into smaller, goal-driven steps. Set up the database first. Then build the dashboard. Each step gives the AI a clear, contained job – and the code quality at each step is dramatically better.

A real example:

❌ Bad prompt:

“Build a SaaS dashboard with user authentication, billing, analytics, and a settings page.”

✅ Good prompt sequence:

1. “Set up the database schema with tables for users, subscriptions, and events. Use PostgreSQL conventions.”
2. “Now create the authentication flow using NextAuth. Support email/password and Google OAuth.”
3. “Build the analytics dashboard component that reads from the events table. Show a 30-day chart.”
4. “Create the billing settings page that integrates with the Stripe Customer Portal.”

Same end result. Dramatically better code at every step.

Step 4: Always Ask for the Plan Before the Code

This is a habit that will save you hours.

Before the AI writes a single line, ask it to explain its approach first.

Even if you can’t read code, ask the AI what it wants to do before it does anything. Nine out of ten times it’ll suggest an overcomplicated approach – and that’s your chance to push back before any code is written.

Use this prompt before any complex feature:

“Before coding, give me a few options for how to approach this, starting with the simplest. Don’t write any code yet.”

Then pick the option that makes sense and say: “Go with option 2. Now write the code.”

This two-step process keeps you in control of architecture decisions – even if you can’t read the code itself.

Step 5: Include Both Functional and Non-Functional Requirements

Most prompts describe what the code should do. Almost none describe what it should be.

This is a critical gap.

Production-ready code isn’t just functional. It’s secure. It’s performant. It handles errors gracefully. It doesn’t expose sensitive data. The best prompts specify both the task and the definition of done.

❌ Functional-only prompt:

“Write a function that fetches user data from the API.”

✅ Full-requirements prompt:

“Write a function that fetches user data from the API. Requirements: handle 401, 403, and 500 errors with appropriate error messages; never log sensitive user fields like email or password; add a 5-second timeout; return null on failure instead of throwing. Add JSDoc comments.”

The second prompt takes 20 extra seconds to write. It saves you 45 minutes of debugging.

Step 6: Set Explicit Constraints to Kill Code Bloat

AI models have a habit of over-engineering.

Ask for a button, get a button with animations, three variants, full Storybook documentation, and a custom hook. Ask for a simple API call, get an entire abstraction layer you didn’t ask for and don’t understand.

Setting clear limits transforms AI from an eager intern into a disciplined collaborator.

Add constraint language to every prompt:

• “Keep it simple. Use the fewest dependencies possible.”
• “Do not introduce new libraries. Use what’s already in the project.”
• “Write this in under 50 lines.”
• “No abstractions. Just the code I need for this specific use case.”

This is especially important if you’re a non-technical founder. You want code you can understand and modify, not a masterpiece you can never touch.

Step 7: Use the “Senior Architect Mindset” for Complex Features

When you’re building something genuinely complex – authentication, payments, multi-tenancy, real-time data – don’t approach it like a user. Approach it like an architect.

The best vibe coders don’t just ask for code. They manage the AI like a junior developer, enforcing strict constraints and clear architectural patterns.

Here’s the prompt structure that works every time:

“You are a senior cloud architect. I need to implement [feature]. Before writing any code: (1) List your assumptions. (2) Outline the plan step by step. (3) Flag any potential risks or edge cases. Then write the code following the plan.”

That three-part structure forces the AI to think before it types. The code that comes out the other side is measurably better.

Step 8: Run These Four Quality Prompts Before You Ship

You’ve built the feature. It seems to work. Don’t ship it yet.

Use these four prompts as your pre-launch checklist every single time:

Security audit:

“Act as a security engineer. Review this code for vulnerabilities: SQL injection, XSS, insecure API keys, exposed sensitive data, missing authentication checks. List every issue and fix each one.”

Performance check:

“Review this code for performance issues. Look for unnecessary re-renders, unoptimized database queries, missing indexes, memory leaks, and blocking operations. Suggest fixes.”

Maintainability review:

“Act as a senior engineer doing a code review. Identify the top 5 functions that are too complex or have unclear names. Refactor them for clarity and add comments.”

Error handling:

“Review this code for missing error handling. Identify every place where the app could crash silently or expose unhelpful error messages to users. Add proper error handling throughout.”

Run all four. Fix what they find. Then ship.

The Prompting Framework That Changes Everything

Every great vibe coding prompt has five elements:

Use all five every time and you’ll stop fighting your AI and start shipping with it.

One Last Thing

Vibe coding isn’t about removing yourself from the process.

It’s about becoming a better director.

The builders winning right now aren’t the ones who type the least. They’re the ones who give the clearest direction, catch problems early, and review every output before it goes live.

Master the prompts. Own the architecture. Ship with confidence.

Now go build something.