<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Why 99% of RAG Apps Crash in Production (Naive vs Scaled Node.js)]]></title><description><![CDATA[Anyone who is developing AI systems, RAG pipelines, Agents.]]></description><link>https://gauravthorat.hashnode.dev</link><image><url>https://cdn.hashnode.com/uploads/logos/6a19b030b55c6a731d1e53c2/5e75fba3-a39a-4bf5-a4a2-9fe3ea4bb37c.jpg</url><title>Why 99% of RAG Apps Crash in Production (Naive vs Scaled Node.js)</title><link>https://gauravthorat.hashnode.dev</link></image><generator>RSS for Node</generator><lastBuildDate>Wed, 18 Nov 2026 15:06:46 GMT</lastBuildDate><atom:link href="https://gauravthorat.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Why Your React Frontend Crashes When an LLM Streams Malformed JSON]]></title><description><![CDATA[A production-minded walkthrough with a live Next.js demo - JSON.parse() vs partial-json + Zod for real-time AI dashboards.]]></description><link>https://gauravthorat.hashnode.dev/why-your-react-frontend-crashes-when-an-llm-streams-malformed-json</link><guid isPermaLink="true">https://gauravthorat.hashnode.dev/why-your-react-frontend-crashes-when-an-llm-streams-malformed-json</guid><dc:creator><![CDATA[Gaurav Thorat]]></dc:creator><pubDate>Tue, 09 Jun 2026 09:51:03 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6a19b030b55c6a731d1e53c2/7a5b98b0-7178-4d0a-8508-56763bb7ead2.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h3><em>canonical:</em> <a href="https://gauravthorat-portfolio.vercel.app/blog/react-llm-stream-json-parser"><em>https://gauravthorat-portfolio.vercel.app/blog/react-llm-stream-json-parser</em></a></h3>
<p>Your React AI dashboard didn't break because of React.</p>
<p>It broke because JSON.parse() was called on a live LLM stream buffer — incomplete JSON by design.</p>
<p>I just published a full walkthrough:</p>
<p>→ Why streaming breaks JSON.parse() → Why try/catch and DIY repair fail in production → A resilient pipeline: partial-json + Zod .partial() → A runnable Next.js demo you can clone in 5 minutes</p>
<p>If you build real-time AI dashboards, this is worth a read.</p>
<p>#react #nextjs #typescript #ai #frontend #webdevelopment</p>
]]></content:encoded></item><item><title><![CDATA[Why 99% of RAG Apps Crash in Production (Naive vs Scaled Node.js)]]></title><description><![CDATA[Monday hits. 1,000 users query at once. Suddenly everything breaks: 502 errors, ECONNRESET, OpenAI 429 rate limits, Pinecone timeouts.]]></description><link>https://gauravthorat.hashnode.dev/why-99-of-rag-apps-crash-in-production-naive-vs-scaled-node-js</link><guid isPermaLink="true">https://gauravthorat.hashnode.dev/why-99-of-rag-apps-crash-in-production-naive-vs-scaled-node-js</guid><category><![CDATA[AI]]></category><category><![CDATA[RAG ]]></category><category><![CDATA[TypeScript]]></category><category><![CDATA[Node.js]]></category><category><![CDATA[agentic AI]]></category><category><![CDATA[Production ai]]></category><category><![CDATA[web dev]]></category><dc:creator><![CDATA[Gaurav Thorat]]></dc:creator><pubDate>Fri, 29 May 2026 15:39:42 GMT</pubDate><content:encoded><![CDATA[<blockquote>
<p><strong>Disclosure:</strong> I am a frontend developer transitioning into AI engineering, sharing real experiments and learnings from building production-style RAG systems.</p>
</blockquote>
<p>Your RAG pipeline works perfectly on Friday. Then Monday hits. <strong>1,000 users query at once.</strong> Suddenly everything breaks: 502 errors, ECONNRESET, OpenAI 429 rate limits, Pinecone timeouts. The demo wasn't wrong—it just wasn't built for production concurrency.</p>
<p><strong>Video:</strong> <a href="https://youtu.be/-2aS3Yl5-5M">https://youtu.be/-2aS3Yl5-5M</a><br /><strong>Code:</strong> <a href="https://github.com/gauravthorath/rag-scale-demo">https://github.com/gauravthorath/rag-scale-demo</a><br /><strong>Full article:</strong> <a href="https://gauravthorat-portfolio.vercel.app/blog/rag-production-architecture">https://gauravthorat-portfolio.vercel.app/blog/rag-production-architecture</a></p>
<h2>The Monday morning problem</h2>
<p>Locally: chunk docs → embed → upsert to Pinecone → query → LLM. Simple.</p>
<p>Under load: socket exhaustion, connection pool saturation, API 429s, token costs exploding.</p>
<h2>Naive RAG (what most people build first)</h2>
<pre><code class="language-ts">for (let i = 0; i &lt; SAMPLE_CHUNKS.length; i++) {
  const values = await embedOne(openai, embedModel, SAMPLE_CHUNKS[i]);
  vectors.push({ id: `demo-naive-${i}`, values, metadata: { text } });
}

const pinecone = new Pinecone({ apiKey: pineconeKey });
for (const v of vectors) {
  await index.namespace(DEMO_NAMESPACE).upsert([v]);
}
</code></pre>
<p>Why it breaks at scale:</p>
<ul>
<li><p>One embedding call per chunk</p>
</li>
<li><p>One upsert per vector</p>
</li>
<li><p>No batching, no connection reuse, no retries</p>
</li>
<li><p>New client instances repeatedly</p>
</li>
</ul>
<p>3 chunks × 1,000 users × retries = thousands of outbound API calls. Sockets and rate limits run out fast.</p>
<h2>Production pattern</h2>
<p>Same RAG logic. Better infrastructure.</p>
<p><strong>Singleton Pinecone client:</strong></p>
<pre><code class="language-ts">let client: Pinecone | undefined;
let indexCache = new Map&lt;string, Index&gt;();

export const getPineconeIndex = (indexName?: string): Index =&gt; {
  const name = indexName ?? getEnv().PINECONE_INDEX_NAME;
  let idx = indexCache.get(name);
  if (!idx) {
    idx = getPineconeClient().index(name);
    indexCache.set(name, idx);
  }
  return idx;
};
</code></pre>
<p><strong>Embedding batching:</strong></p>
<pre><code class="language-ts">const res = await openai.embeddings.create({
  model: model,
  input: inputs,
});
</code></pre>
<p>64 texts → 1 API call instead of 64. Big win on latency, cost, and rate limits.</p>
<blockquote>
<p>In-process batching only. For multiple servers, add Redis caching and a task queue.</p>
</blockquote>
<h2>Naive vs production</h2>
<table>
<thead>
<tr>
<th>Naive</th>
<th>Production</th>
</tr>
</thead>
<tbody><tr>
<td>New Pinecone client per call</td>
<td>Singleton client</td>
</tr>
<tr>
<td>One embedding per chunk</td>
<td>Batched embeddings</td>
</tr>
<tr>
<td>One upsert per vector</td>
<td>Bulk upsert</td>
</tr>
<tr>
<td>Raw env vars</td>
<td>Zod validation</td>
</tr>
<tr>
<td>No retries</td>
<td>Backoff + retry</td>
</tr>
<tr>
<td>No metrics</td>
<td>Tracing + metrics</td>
</tr>
</tbody></table>
<h2>Before real scale</h2>
<ol>
<li><p>Exponential backoff + jitter on OpenAI and Pinecone</p>
</li>
<li><p>Top-K + reranking (don't dump every chunk into the prompt)</p>
</li>
<li><p>Distributed rate limiting across instances</p>
</li>
<li><p>Metrics: embed latency, retrieval quality, token usage</p>
</li>
<li><p>Stable vector IDs for safe retries</p>
</li>
</ol>
<h2>Try it</h2>
<pre><code class="language-bash">git clone https://github.com/gauravthorath/rag-scale-demo
cd rag-scale-demo
cp .env.example .env
npm install
npm run naive
npm run production
</code></pre>
<p>Use separate Pinecone namespaces so runs don't overwrite each other.</p>
<h2>Final thoughts</h2>
<p>Most RAG tutorials stop at "it answers my PDF." Production is about surviving concurrency, retries, rate limits, and cost pressure.</p>
<p>Questions or repo fixes? Drop a comment. I reply here and on YouTube.</p>
<p>Originally published on my portfolio: <a href="https://gauravthorat-portfolio.vercel.app/blog/rag-production-architecture">https://gauravthorat-portfolio.vercel.app/blog/rag-production-architecture</a></p>
]]></content:encoded></item></channel></rss>