Vault
  • Content
    • Videos
    • Bulk Upload
Sign In
  1. Video Catalog
  2. techwithprateek

techwithprateek

2 videos archived 1,332 total views

Back to Catalog
Everyone thinks LLM leaks are “model problems.” Actually: they’re architecture problems.

Here’s the framework I use in production
Access, Context, Output.

⚡ The Over-Entitled Retriever
Insight: Most leaks happen before generation  because your retriever sees too much

• Enforce row-level ACLs → filter before embedding search
• Partition vector indexes by tenant → zero cross-org bleed
• Sign queries with user identity → audit every retrieval

Result: 100% tenant isolation. Zero accidental cross-access

—

⚡ The Prompt Injection Trap
Insight: A single malicious sentence can override your system prompt.

“Ignore previous instructions…” → goodbye guardrails.

• Strip tool instructions from retrieved text → no tool hijacking
• Freeze system prompts server-side → never client-controlled
• Run injection classifier → block risky queries pre-generation

Payoff: 80% of jailbreak attempts stopped before inference.

—

⚡ The Memory Time Bomb
Insight: Long-term memory becomes long-term liability.

• Encrypt embeddings at rest → reduce blast radius
• Set TTL on conversation memory → auto-expire after 24h
• Disable training retention → no vendor data reuse

Outcome: Sensitive data lifespan drops from months to hours. 

—

⚡ The Output Spill
Insight: The model can echo secrets it shouldn’t.

Especially in summarization and Q&A.

• Add regex + NER redaction → mask PII before response
• Apply policy LLM pass → secondary compliance filter
• Log every response with hash → traceability under 200ms

Result: 90% fewer policy violations.

—

Secure LLM ≠ better prompts.
It’s layered defense.

Access → Context → Output.

🔖 Save this before your next security review
💬 Comment “SECURE” if you also building safe LLM Appa
➕ Follow for production-grade AI system design breakdowns
Everyone thinks LLM leaks are “model problems.” Actually: th...
246
Feb 25, 2026
Everyone thinks RAG fails because models hallucinate. Actually: your chunks are dumb.

If retrieval feeds garbage structure, generation can’t recover.

Three upgrades:

Semantic Chunking > Token Slicing
500-token splits ignore meaning boundaries.

→ Split by headings, sections, logical claims
→ Keep chunks 300–800 tokens max
→ Add 10–20% overlap for context continuity

Payoff: Retrieval relevance improves 30–50%.

Aha: Chunk size should match how humans think. Not tokenizer limits.

___

Connection-Aware Retrieval
Most teams store chunks like isolated PDFs.
But your data has relationships.

Policies reference sections.
APIs reference schemas.
Research cites experiments.

→ Store metadata: author, version, section, entity
→ Use hybrid search: BM25 + embeddings
→ Re-rank top 20 → send top 5

Payoff: Answer accuracy jumps 2×. Latency barely changes.

Aha: Retrieval isn’t about similarity. It’s about structure

___

The Knowledge Graph Layer
Flat vector stores miss cross-document reasoning.
Graphs preserve relationships.

Instead of “find similar text”
You ask: “What links A → B → C?”

→ Extract entities + relations during ingestion
→ Store triples alongside embeddings
→ Traverse graph, then retrieve supporting chunks

Payoff: Multi-hop questions improve 3×.

Think of it like this:
Vectors = fuzzy memory.
Graphs = connected memory.

Best systems use both.

Chunk smart.
Store relationships.
Retrieve with structure.

🔖 Save this for your next RAG architecture review
💬 Comment your struggles while building a RAG application 
➕ Follow for more production-grade AI system design 0:08
Everyone thinks RAG fails because models hallucinate. Actual...
1,332 59
Feb 26, 2026
Upload Video
Upload MP4, MOV, AVI, or MKV files (max 2GB)