← Back to portfolio

Demo RAG V5 – Trusted Knowledge Service Case Study

A production retrieval-augmented generation service built for one job: put trusted, cited, source-grounded answers in front of the person who depends on them. Built for AI Empower (medical education) as the V5 Storage Decoupling release, with a live Streamlit demo on an asynchronous GCP fan-out architecture. The same pattern serves the operator in the field: SOPs, maintenance manuals, and policy.

Role Architect & Lead Engineer Domain Enterprise RAG / GCP Stack GCP · Firestore Vector Search · Vertex AI · Streamlit · Terraform

Overview

Demo RAG V5 is the live, demo-friendly face of an enterprise Retrieval-Augmented Generation service built for AI Empower, a medical education platform. Under the hood, it runs a fully asynchronous GCP fan-out pipeline that can ingest textbook-scale PDF/PPTX content, index it with a Parent–Child strategy in Firestore Vector Search, and serve conversational answers backed by Gemini 2.5 Pro.

Version 5 introduces a critical architectural shift: storage decoupling. The ingestion worker now performs a dual-write: persisting child vectors to Firestore for real-time search, while simultaneously archiving all RAG data to Parquet in Cloud Storage. This prepares the system for a V6 “Library Scale” upgrade where a dedicated vector engine (e.g., Vertex AI Vector Search) can be layered in without re-ingesting the corpus.

Users: medical students & faculty Mode: multi-tenant RAG backend + Streamlit demo UI Goal: accurate, explainable retrieval from large medical corpora

Problem Context

Earlier versions of the platform (V2/V3) proved out the core RAG idea but hit scaling and cost constraints:

  • Large files timed out. Monolithic ingestion functions couldn’t reliably process 500+ page textbooks without hitting time or memory limits.
  • Storage costs would explode at “library” scale. Storing every vector exclusively in Firestore would become cost-prohibitive as document counts grew into the hundreds of thousands.
  • Tenants needed isolation and memory. Each institution’s data had to remain isolated, and users expected follow-up questions to respect prior context.

The mandate for V5 was clear: keep the real-time experience of V4’s Firestore Vector Search, but quietly lay down the plumbing for a future storage and vector-engine flip without breaking clients.

Solution Overview

The Enterprise RAG backend now runs as three containerized Cloud Run services plus a Streamlit front-end:

  • An ingestion dispatcher that fans out uploads into per-page Pub/Sub tasks and scales horizontally.
  • A dual-write ingestion worker that performs Parent–Child chunking, generates embeddings with text-embedding-004, writes to Firestore for search, and archives to Parquet for future vector indexing.
  • A retrieval API that runs RAG with multi-tenant filters, conversational memory, and Gemini 2.5 Pro for answer generation.
  • A Streamlit demo app that exposes file upload, tenant selection, and a chat interface, wired into the same backend used in production.
Design intent: V5 is intentionally conservative at the surface (same Firestore-based semantics as V4) but aggressive behind the scenes, decoupling storage to unlock future vector search engines without a painful migration.

Architecture

At a high level, Demo RAG V5 implements a serverless fan-out ingestion pipeline, a Firestore-backed vector store with Parent–Child indexing, and a conversation-aware retrieval API, all fronted by a multi-tenant Streamlit UI.

Demo RAG V5 architecture diagram
Demo RAG V5 architecture: GCS uploads → Eventarc → Dispatcher → Pub/Sub → Dual-Write Worker → Firestore Vector Search (real-time) + GCS Parquet archive (future vector engine) → Retrieval API → Gemini 2.5 Pro → Streamlit demo.

Key Architecture Responsibilities

  • Upload & Dispatch: Files are uploaded to GCS under a tenant-scoped path (e.g., uploads/{client_id}/...); Eventarc triggers the dispatcher, which inspects page counts and publishes one Pub/Sub message per page.
  • Parent–Child Indexing: Workers split text into large “parent” chunks (~2000 chars) for LLM context and smaller “child” chunks (~400 chars) for vector similarity search.
  • Vector Storage: Child chunks are embedded with text-embedding-004 and stored in Firestore as vector fields, filtered by client_id at query time.
  • Storage Decoupling (New in V5): The same worker writes a Parquet row per child chunk into GCS, capturing embedding arrays, source, page, and timestamps for future batch indexing jobs.
  • Retrieval & LLM Orchestration: The retrieval API runs a vector search on child chunks, resolves the parents, merges that with the last four turns of chat history, and passes the full context into Gemini 2.5 Pro.

Demo Experience

The Demo RAG V5 Streamlit app is deliberately uncluttered: upload content, select a tenant context, and start asking questions. Behind each query is the same RAG pipeline that runs in the backend.

Demo RAG V5 Streamlit application screenshot showing chat experience and document context
Demo RAG V5 Streamlit front-end: tenant-aware document upload and a conversational RAG interface backed by Firestore Vector Search, chat history, and Gemini 2.5 Pro.

Typical User Flow

  • Institution uploads a medical textbook or lecture deck into their tenant path.
  • The fan-out pipeline ingests and indexes the content asynchronously.
  • A student opens the demo, selects the same tenant, and asks clinical questions.
  • The system returns grounded answers with source snippets, leveraging both the Parent–Child hierarchy and recent chat history to keep the conversation coherent.

Implementation Highlights

1. Asynchronous Fan-Out Ingestion

  • GCS upload events are translated into a batch of Pub/Sub messages, one per page, to avoid large single-transaction jobs.
  • Cloud Run workers scale horizontally with traffic, keeping ingestion latency bounded even for multi-hundred-page documents.

2. Parent–Child Indexing Strategy

  • Parent chunks provide the semantic “canvas” for Gemini, ensuring that responses are grounded in full paragraphs rather than isolated sentences.
  • Child chunks are optimized for vector search accuracy, balancing recall with storage footprint.

3. Storage Decoupling via Dual-Write Worker (V5)

  • The worker’s main loop now accumulates Parent/Child metadata and embeddings into a DataFrame, writes them to Firestore in a batch, and then exports the same rows to GCS as Parquet.
  • This enables a smooth V6 migration path where an offline job can build a Vertex AI Vector index from Parquet without re-processing the raw documents.

4. Retrieval API with Memory

  • Each session is keyed by session_id and persists the last four turns of user/assistant messages in Firestore for conversational continuity.
  • Vector search is always pre-filtered by client_id, guaranteeing tenant isolation and predictable performance.

5. Streamlit Demo Layer

  • The demo uses the same retrieval API and authentication model that a production application would use, making it a realistic sandbox for stakeholders.
  • This front-end also doubles as a test harness for validating ingestion, retrieval, and relevance before shipping additional features.

Impact & What I’d Do Next

By V5, the AI Empower Enterprise RAG Service has moved from a robust prototype into a platform with serious headroom:

  • Scalability: Ingestion is no longer a bottleneck for long textbooks or large tenant libraries.
  • Cost Control: Firestore remains the real-time engine, but Parquet archives ensure future vector storage can be optimized without data loss.
  • Product Readiness: The demo app provides a concrete, shareable interface that non-technical stakeholders can use to validate the system’s value.

If I extended this further into a V6 release, the next steps would be:

  • Stand up a Vertex AI Vector Search index and a batch “index builder” job that reads from the Parquet archive.
  • Add evaluation tooling (e.g., RAG evaluation harness with labeled queries) to measure retrieval quality across versions.
  • Extend beyond text to support multimodal content (slides + video transcripts) while keeping the same storage and indexing pattern.

Assets