← Back to ChatBridge

Architecture

ChatBridge is a K-12 AI chat platform where a GPT-4o-powered assistant orchestrates five embedded educational apps via function calling. This page covers the system architecture, data flows, and key design decisions.

System Overview

┌─────────────────────────────────────────────────────────┐
│  Browser (Vite SPA)                                     │
│                                                         │
│  ┌──────────────┐    PostMessage    ┌────────────────┐  │
│  │  Chat Panel   │◄───(CHATBRIDGE───►│  App Panel     │  │
│  │  (35%)        │     _V1)         │  (65%)         │  │
│  │               │                  │                │  │
│  │  React +      │                  │  ┌──────────┐  │  │
│  │  Mantine UI   │                  │  │ iframe   │  │  │
│  │               │                  │  │ sandbox  │  │  │
│  │  useToolExec  │                  │  │ allow-   │  │  │
│  │  state machine│                  │  │ scripts  │  │  │
│  └──────┬───────┘                  │  └──────────┘  │  │
│         │                           │                │  │
│         │  SSE stream               │  CV Safety     │  │
│         │                           │  (Web Worker)  │  │
└─────────┼───────────────────────────┼────────────────┘  │
          │                           │                    │
          v                           v                    │
┌─────────────────────┐   ┌────────────────────┐          │
│  Express API        │   │  NSFWJS + TF.js    │          │
│  (Railway)          │   │  (on-device)        │          │
│                     │   └────────┬───────────┘          │
│  /api/chat (SSE)    │            │ flagged only          │
│  /api/nature/*      │            v                       │
│  /api/spotify/*     │   ┌────────────────────┐          │
│  /api/oauth/*       │   │  OpenAI Moderation  │          │
│  /api/moderation    │   │  (server-side)      │          │
│                     │   └────────────────────┘          │
│  GPT-4o (OpenAI)    │                                    │
│  Clerk Auth (JWT)   │                                    │
│  PostgreSQL         │                                    │
└─────────────────────┘                                    │
                                                           │
  External APIs: iNaturalist, Perenual, Spotify            │
└──────────────────────────────────────────────────────────┘

Frontend Architecture

The SPA is a Chatbox fork rebuilt with a split-panel layout: 35% chat, 65% app panel.

  • ChatBridgeApp: Root component handling tool dispatch, layout management, and app lifecycle
  • useToolExecution: State machine (Map-based) tracking concurrent tool calls with request IDs, timeouts, and result routing
  • IframeManager: Renders sandboxed iframes with allow-scripts, credentialless, and no-referrer policies
  • PostMessageBroker: Versioned message protocol (CHATBRIDGE_V1) with MessageChannel isolation for request/response flows
  • Content Safety Monitor: Web Worker running NSFWJS with hysteresis state machine for blur/block overlays

AI Orchestration Layer

GPT-4o with OpenAI function calling powers all app interactions. The system prompt dynamically includes available tools based on the currently active app.

Student: "Show me a red panda"
    │
    v
[GPT-4o] → tool_call: launch_app("nature-explorer")
    │
    v
[Frontend] → PostMessage → iframe loads Nature Explorer
    │
    v
[GPT-4o] → tool_call: search_species("red panda")
    │
    v
[PostMessage] → bridge.js → GET /api/nature/search?q=red+panda
    │
    v
[Nature Explorer renders results]
    │
    v
[GPT-4o] narrates: "Here's the red panda! It's actually
 more closely related to raccoons than giant pandas..."

Tool Flow:
  1. launch_app (opens iframe)     ─── always first
  2. App-specific tools            ─── search, play, etc.
  3. Tool result → PostMessage     ─── back to parent
  4. Parent → SSE → GPT-4o        ─── AI narrates result

Tool Registration

Each app declares its tools in the server-side LLM service. Tools use OpenAI's strict mode with full JSON Schema validation — no freeform parameters. The system prompt includes a description_for_model for each app so the AI knows when to suggest which app.

Backend Architecture

Express server on Railway with SSE streaming for chat responses.

  • Chat endpoint: /api/chat — SSE stream with progressive token delivery, tool call extraction, and history trimming at 8K token budget
  • Nature API: 7 endpoints proxying iNaturalist + Perenual with blocklist filtering, license validation, and 8s request timeouts
  • Spotify proxy: OAuth token exchange, search, recommendations, and playlist management — Spotify secret never exposed to client
  • Moderation API: Server-side OpenAI moderation for the CV safety pipeline's secondary check
  • Auth middleware: Clerk JWT verification on all chat routes; rate limiting on all routes with stricter limits on chat/moderation

Embedded App Architecture

Each app follows the same pattern: a standalone HTML/JS app with a bridge.js that implements the ChatBridge PostMessage protocol.

App Structure (each app):
  index.html   ─── entry point, loads bridge + app
  bridge.js    ─── PostMessage protocol handler
                   ChatBridge.on('toolInvoke', handler)
                   ChatBridge.respondToTool(id, data)
                   ChatBridge.sendState(state)
                   ChatBridge.resize(height)
  app.js       ─── application logic + UI rendering
  styles.css   ─── app-specific styles (dark theme support)
  engine.js    ─── game logic (chess/go only)

Apps:
  Nature Explorer ─── iNaturalist biodiversity browser
  Chess           ─── chess.js rules + minimax AI (depth 2)
  Go              ─── Custom rules engine + greedy AI
  DOS Arcade      ─── js-dos v8 emulator, 17 curated titles
  Spotify         ─── OAuth-authenticated music search

Data Flow & Privacy

  • Chat history: Browser memory only — no server-side persistence of conversations
  • App state: localStorage per-app (chess/go saves), capped at 512KB, never transmitted
  • API keys: All keys server-side only (OpenAI, Spotify, Perenual); never exposed in client bundle or API responses
  • CV pipeline images: Processed entirely on-device via Web Worker; only flagged content descriptions (not images) reach the moderation API
  • Logging: Pseudonymized with generated names; no PII in server logs

Deployment

Split deployment optimized for cost and performance:

  • Frontend: Vite SPA on Vercel (static hosting + CDN)
  • Backend: Express on Railway (auto-deploy from GitHub, always-on)
  • DNS: Cloudflare managing chatbridge.aaroncarney.me
  • Database: PostgreSQL on Railway (Clerk session data only)
  • ML model: NSFWJS weights bundled as static assets, served from Vercel CDN

Technology Stack

React 18TypeScriptViteMantine UIExpressGPT-4oOpenAI Function CallingClerk AuthPostgreSQLTensorFlow.jsNSFWJSchess.jsjs-dos v8Spotify Web APIiNaturalist APIVercelRailway