Real-Time AI Chat: WebSockets vs Polling in Production

Here's an unfashionable production fact: we shipped an AI chat used daily by a professional team on SWR polling at 5-second intervals, and it ran for months before WebSockets earned a place on the roadmap. "Real-time" is a spectrum, and most teams pick their spot on it by fashion instead of by requirements. Here's how to pick it by requirements.

What AI Chat Actually Needs

AI chat is not multiplayer gaming. The latency profile is unusual: responses take 2–15 seconds to generate anyway, users are reading and thinking between messages, and message volume per conversation is low. A 5-second polling delay on top of an 8-second generation is barely perceptible. What users do notice: their own message appearing instantly (solved by optimistic UI, not transport), progress feedback during generation, and messages never getting lost.

The Honest Comparison

Making Polling Feel Real-Time

Polling gets an undeserved reputation because of naive implementations. The production version:

• Optimistic sends. The user's message renders immediately from local state and reconciles with the server on the next revalidation. This is the single biggest perceived-speed win — and it's transport-independent.
• Stable identity for reconciliation. Decide who assigns message IDs — client or server — and keep it consistent, or optimistic messages will duplicate or vanish on merge. We've been bitten by this exact bug in production; it presents as "disappearing messages" and it is always an ID reconciliation problem.
• Tiered intervals. 5s on the active thread, 15–30s for sidebar unread counts, 60s for analytics. Don't poll everything at chat speed.
• Pause on hidden tabs. SWR's focus revalidation gives you an instant refresh the moment the user returns.
• Poll fast during generation. While the agent is working, tighten the interval and render its progress (thinking steps, partial status) as it lands.

Rule of thumb: if your median agent response takes longer than your polling interval, transport is not your bottleneck. Fix the experience with optimistic UI and progress feedback first.

When WebSockets Become the Right Answer

1. Token streaming. If you want text appearing word by word, polling can't fake it. (Server-sent events are a lighter middle ground worth considering here.)
2. True multi-user liveness. Typing indicators, presence, several people in one thread reacting to each other in real time.
3. Scale economics. Thousands of concurrent users polling every 5s is a lot of empty 200s; connections get cheaper than requests.
4. Push-initiated interactions. Agent proactively interrupting with time-critical alerts where 5 seconds genuinely matters.

The Migration Path

The pragmatic sequence we follow: ship on SWR polling; add optimistic UI immediately; add SSE for token streaming when response quality demands it; adopt WebSockets only when presence features or scale force the issue — and even then, keep polling as the degraded-network fallback, because your reconnection edge cases will have bugs and the poll loop won't.

Transport is invisible when it works. Users judge your chat on whether their message appears instantly, whether they can see the agent working, and whether anything ever gets lost. Two of those three are solved above the transport layer — solve them first, then let real requirements, not architecture aesthetics, decide the socket question.