Real-Time AI Chat: WebSockets vs Polling in Production
We shipped a production AI chat on 5-second SWR polling before touching WebSockets. When polling is the right call, when it stops being enough, and how to migrate.
Ostap Kovalisko
Founder & AI Systems Architect
Updated 2026-10-19 · Sources and technical review
In this article · 7 min
Here's an unfashionable production fact: we shipped an AI chat used daily by a professional team on SWR polling at 5-second intervals, and it ran for months before WebSockets earned a place on the roadmap. "Real-time" is a spectrum, and most teams pick their spot on it by fashion instead of by requirements. Here's how to pick it by requirements.
What AI Chat Actually Needs
AI chat is not multiplayer gaming. The latency profile is unusual: responses take 2–15 seconds to generate anyway, users are reading and thinking between messages, and message volume per conversation is low. A 5-second polling delay on top of an 8-second generation is barely perceptible. What users do notice: their own message appearing instantly (solved by optimistic UI, not transport), progress feedback during generation, and messages never getting lost.
The Honest Comparison
| SWR polling (5s) | WebSockets | |
|---|---|---|
| Time to ship | Depends on existing infrastructure and reliability requirements | Depends on hosting, reconnection and replay requirements |
| Perceived latency | Up to one interval before the next scheduled poll, plus request and processing time; failures or throttling add delay | Push updates; SSE also supports token streaming |
| Failure behavior | A later successful poll can recover state; retries, outages and reconciliation still need handling | Reconnection logic, missed-message replay, heartbeats — all yours to build |
| Serverless / Next.js fit | Native | Needs a persistent server or a third-party layer |
| Server load | Requests × users × 12/min — fine at tens, real at thousands | Cheap per message, but connection state to manage |
| Typing indicators / presence | Awkward | Natural |
Making Polling Feel Real-Time
Polling gets an undeserved reputation because of naive implementations. The production version:
- Optimistic sends. The user's message renders immediately from local state and reconciles with the server on the next revalidation. This is the single biggest perceived-speed win — and it's transport-independent.
- Stable identity for reconciliation. Decide who assigns message IDs — client or server — and keep it consistent, or optimistic messages will duplicate or vanish on merge. We've been bitten by this exact bug in production; it presents as "disappearing messages" and ID reconciliation is one possible cause, alongside failed writes, stale responses and incorrect cache updates.
- Tiered intervals. 5s on the active thread, 15–30s for sidebar unread counts, 60s for analytics. Don't poll everything at chat speed.
- Pause on hidden tabs. SWR can revalidate on focus, subject to configuration, throttling and request latency.
- Poll fast during generation. While the agent is working, tighten the interval and render its progress (observed tool activity and partial status, rather than a claimed trace of private reasoning) as it lands.
Rule of thumb: if your median agent response takes longer than your polling interval, transport is not your bottleneck. Fix the experience with optimistic UI and progress feedback first.
When to Evaluate a Push Transport
Server-sent events support server-to-browser event streaming and can deliver generated text. WebSockets support bidirectional messaging. Neither guarantees instant delivery or durable messages.
- Token streaming. Use SSE, streamed HTTP responses, or WebSockets when incremental output matters. Polling can fetch partial output too, but the interval adds delay.
- True multi-user liveness. Typing indicators, presence, several people in one thread reacting to each other in real time.
- Scale economics. Thousands of concurrent users polling every 5s is a lot of empty 200s; connections get cheaper than requests.
- Push-initiated interactions. Agent proactively interrupting with time-critical alerts where 5 seconds genuinely matters.
The Migration Path
The pragmatic sequence we follow: ship on SWR polling; add optimistic UI immediately; add SSE for token streaming when response quality demands it; adopt WebSockets only when presence features or scale force the issue — and even then, keep polling as the degraded-network fallback, because your reconnection edge cases will have bugs and the polling fallback also needs failure and recovery tests.
Transport is invisible when it works. Users judge your chat on whether their message appears instantly, whether they can see the agent working, and whether anything ever gets lost. Two of those three are solved above the transport layer — solve them first, then let real requirements, not architecture aesthetics, decide the socket question.
Frequently asked questions
Should I start a new AI chat feature with polling or WebSockets?+
Start with polling if you can tolerate a few seconds of latency, because it is simpler to build, debug, and deploy. We shipped a production AI chat on 5-second SWR polling before adding any WebSocket infrastructure. Reach for WebSockets only when polling latency or server load becomes a real problem.
What does 5-second SWR polling actually mean in this context?+
SWR stands for stale-while-revalidate, a data-fetching pattern where the client shows cached data immediately and re-fetches in the background. Polling every 5 seconds means the client requests the latest chat state on a fixed 5-second interval. That interval is not a maximum latency guarantee: network time, failures and background-tab throttling can add delay.
When does polling stop being good enough for AI chat?+
Polling stops being enough when users need near-instant updates, such as token-by-token streaming of AI responses, or when the polling volume puts too much load on your servers. Fixed-interval requests also waste bandwidth when nothing has changed. Evaluate SSE for one-way streaming and WebSockets for bidirectional updates. Choose against latency, hosting and recovery requirements.
How hard is it to migrate from polling to WebSockets later?+
Migration is manageable if you isolate the data-fetching layer so the chat UI does not care whether updates arrive by poll or by push. You swap the transport underneath while keeping the same message-handling logic. Starting with polling does not lock you in, which is why shipping it first is a reasonable early decision.
Let's Talk About Your Project
Have questions about nearshoring or AI development? Our team is here to help you make the right decision.
- ✓Free consultation on your AI project
- ✓Custom cost estimates and timeline
- ✓Access to nearshore talent pools