Scaling In-Game Chat for Online Games: What Breaks First and How to Design for It

Your chat service holds 20,000 connected players in a soak test. Delivery latency is flat, CPU is boring, and every dashboard is green. Then launch night arrives.

Ten thousand players pile into the same event channel. A community post lands and thousands answer inside a few seconds. A network blip drops a chunk of the room, so those clients reconnect together, re-authenticate, rebuild subscriptions, and ask for missed history while the room is still busy.

Testing if chat can hold 20,000 users misses the true challenge. The real question is how much work a single second of player activity generates, which depends heavily on player grouping and communication patterns rather than online count alone.

Chat fails not from single-message costs, but because messages trigger cascading work across subsystems: connections, fan-out, persistence, moderation, rate limits, presence, and reconnects. Primary bottlenecks are fan-out, hot topics, reconnect churn, downstream tasks, and missing backpressure controls.

This guide reasons through the shape of your chat workload before launch, so the busy minute is something you designed for instead of something you discover live. If you are still deciding how parties, lobbies, sessions, DMs, and channels should fit together, start with Designing In-Game Chat for Online Games. This piece assumes the model already exists and focuses on what changes when it gets busy.

Chat Load Is Workload Shape, Not Player Count

CCU is useful, but it is a poor chat capacity model. Two games can have the same number of online players and produce completely different backend work. A useful way to think about it: chat load is roughly the product of a few independent factors, not a single count.

That's not a production formula. It's a way to see why one message to a four-player party and one message to a 20,000-player global room are completely different ops. The party message becomes three deliveries. The global message becomes up to 19,999 deliveries, plus whatever history, moderation, and notification work each of those triggers.

So a game with 50,000 players split into four-person parties and a game with 5,000 players sitting in one global channel are not comparable chat workloads, even though the first has ten times the CCU. The first is dominated by connection count. The second is dominated by fan-out. They will break in different places, and they need different designs. So, it's better to stop thinking in CCU and start thinking in five separate scaling dimensions:

• Connection scale: how many long-lived sockets you hold.

• Message scale: how many messages come in.

• Fan-out scale: how many deliveries those messages become.

• State scale: memberships, subscriptions, presence, history.

• Downstream scale: moderation, persistence, notifications, analytics per message.

Every section below is really about one or two of these.

Classify the Chat Surfaces Before You Size the System

Don't load-test "chat" as one generic feature. Party chat, session chat, guild chat, DMs, and a global event room often stress different parts of the system.

Treat those boxes as archetypes, not laws. A persistent DM with years of history can look more like the guild case than the party case. A 100-player match with constant join/leave churn may be more expensive than a quiet 500-player clan. The point is to identify the dominant cost for each surface instead of assigning one CCU assumption to all of them.

That classification should drive both architecture and test traffic. If you cannot say which surfaces are connection-bound, churn-bound, history-bound, or fan-out-bound, you do not yet know what "chat scales" means for your game.

Connections Are Cheap Until Everyone Reconnects

In-game chat almost always runs over long-lived WebSocket connections rather than request-response HTTP. That changes the whole cost model. Instead of:

request -> response -> connection ends

you get:

connect -> authenticate -> subscribe -> stay connected -> receive events continuously

Now the backend holds state for every connected player whether or not they're talking: connection identity, connection health, active subscriptions, session and topic membership, heartbeats, and cleanup on disconnect. A game with 100,000 logged-in players can be carrying 100,000 open sockets even if 95,000 of them haven't typed anything in ten minutes.

Idle users still consume connection infrastructure. That's the part teams underestimate. Each open connection ties up a file descriptor and socket buffers, sits in a connection table somewhere, and generates a steady trickle of keepalive traffic. The cost of "everyone is online and quiet" is not zero, and it sets a floor under your capacity planning that has nothing to do with message rate.

The things to reason about here are connection limits per node, file descriptor and socket resource ceilings, how connection state is distributed across your fleet, keepalive and heartbeat overhead, and how your gateway or load balancer spreads and pins connections. A single node can only hold so many sockets before memory and event-loop pressure degrade everyone on it.

This is not a hypothetical ceiling. When Helldivers 2 launched in February 2024, it hit a hard concurrent-player cap and Arrowhead raised it repeatedly over the following weeks, from 250,000 to 360,000 to 450,000 and eventually to 700,000, while players sat in queues and got kicked mid-session. That's a game-server and login story rather than a chat story specifically, but it's the same shape. Holding a very large number of simultaneous connections is a real engineering problem, and you can't always throw hardware at it on launch night.

Reconnect Storms Are Harder than Steady State

A backend that comfortably holds 50,000 stable connections can still fall over when 15,000 clients try to reconnect inside 20 seconds. Steady state and churn are different tests, and the second one is usually the one that bites.

The reason is that a reconnect is not free. Each returning client has to redo the expensive part of connecting all at once, at the same moment everyone else is doing it too.

Every reconnect can involve a TLS/WebSocket handshake, authentication or token checks, subscription restoration, presence updates, and a history delta. Multiply that by a service restart, a regional network interruption, a game update, or thousands of handhelds resuming at once and the recovery traffic can exceed the traffic that caused the original problem.

Anyone who plays MMOs knows the symptoms from the outside. Final Fantasy XIV players have spent real time staring at congestion and login-queue screens during expansion launches, which is exactly this problem at the identity and session layer: too many clients trying to establish state at once.

The practical rule is to test reconnect rate as a first-class metric, not just peak connection count. Design choices that help: stagger reconnects with jittered backoff so clients don't synchronize, cache enough client-side that a reconnect doesn't require a full history refetch, and make subscription restoration cheap.

Fan-Out Is the Multiplier that CCU Hides

This is the center of the whole topic. One player sends "gg". In a four-person party, the service does maybe three deliveries. In a 5,000-member channel, that same single incoming message becomes up to 4,999 deliveries.

Now multiply by message rate. Five hundred messages per second into a 5,000-recipient room is not a 500-per-second workload. It's potentially 2.5 million delivery operations per second in the worst case.

So the metric that matters is delivery fan-out, or recipient deliveries per second, not just messages per second. Graph recipient deliveries per second and deliveries per inbound message. If you only graph messages per second, a room can become expensive without the top-line metric looking dramatic.

The extreme version of this is familiar to anyone who's watched Twitch chat during a big stream: one shared room with thousands of people, where the message rate is high and every message theoretically wants to reach everyone. Nobody can actually read it, which is a hint about the design fix we'll get to shortly, and the delivery volume is brutal. That's the shape a naive "one global channel" reproduces.

Hot Topics Are Usually Worse than High Averages

Aggregate throughput can look healthy while one room is melting a partition. Ten thousand messages per second spread across thousands of small topics is not the same system as ten thousand messages per second with most of the traffic concentrated in one event room.

Watch traffic concentration: messages per second by topic, members per topic, deliveries per second by topic, and what share of total work the hottest one percent of topics carries. Averages hide the exact room you care about.

For genuinely large audiences, the right answer may be segmentation or sharding by region, mode, or cohort; a capped room population; a slower send budget; or a read-only announcement surface. None is free. Sharding fragments the community, caps change the social feel, and read-only events change the product. Large-room design is a gameplay/community decision with an infrastructure consequence.

The same logic applies to partitioning. Topic ID is a convenient key because membership, ordering, and fan-out line up around it, but it also means one enormous topic can become one enormous hot key. Choose the partition boundary knowing what it allows to fail together.

Rate Limits Protect Capacity, but They Are Only One Layer

Rate limiting belongs in the capacity conversation as much as the anti-spam conversation. A token bucket is useful because it allows a short human burst while putting a ceiling on sustained production.

For a concrete example, Chat service in AccelByte Gaming Services (AGS) uses a token-bucket rate limit with a default burst of up to 10 messages in a one second window, after which the sender waits for tokens to replenish before sending again. The default single-message character limit is 500. Those are starting points, not the universal truth or hard ceiling. We went deeper on picking these values in a separate piece on handling rate limits in a multiplayer backend, including why token bucket usually beats a fixed window.

The important distinction is scope. A connection-level or user-level limit stops one client from flooding. It does not protect a 5,000-player room when thousands of legitimate users all send once. If that room can threaten the service, you also need aggregate protection around whatever layer owns the shared capacity.

Spam control is separate again. Repeating the same message on a loop is a behavioral problem; a crowd of users sending different short messages at once is a throughput problem. AGS keeps those as separate knobs: its repeated-message spam control is distinct from the general chat rate limit. That is the right mental model even if you implement the controls differently.

Backpressure, Slow Clients, and Ordering

Every real-time system eventually reaches the same condition: producers can create work faster than consumers can finish it. If you do not define what happens next, your queue defines it for you by growing until memory, latency, or both become unacceptable.

Bound the queue and choose the failure behavior explicitly. You can throttle or reject producers, drop low-value ephemeral events such as typing indicators, shed non-critical enrichment, or disconnect a slow client whose outbound buffer has crossed a limit. The exact policy depends on the surface.

Be careful with the word "shed." Analytics, typing state, or nonessential history enrichment can be disposable under load. Safety checks may not be. If a moderation dependency is required to block delivery for your game, define a fail-closed or restricted mode instead of quietly bypassing it.

Slow clients need the same discipline. Reliable delivery does not mean holding an infinite per-connection buffer. Put a ceiling on outbound backlog, disconnect clients that cannot consume within it, and let a reconnect/history path catch them up.

Ordering is another place to buy only what the product needs. Global ordering is expensive and rarely useful for ordinary game chat. Per-topic ordering, or even per-sender ordering for some surfaces, is usually easier to reason about. The guarantee should come from the player experience, not from a distributed-systems reflex.

The Work Around the Message Can Cost More than the Message

Messages are only one stream. The auxiliary events around chat can quietly exceed the chat traffic itself.

Chat is surrounded by state: membership, presence, typing, reactions, history, unread state, moderation evidence, notifications, and analytics. At scale, those auxiliary events can dominate the payload players think of as "chat."

Membership churn is the obvious example in multiplayer games. Every match start and end can create or destroy a room and subscribe or unsubscribe a batch of players. Track subscriptions and unsubscriptions per second separately from message traffic.

A clean pattern is to derive session and party chat membership from the session system that already owns that state instead of maintaining a second membership model. AGS does this for session and party topics: players are subscribed when they join and removed when they leave. It prevents state drift, even though the subscription churn itself still needs capacity.

History changes the architecture too. The moment you persist messages for replay, moderation, search, or support, chat becomes a real-time system plus a storage system. During recovery, those two systems collide: 10,000 reconnecting users asking for the last 100 messages means a million historical message reads at the same time you are rebuilding connections.

Cap replay depth, paginate, fetch by cursor, and load history when the UI actually needs it. Treat reconnect replay as its own load-test scenario rather than assuming your normal history query numbers cover it.

The Quiet Amplifiers: Presence, Typing, and Membership Churn

Messages are only one stream. The auxiliary events around chat can quietly exceed the chat traffic itself.

Picture a 500-member guild room. If every connection event fans out a "Player X is online" to the other 499 members, a reconnect storm generates enormous event volume before anyone types a word. The same is true for typing indicators, read receipts, reactions, and membership changes. These feel small individually and add up to a firehose. The question to ask of each one is whether it really needs to fan out to everyone in real time, and the techniques when it doesn't are coalescing, debouncing, batching, suppressing them entirely above a certain room size, or making the state pull-based instead of pushing an event on every change.

Membership churn is its own traffic source, separate from messages. Players are constantly joining parties, leaving them, entering sessions, getting kicked, reconnecting, changing guilds, subscribing and unsubscribing. During matchmaking spikes or rapid match turnover, membership can churn massively even when nobody is chatting heavily, because every match that starts and ends creates and tears down rooms and their subscriptions.

There's a good architectural pattern here worth calling out: derive chat membership from session state instead of maintaining it independently. When a player's presence in a chat room follows automatically from their presence in a session or party, chat doesn't have to reinvent membership or risk drifting out of sync with the thing that actually owns it. That's how AGS models it, session and party chat rooms are represented as topics, and players are automatically subscribed when they join a session and removed when they leave, so membership has one source of truth. It's clean, and it still generates work: every join is a subscription, every leave an unsubscription. Track subscriptions and unsubscriptions per second as a real metric. Benchmarking messages alone misses an entire category of load.

History, Persistence, and Reconnect Replay

Whether chat is ephemeral or persistent changes the architecture. If chat is pure fire-and-forget:

send -> deliver -> discard

The storage problem barely exists. The moment the game needs history, unread state, reconnect replay, moderation evidence, search, or support investigation, each message can turn into a durable write. Now you're scaling write throughput, storage growth, indexes, retention, history queries, pagination, and deletion, on top of real-time delivery. The decision to persist messages turns a real-time delivery system into a real-time system plus a storage system, and those two halves may need to scale independently.

Reconnect replay is where persistence and reconnect storms collide into a second fan-out problem. Suppose 10,000 users reconnect after an outage and each asks for the last 100 messages. That's a million historical messages that reads landing at the exact moment the service is also rebuilding sockets and memberships. The recovery traffic is more expensive than steady state, and it arrives when the system is least able to absorb it.

The mitigations are all about not doing the expensive thing all at once: cap replay depth, paginate history, load it lazily only when the chat UI is actually opened, stagger reconnects, and lean on a client-side cache so a returning player fetches a small delta instead of a full backlog. The deeper questions of what to store, for how long, and why belong to retention and privacy, which is a topic of its own. For scaling purposes, the rule is that mass reconnect plus eager history replay is a recovery spike you have to design against.

Decide What Sits on the Delivery Path

Moderation, notifications, and other downstream work create a different scaling decision: which work must complete before a message is delivered, and which work can happen after?

Putting every moderation step inline gives you a simple guarantee, but it also couples chat latency and availability to the slowest safety dependency. Moving heavy classification or evidence work off the critical path keeps chat responsive, but creates an exposure window before post-delivery enforcement can act.

There is no universal answer. Cheap deterministic checks may belong inline; higher-cost classification, evidence capture, reporting, or enforcement may be asynchronous. Some products will intentionally block on more categories than others. The important thing is that the fallback behavior is explicit.

Apply the same rule to notifications and analytics. A chat send should not synchronously wait for every system that happens to care about the message. Fan secondary work out through queues or events where the semantics allow it.

And do not force system broadcasts through the same path as player conversation. "Servers restart in 15 minutes" is a broadcast problem, not a social-room problem. AGS keeps conversational Chat separate from System Transient and System Inbox Notifications, which is a useful design boundary even if your implementation uses different components.

Operate the Workload: Metrics, SLOs, and Graceful Degradation

A chat dashboard with only CCU and inbound messages per second is missing most of the failure surface. Keep the health model small enough to use during an incident, but make sure it spans the actual bottlenecks:

• Connections: active sockets, new connections per second, disconnects per second, reconnects per second, handshake and auth latency.

• Messages: inbound per second, delivered per second, fan-out per message, send-acknowledgement latency, and p50 / p95 / p99 delivery latency.

• Topics: active topics, members per topic, messages per topic, the hottest topic right now, subscriptions and unsubscriptions per second.

• Rate limiting: throttled messages, rate-limited users, rate-limited topics, spam mutes.

• Queues and backpressure: queue depth, processing lag, dropped messages, slow-consumer disconnects.

• History and safety: persistence latency, history query latency, replay requests per second, moderation latency and pipeline lag.

Set SLOs per surface. A tactical team message may need a much tighter latency target than a guild message. A global event room may tolerate throttling that would be unacceptable in party chat. One SLO for "chat" hides those differences.

Define graceful degradation the same way. Many games can drop typing indicators, reduce presence updates, defer history replay, or slow global chat before touching party/team communication. The exact order is game-specific; what matters is deciding it before the launch event.

Load-Test the Communication Pattern, Not the CCU

A 50,000-bot soak where every bot sends once every 30 seconds is a capacity test for that one pattern. It does not tell you what happens during the busy minute. Build test scenarios that deliberately stress the shapes you expect to hurt:

• Steady state: normal CCU, your real party and session mix, realistic message frequency.

• Launch burst: a rapid connection ramp with high global activity on top.

• Reconnect storm: thousands of connections returning at once, restoring subscriptions and history.

• Hot channel: thousands of users concentrated in one topic at a high message rate.

• Match turnover: mass session creation and destruction driving subscription churn.

• Abuse: users hitting rate limits, repeated messages, moderation and reporting load.

• Downstream slowdown: artificially slow persistence, moderation, or the notification consumer, and confirm chat itself stays healthy.

That last category is the difference between "how many users can we hold?" and "what does the system do when something is unhealthy?" Both matter. The second is usually what determines whether a production incident stays local or cascades.

If your team can't yet answer what the largest expected topic is, what the peak reconnect rate looks like, which channels can go hot, what happens when a queue fills, and whether global chat can degrade without touching party chat, then the honest status is that you don't know how your chat scales. Those answers are the deliverable.

A Reference Architecture for Chat at Scale

The whole system can be reduced to a few stages: hold connections, resolve membership, route the topic, validate and rate-limit the message, fan it out, then let recipients and downstream consumers do their work. The important part is seeing where one unit of input becomes many units of output.

Partitioning, ordering, observability, reconnect handling, and graceful degradation are not boxes you bolt on at the end. They are properties of the whole path. If your diagram cannot show where backpressure happens or how a hot topic is isolated, the implementation probably cannot either.

What This Looks Like in AGS Chat

Most of this piece is deliberately vendor-neutral, because the failure modes are universal and worth understanding whatever you build on. It's still useful to see the decisions made concretely in one place, so here's how AGS Chat lines up with the design points above, as one reference implementation rather than the only way to do it.

Chat runs over real-time WebSocket connections established after AGS Identity authenticates the player, which is the persistent-connection model from earlier. Chat rooms are modeled as topics, and for session and party chat those subscriptions are derived from session state automatically, players are subscribed when they join and removed when they leave, so membership has a single source of truth instead of a duplicated one. Throughput and abuse are separate controls: a token-bucket chat rate limit (default burst of 10 in a one-second window) for capacity, and a distinct repeated-message spam limit (default 5 identical messages in 30 seconds, then a temporary mute) for behavior. There's a default profanity filter that admins configure in the portal or override through AccelByte Extend when a game needs custom logic. And system messaging lives on its own path, with System Transient and System Inbox Notifications kept separate from conversational Chat so broadcasts don't ride the same infrastructure as player conversation. The full behavior and configuration are in the AGS Chat documentation.

None of that removes the work of classifying your surfaces, sizing your fan-out, and load-testing the busy minute. It means several of the structural decisions, deriving membership from sessions, splitting throughput from spam control, keeping broadcasts off the chat path, come already made in a defensible way, which is a reasonable thing to want when the alternative is discovering them during launch week.

If you're still designing the chat model itself, how channels, parties, lobbies, DMs, and message flow fit into identity and sessions before any of this scale pressure arrives, the companion architecture piece is the place to start.

The real takeaway is a reframe of the question you began with. It was never "can chat handle N users." It's "how much work does one busy second create, where does that work pile up first, and what have I already decided to shed when it does." Answer those three before launch, and the busy minute is a graph you watch instead of an incident you survive.

Design for the busy second, not the average one.