You want to build a live sports scoring app that supports 100K individual users by broadcasting separate WebSocket message to each user.
Is this a good idea? Let's dive into this:
The main issue is that a single server instance is trying to push the same message to 100K individual WebSocket connections - sequentially.
This will saturate the CPU, network buffers, and the event loop. It doesn't matter even if the payload is tiny.
We should ideally use "pub/sub wit horizontal fan-out". Instead of one server owning all connections, we should deploy multiple stateless WebSocket servers behind a load balancer. Ever server subscribes to a Redis pub/sub channel or a Kafka topic.
When a score event comes, we publish it "once" to Redis. Every server instance receives the message and forwards it "only" to the local clients it holds. This converts an O(n) broadcast into O(instances) with constant effort per server. This scales linearly as we add more instances.
For a purely server-to-server client feed like live scores, we should also consider replacing WebSocket with Server Sent Events (SSE) which is a simpler (unidirectional, automatic reconnection, native browser support), allows multiplexing and efficient use of a single TCP connection. This reduces overhead per client.
Finally, we must offload the heavy lifting to a managed real-time service if we don't want to build an operate an in-house fan-out layer such as Ably, Pusher, AppSync, etc.
This is the pragmatic choice when ops cost exceeds integration cost.
You're building a live sports score app. 100K users are watching same game. Broadcasting individual WebSocket messages to each user crashes your server.
How will you optimize?
[Asked at ESPN interview]