1. We choose OpenSwoole. It is more performant then node.js, it uses PHP so we do not have to hire more devs and it supports both cooperative multi-threading (as node) as well as process forking. That means that we can squeeze out every resource from one instance until we need to spawn additional one. Major criteria was, of course, same tech stack - we can use existing workforce and knowledge base
There are several goals which we wanted to achieve:
1. Keep same tech stack, expand it as less as possible as pure necessity
2. Keep ALL business logic in one place
3. Adding new features MUST be equally simple as if we are working without real time updates
4. Security (of course)
5. Horizontal scaling
So, here is how we done it:
1. We choose OpenSwoole. It is more performant then node.js, it uses PHP so we do not have to hire more devs and it supports both cooperative multi-threading (as node) as well as process forking. That means that we can squeeze out every resource from one instance until we need to spawn additional one. Major criteria was, of course, same tech stack - we can use existing workforce and knowledge-base. Of course, cooperative multitasking/process forking is not something which is second nature of every PHP dev, however, our implementation abstracted that with event-based system, so additional features/maintenance should not be an issue.
2. Web socket server does not do anything except message routing. It will accept subscription request, check token validity, and if it is fine, it will update subscription ledger. When message is received from queue, it will route it to recipients (clients). NO ADDITIONAL LOGIC! WSS is stupid, it has one simple task, and it can execute it FAST! Actually, 4 :-) but you get the idea (subscribe, unsubscribe, routing, ping/pong).
3. All business logic is in server application. API to accept request (save, edit, delete), store data into DB, security, validation, etc... You would add all logic as usual, except, if you want realtime-update, you would push that message to the queue. Recipient is topic or user. We don't use topic, we use aggregate ID from DDD. Per example, if some media is commented, or comment is replied, ID of media is used as topic, since media is root aggregate.
To achieve this, we have "envelope" interface. Envelope has various data, such as topic, timestamp... and payload. Payload varies depending on feature/type of message. New feature will require new contract for payload, of course. However, WSS is not subject of change, only client and server - which added new feature.
4. Security is, again, on server side. If client wants to subscribe to some "topic", it will send request for subscription to API requesting token. If token is received, client will send "subscribe" to WSS. WSS will decode token and if valid, subscribe it to topic.
Make sure that each token has TTL encoded.
You can see here with ease that new topic does not require any change on WSS side. Only business logic needs to implement auth checker which will decide if user can subscribe to topic.
5. Presented arch is scalable, to some extend which we do not know since WSS with OpenSSL can handle millions of clients and RabbitMQ millions of messages. We will not do perf. tests to determine that since we don't have any need for it. I do not have capacity info, sorry. But intuitively, you can imagine where are bottlenecks and how those can be handled if such need arise.
---------
So, Client sends POST, PUT, etc. request to server side where is business logic (POST comment, PATCH reply, etc..). Server does business logic and if all fine, creates ENVELOPE and puts it into queue (fan-out, all WSS will receive same envelope). WSS will receive envelope and check against subscription pool. If there is recipient/client with suitable subscription, message is delivered, otherwise, just dropped.
Client will display some page for some root aggregate (blog post). It will be interested for all notification for that root aggregate. Client sends "getToken(rootId)" to server, server will check security and issue token. Client will send "subscribe(rootId, token)" to WSS and receive all envelopes with "rootId" as destination.
We have one more thing, we store every envelope, server side, to DB/log. Client can, in case of websocket connection issue, to do reconciliation of all lost messages during connection downtime via API endpoint (of course, security is considered here as well).