Design a chat app (1:1 and group messaging)

1. Clarify the requirements

Ask: 1:1 only or groups too, and how big can groups get? Text only or media? Do we need read receipts, typing indicators, online presence? Multi-device? End-to-end encryption? Message history kept forever?

Functional requirements

• 1:1 and group chat (up to, say, 500 members).
• Delivery while online; offline users get messages when they reconnect, plus a push notification.
• Sent / delivered / read receipts; online presence and last seen.
• Message history synced across a user's devices.

Non-functional requirements

• Low latency: under about 200 ms end-to-end when both users are online.
• No lost messages once the server has acknowledged them; per-conversation ordering.
• High availability and graceful reconnects on flaky mobile networks.

2. Estimates

Two conclusions: you need a large, horizontally scaled connection tier with a way to find which server holds a given user's connection, and message storage must be append-optimised and partitioned.

3. High-level architecture

Connections. Each client keeps one WebSocket to a chat server. Chat servers are stateful (they hold connections), so put a layer-4 load balancer in front and register each connection in a session registry (user_id/device_id → chat server) with a TTL refreshed by heartbeats.

Sending. The sender's chat server assigns a per-conversation sequence number, persists the message, and acknowledges the sender ("sent", one tick). It then looks up each recipient device in the registry and forwards the message to the right chat server, which pushes it down the socket. When the recipient's client acknowledges, a "delivered" receipt flows back the same way.

Offline. If the recipient has no live connection, hand off to the push notification service. On reconnect, the client asks for everything after the last sequence number it has for each conversation.

4. Data model

messages            PARTITION KEY conversation_id
                    CLUSTERING KEY seq DESC
                    sender_id, body (or ciphertext), media_ref, client_msg_id, created_at
conversations       conversation_id, type (direct | group), member_ids, created_at
user_conversations  user_id → [(conversation_id, last_read_seq, muted)]   -- inbox
receipts            conversation_id, user_id, delivered_seq, read_seq

Storing receipts as "highest sequence delivered/read per user" instead of one row per message keeps receipts cheap even in large groups. The client_msg_id generated on the device lets the server de-duplicate retries, which makes sending idempotent.

5. Ordering, delivery guarantees, and multi-device

• At-least-once delivery, idempotent receipt. Clients retry until they get a server ack; the server de-duplicates by client_msg_id. Recipients de-duplicate by seq. Together that behaves like exactly-once for the user.
• Per-conversation ordering. Route all writes for a conversation through one owner (consistent hashing on conversation_id) to assign sequence numbers without coordination.
• Multi-device sync. Treat each device as a separate recipient in the registry, and let each device track its own last-synced sequence per conversation.

6. Groups and scaling

• Small groups (up to a few hundred): fan out on write, sending one copy per online member device through their chat servers. The message is stored once.
• Very large channels (Slack/Discord scale): don't push each message to every member. Push a lightweight "channel updated" signal and let clients pull, or fan out only to members currently viewing the channel.
• Presence is the sneaky hot spot: broadcasting every online/offline change to every contact is expensive. Send presence only for conversations currently on screen, debounce flapping connections, and accept that "last seen" is approximate.
• Media uploads go directly to object storage with a pre-signed URL; the message carries only a reference and a thumbnail.
• Chat server failure drops its connections; clients reconnect (with jittered backoff to avoid a thundering herd) to another server and resync from their last sequence numbers. Nothing is lost because messages were persisted before delivery.

Monitoring and failure modes

Track connected sockets per server, message send-to-deliver latency (p50/p99), the rate of messages waiting for offline delivery, and reconnect storms. The dangerous failure is the session registry: if it's unavailable, servers can't find recipients. Keep it replicated, let chat servers fall back to "store and notify" when a lookup fails, and rely on clients' resync-on-reconnect to fill any gaps. Because every message is persisted before delivery, a routing failure delays messages but never loses them.

7. Trade-offs to call out

• Persist-then-deliver vs deliver-then-persist: persisting first adds a few milliseconds but guarantees no acknowledged message is lost.
• End-to-end encryption means the server stores ciphertext only, so server-side search, moderation, and multi-device history become client problems (each device has its own keys; the sender encrypts per device).
• Push vs pull for big groups, and where you draw the line.
• Retention: keeping history forever on servers versus device-only storage (WhatsApp's original model) changes your storage bill by orders of magnitude.

8. Common follow-up questions

1. How do typing indicators work? Ephemeral events sent over the socket and never stored; drop them under load.
2. How do you find a user's chat server quickly? The session registry is an in-memory store sharded by user ID; cache lookups briefly on chat servers for active conversations.
3. How do you support search? Without E2EE, index messages asynchronously into a search cluster partitioned by user; with E2EE, search on the device.
4. How do you deploy chat servers without dropping everyone? Drain servers gradually, telling clients to reconnect elsewhere, and spread reconnects over minutes.
5. How do you deliver to a user in another region? Route between regions over a message bus; store the conversation in a home region and replicate.

The offline path relies on a notification system, which is often the next question in the same interview loop.