FlowMQ Architecture

FlowMQ separates message processing from durable state. Stateless brokers handle client connections, protocol processing, routing, and delivery, while metadata and persistent message data are stored in dedicated shared systems. This separation keeps the data plane easy to scale and makes storage capacity independent of broker capacity.

Architecture at a Glance

Stateless Broker Cluster

FlowMQ brokers form a cluster and provide the compute layer of the platform. Each broker can enable one protocol or multiple protocols, allowing a deployment to match its workload. For example, one node can serve MQTT and Kafka clients while another serves AMQP clients.

All protocol adapters use the same routing model. A message published through one protocol can therefore be consumed through another without an external bridge or synchronization pipeline.

Brokers do not own durable partitions or store persistent message data on local disks. A broker may keep short-lived connection context while serving a client, but all durable state lives outside the broker layer. As a result:

• Fast horizontal scaling: Add or remove brokers according to connection count and throughput without moving persistent data.
• Predictable recovery: A failed broker can be replaced without restoring local message data or performing a lengthy data rebalance.
• Flexible protocol capacity: Scale the broker nodes that serve a particular protocol, or run several protocols on the same nodes.

Metastore Cluster

The metastore cluster is the source of truth for FlowMQ metadata. It stores definitions and configuration for resources such as streams, queues, and access policies.

Because metadata is shared, any eligible broker can serve a request using the same logical configuration. Brokers remain interchangeable, while the metastore cluster provides a dedicated, highly available home for control information.

Metadata is kept separate from message payloads. This lets FlowMQ optimize metadata operations independently from high-volume message storage.

S3-Compatible Object Storage

Persistent message data, such as messages retained in streams, is stored in S3-compatible object storage rather than on broker-local disks.

This design provides several practical benefits:

• Approximately 10× lower storage cost: Compared with disk-based storage, object storage has a lower unit price, and its built-in durability eliminates the need for FlowMQ to maintain multiple full copies of the data on separate disks.
• Independent scaling: Broker compute and retained data grow according to different demands.
• High durability: FlowMQ benefits from the replication and durability guarantees of the underlying object storage service.
• Simpler capacity planning: Storage can expand without provisioning and rebalancing local broker disks.

Architectural Advantages

The combination of stateless compute and shared durable storage gives FlowMQ the following advantages:

• Cloud-native elasticity: Scale connection and routing capacity quickly, without coupling broker changes to data placement.
• High availability with shorter recovery paths: Broker failures do not require local data recovery, making replacement and self-healing faster and more predictable.
• One platform for multiple messaging patterns: MQTT, Kafka, and AMQP clients can use a unified platform for pub/sub, queues, and streams.
• Fewer integration components: Native cross-protocol routing removes many bridges, replication jobs, and the failure points they introduce.
• Lower total cost of ownership: Stateless brokers reduce capacity overprovisioning, while object storage lowers the cost of durable retention and a unified platform reduces operational overhead.

This architecture lets teams scale and operate a single messaging platform across IoT, event streaming, and enterprise messaging workloads while keeping compute, metadata, and persistent data independently manageable.