FAQ
Here are some frequently asked questions about FlowMQ.
General
Q: What is FlowMQ?
A: FlowMQ is a modern, high-performance message broker designed for building scalable and reliable distributed systems. It provides flexible messaging patterns like queuing for work distribution and streaming for event logs, making it suitable for a wide range of use cases from microservice communication to real-time data pipelines.
Q: How does FlowMQ compare to Kafka or RabbitMQ?
A: FlowMQ combines concepts from both log-based systems (like Kafka) and traditional message queues (like RabbitMQ) into a single, cohesive platform.
- Like Kafka, it offers persistent, replayable streams with consumer-managed offsets, ideal for event sourcing and stream processing.
- Like RabbitMQ, it provides robust server-acknowledged queues for reliable task distribution and work offloading. FlowMQ aims to provide the benefits of both models with a simpler operational model and a unified client API.
Concepts
Q: What is the difference between a Topic, a Queue, and a Stream?
A:
- Topic: A named channel to which producers publish messages. Topics are used for routing. You don't consume from a topic directly. Instead, FlowMQ routes messages from topics to queues or streams based on rules you configure in the broker.
- Queue: A destination for messages that are processed by one of many competing consumers. Each message is delivered to only one consumer in the group. Once a consumer processes and acknowledges a message, it is permanently removed from the queue. This pattern is ideal for distributing tasks.
- Stream: A persistent, ordered log of messages. Multiple consumers (or consumer groups) can read from the same stream independently, tracking their own position using an offset. Messages are not removed when read; they expire based on retention policies. This is ideal for pub/sub, event sourcing, and replayable event histories.
Q: When should I use a Queue vs. a Stream?
A:
- Use a Queue when you want to distribute a set of tasks among a pool of workers and ensure each task is processed exactly once by one worker (e.g., a request processing service, an image resizing pipeline).
- Use a Stream when you need to broadcast a message to multiple independent consumers, or when you need a persistent, replayable history of events (e.g., user activity tracking, financial tickers, system audit logs).
Q: What happens if my consumer fails to ack() a message from a Queue?
A: If a consumer receives a message but crashes or disconnects before acknowledging it, FlowMQ will not consider the message processed. After a configurable timeout, FlowMQ will redeliver the message to another available consumer on that queue. This ensures that messages are not lost if a worker fails.
Q: What are message durability guarantees?
A: Messages published to FlowMQ are persisted to disk on the broker before a publish receipt is sent back to the producer. This ensures that even if the broker restarts, messages are not lost.
Client & SDK Usage
Q: Is the FlowClient thread-safe? Should I create a new one for each task?
A: Yes, the FlowClient is designed to be thread-safe and is a heavyweight object that manages underlying network connections. You should create a single client instance when your application starts up and share it across threads/goroutines for the lifetime of your application. Creating a new client for each publish or consume operation is an anti-pattern that will lead to poor performance.
Q: Does the client automatically reconnect if the connection to the broker is lost?
A: Yes, the FlowMQ SDKs are designed with resilience in mind. If the client loses its connection to the broker, it will automatically attempt to reconnect in the background using an exponential backoff strategy. While disconnected, publish calls will fail or buffer (depending on configuration), and consumers will pause. Once reconnected, operations will resume automatically.
Q: What ordering guarantees does FlowMQ provide?
A:
- Streams: FlowMQ guarantees that messages within a single stream partition are stored and delivered in the order they were published. Ordering across different partitions is not guaranteed.
- Queues: By nature of competing consumers, queues do not provide strict FIFO ordering guarantees across the entire queue. If you require strict ordering, you should use a stream with a single partition or design your message consumers to handle out-of-order data.
Security
Q: How do I secure my connection to the FlowMQ broker?
A: Connections to FlowMQ can be secured in two primary ways:
- Authentication: The SDKs support token-based authentication to identify and authorize clients.
- Encryption: You can configure TLS on the FlowMQ broker to encrypt all network traffic between clients and the broker, preventing eavesdropping. The client SDKs provide options to connect to a TLS-enabled endpoint.