Streams

A stream is a durable, ordered, replayable log of messages.

Use streams when messages must be stored and read later. A stream keeps messages according to its retention settings. Reading a message does not remove it.

How Streams Receive Messages

A stream is a destination. It receives messages through a binding to one or more topic filters.

When a producer publishes a message to a matching topic, FlowMQ appends a copy of the message to the stream.

Partitions

A stream can have one or more partitions.

Each partition is an append-only sequence of messages. FlowMQ appends each message to one partition in the stream.

Partitions allow a stream to scale reads and writes in parallel. Ordering is defined within a partition by append position; there is no single global order across all partitions in a stream.

Offsets

Each message in a stream partition has an offset. Offsets are scoped to a partition and are continuous within that partition.

Stream consumers track their position with offsets for each partition they read.

This allows a consumer to:

• Resume after reconnecting
• Read from the beginning
• Read only new messages
• Reprocess historical messages

Retention

Streams retain messages according to their retention settings.

A stream can be configured with a retention period. Messages older than the retention period are removed from the stream. Consumers can only replay messages that are still retained.

When to Use Streams

Use a stream for:

• Event history
• Analytics pipelines
• Audit logs
• IoT data retention
• Replayable consumption