Bridging: Edge to Cloud

This document outlines the design for FlowMQ's bridging functionality, which enables a lightweight FlowMQ broker deployed at the edge to establish a secure, reliable connection to a central FlowMQ project in the cloud. This feature extends the core messaging infrastructure to edge locations, facilitating real-time local communication while ensuring seamless data synchronization with the central cloud.

---

1. Goals

• Low-Latency Edge Communication: Allow devices at an edge location (e.g., a factory floor, a retail store) to communicate with each other with minimal latency, without needing a round trip to the cloud.
• Offline Resilience: The edge broker must continue to operate and buffer messages even if its connection to the cloud is temporarily lost.
• Centralized Data Aggregation: Provide a reliable mechanism to forward all, or a subset of, edge data to the central cloud for long-term storage, analytics, and global visibility.
• Reduced Bandwidth: Enable local processing and filtering at the edge so that only essential or aggregated data is transmitted to the cloud, saving on bandwidth costs.
• Simplified Management: The bridging configuration should be simple to set up and manage.

---

2. Architecture Overview

The bridging architecture involves two main components: one or more Edge Brokers and a central Cloud Project. The edge broker initiates a persistent connection to the cloud broker, acting as a specialized client.

• Edge Broker: A lightweight instance of FlowMQ running on-premises or at a remote location. It provides the same core features (topics, queues, streams) for local devices.
• Cloud Project: A standard, fully-managed FlowMQ project running in the cloud.
• Bridge Connection: A secure, outbound connection from the edge broker to the cloud project's endpoint. This connection is durable and will automatically attempt to reconnect if lost.

---

3. Key Design Considerations

3.1. Data Synchronization and Topic Mapping

The flow of messages across the bridge is controlled by topic mapping rules configured on the edge broker. This provides fine-grained control over what data is shared.

• Directional Mapping: Mappings can be configured in either direction:
  • Edge-to-Cloud: Messages published to a local topic prefix (e.g., local.telemetry.#) are forwarded to a specific topic in the cloud (e.g., edge.site-A.telemetry.#).
  • Cloud-to-Edge: Messages published to a cloud topic (e.g., cloud.commands.site-A) are sent across the bridge to a local topic on the edge broker (e.g., local.commands).
• Data Transformation: The bridge can be configured to add metadata to messages as they cross the bridge, such as an edge-site-id header, to enrich the data for cloud-based applications.

3.2. Security

Security is paramount. The bridge connection is secured using multiple layers:

• Authentication: The edge broker authenticates with the cloud project using client certificates or secure tokens with specific permissions.
• Encryption: All traffic over the bridge is encrypted using TLS 1.3, ensuring data confidentiality and integrity.
• Authorization: The credentials used by the bridge can be scoped with fine-grained permissions, limiting which topics it can read from or write to in the cloud.

3.3. Fault Tolerance: Store-and-Forward

The bridging mechanism is designed to be resilient to network instability.

• If the connection to the cloud is lost, the edge broker continues to accept messages from local producers.
• Messages destined for the cloud are spooled into a durable local queue or stream.
• When the connection is re-established, the edge broker automatically begins forwarding the buffered messages in the correct order, ensuring at-least-once delivery to the cloud and preventing data loss.

---

4. Practical Use Cases

• Smart Factories (IIoT): PLCs, sensors, and robotic arms communicate on the factory floor via the local edge broker with sub-millisecond latency. Aggregated production counts, fault alerts, and quality metrics are forwarded to the cloud for analysis by the engineering team. Cloud-based services can send new configuration or commands back down to the edge.
• Retail Chains: Point-of-Sale (POS) systems, inventory scanners, and local servers in each store communicate in real-time. Sales data is securely forwarded to a central cloud project for corporate accounting and supply chain management.
• Connected Vehicles: An in-vehicle FlowMQ broker manages communication between various electronic control units (ECUs). Critical events like airbag deployment or summary telematics data are forwarded to the cloud for analysis when the vehicle has a stable network connection.