Getting Started with Kafka

Prerequisites

• Basic understanding of Apache Kafka
• Apache Kafka CLI tools installed

Step 1: Sign up

If you haven't signed up yet, you can sign up for free here: https://flowmq.io/auth/sign-up

Sign up with your email, GitHub, or Google account.

Step 2: Create a Project

Once logged in, you'll need to create your first project for free:

1. Click Create your first project on the dashboard
2. Enter a project name (e.g., "my-kafka-app")
3. Choose your preferred cloud platform and region
4. Click Create

Your project will be provisioned within a few seconds. Once ready, you'll see your project dashboard with connection details.

Step 3: Set up Authentication Credentials

NOTE

FlowMQ Platform uses TLS and secure authentication for all connections.

To set up your credentials:

1. In your project dashboard, navigate to Authentication
2. Click Add Credential
3. Enter Username and Password
4. Click Create Credential

IMPORTANT

Save your password. The password will not be shown again for security reasons.

Step 4: Create a Stream

In FlowMQ, a stream stores messages as an append-only log. For users familiar with Kafka, a stream is functionally equivalent to a Kafka topic.

Before producing and consuming messages from a stream, let's create one first:

1. In your project dashboard, go to Streams
2. Click Create Stream
3. Enter Stream Name: my-kafka-stream
4. Click Create Stream

Step 5: Produce Messages to Stream

Let's use the Kafka console producer to send messages to your FlowMQ project.

IMPORTANT

Must use TLS and SASL authentication to connect to your FlowMQ project.

First, get your project's connection details from the Overview section in your project dashboard. Then, prepare your credentials created in Set up Authentication Credentials.

Create a client.properties file with your connection details:

bootstrap.servers=your-project-id.flowmq.cloud:9093
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="your-username" password="your-password";

Produce messages using Kafka console producer:

kafka-console-producer.sh \
  --bootstrap-server your-project-id.flowmq.cloud:9093 \
  --topic my-kafka-stream \
  --producer.config client.properties

Type your messages and press Enter to send them:

Hello, FlowMQ Kafka!
This is my first Kafka message

Step 6: Consume Messages from Stream

Start another terminal window and consume messages using Kafka console consumer:

kafka-console-consumer.sh \
  --bootstrap-server your-project-id.flowmq.cloud:9093 \
  --topic my-kafka-stream \
  --from-beginning \
  --consumer.config client.properties

You should see all the messages you produced earlier.

Step 7: Receive Kafka Messages Using MQTT Client

IMPORTANT

FlowMQ is not just another Kafka or MQTT broker; its unified architecture enables seamless interoperability, allowing Kafka clients to publish messages that MQTT clients can receive, and vice versa.

Try starting a MQTT subsciber subscribing to my-kafka-stream using MQTTX CLI:

mqttx sub \
  -h your-project-id.flowmq.cloud \
  -p 8883 \
  -l mqtts \
  -t my-kafka-stream \
  --username your-username \
  --password your-password

Reproduce some Kafka messages like in Produce Messages to Stream, your MQTT subscriber above should print all messages your Kafka client produced.

Next Steps

🎉 Congratulations! You've successfully:

• ✅ Created a FlowMQ project
• ✅ Set up secure authentication credentials
• ✅ Produced and consumed Kafka messages using Kafka CLI
• ✅ Experienced unified MQTT-Kafka messaging

What's Next?

• MQTT Integration: Check out Getting Started with MQTT to see how MQTT and Kafka work together
• Explore FlowMQ Features: Learn about Unified Topics, Streaming, and Pub/Sub
• SDK Documentation: Explore language-specific guides in our Kafka SDK and MQTT SDK sections

Happy streaming with FlowMQ!