Why Conduktor Uses Redpanda for Local Kafka Development

What Each Product Does

Conduktor provides tooling for Kafka developers and infrastructure teams. Two products: a UI for developer productivity and Conduktor Gateway, a Kafka proxy for DevSecOps (encryption, multi-tenancy, failover, governance).

Redpanda is a Kafka-compatible streaming platform. Single binary. No ZooKeeper. Source available. Fast.

The Local Development Problem

Getting started with Kafka traditionally means running Kafka, ZooKeeper, and Schema Registry. Multiple containers. High memory usage. Slow startup. Race conditions during initialization.

Not a great first experience.

One Command to Start Everything

Redpanda solves this. Starting Conduktor with an embedded Kafka-compatible cluster:

curl -L https://releases.conduktor.io/quick-start -o docker-compose.yml && docker compose up

This docker-compose.yml starts Conduktor on http://localhost:8080 with Redpanda.

What this gives us:

• Single command startup
• Redpanda is 130MB (matters for users worldwide with varying bandwidth)
• Schema Registry is built into Redpanda, no separate container
• No startup synchronization issues between components
• Instant boot

To connect to your own Kafka cluster instead:

curl -L https://releases.conduktor.io/console -o docker-compose.yml && docker compose up

How Gateway Routes Kafka Traffic

Conduktor Gateway is a Kafka proxy. It listens on the Kafka protocol (my-kafka:9092).

It intercepts traffic from applications that think they're talking to Kafka directly. Gateway can accept, deny, or transform requests and responses. No code changes required in clients or brokers.

This provides proactive safeguarding, self-service with governance, and encryption without tech debt.

Gateway implements the full Kafka protocol: produce, fetch, consumer groups, transactions, metadata. Redpanda implements the same protocol. They work together.

Why Redpanda

Redpanda has:

• Small compute footprint
• Instant boot
• Built-in Schema Registry
• HTTP proxy included
• Performance autotuner
• No JVM, ZooKeeper, or KRaft dependencies

Available as a free Community Edition or managed Cloud.

Conduktor Works With Redpanda

Conduktor Console and Gateway are fully compatible with Redpanda. We use Redpanda in our internal CI/CD for the same reasons: fast startup, small footprint, no dependencies.

Multi-Cloud Routing With Gateway

Organizations often run multiple clouds and multiple Kafka providers. This creates networking complexity: VPCs, peering, BGP routing, DNS resolution. Debugging connection issues is painful.

Gateway is a Kafka proxy deployable anywhere. It bridges networks and routes traffic to the right cluster and topic. Configuration changes at runtime without impacting applications.

Think of it as nginx for the Kafka protocol.

Use Redpanda Cloud for streaming infrastructure. Use Gateway to connect from on-premises or other cloud providers.

Adding Redpanda to Existing Kafka Infrastructure

Gateway sits on top of your Kafka providers. Security, audit, governance, routing, and encryption are centralized at the Gateway layer.

This decouples your applications from specific Kafka providers. You can introduce Redpanda to part of your infrastructure through Gateway without changing application code.

Community

Join the Conduktor Community on Slack or the Redpanda Community on Slack.