What We Shipped in 2023 and Why It Matters

2023 was a turning point for Conduktor. Here's what we built, what we learned, and where we're going.

From Desktop App to Platform Company

Conduktor started as a Kafka Desktop application for developers. Hundreds of thousands of developers used it to navigate their Kafka ecosystem.

But we had a bigger goal: accelerate real-time data adoption in organizations.

Platform teams need to establish security standards and rules. Developers need flexibility and independence. These two needs often conflict.

Developers focus on their immediate problems. Organizations have non-negotiable obligations: security, compliance, data management policies. Balancing both led us to build Console.

Console: A Platform for Developers and Operators

We introduced Console in October 2022 during our Milan offsite. It's a platform managed by organizations but designed for developers. Platform teams can organize, secure, and support developers without creating bottlenecks.

2023 was about getting Console to market and helping Desktop users migrate. We learned from a broader range of customers, broke into the enterprise market, and clarified our purpose.

The early days had hiccups. Customer feedback and fast iteration fixed them.

We noticed a trend: enterprises are moving away from many specialized products where only 20% of features get used. They want fewer, more versatile products covering multiple functionalities. This reduces costs and operational complexity.

Enterprise Features We Added in 2023

As we shifted toward platform teams and mission-critical systems, we had to meet enterprise standards:

• Enterprise Environment: LDAP, RBAC, Audit system, Public APIs for automation and GitOps, OpenShift support, third-party Service Accounts.
• SOC2 Type II: We achieved SOC2 Type II certification, proving our data security commitments. Read the full story.
• Solutions Engineering: We support customers closely to solve technical and organizational problems. We build confidence for production use and critical path deployment.
• Zendesk: We switched from Intercom to Zendesk. Better suited for enterprise use. Significant improvement in support quality.

Console Metrics from 2023

Console reached version 1.20 (changelog). Key improvements since launch:

• Docker size: 1.66GB to 800MB. Still optimizing.
• Vulnerabilities: From 1 Critical, 27 High, 14 Medium to 0 Critical, 0 High, 19 Medium.
• Onboarding: 5 screens to 1. Obvious in hindsight, hard in practice.
• Product Switcher: 8 separate features to 0. We unified the user experience.
• Topics: 1TB. The largest topic we saw in Console. Performance matters.

We also improved the user experience:

• Faster Rendering & Unified Navigation: More reactive interface, easier feature discovery.
• Richer Controls: More options to customize the interface.
• Performance: Users deal with enormous Kafka clusters. We optimized for smooth experience on clusters of any size.
• Deployment Simplification: Docker and Kubernetes setup became much easier. Our initial setup was resource-heavy and caused issues.

Console alone couldn't achieve our full mission. An interface has limits. To help platform teams enforce data security standards, we built something new.

Gateway: Control at the Data Plane

In mid-2022, we started R&D on a flexible Kafka Proxy called Gateway.

Gateway deploys as a service in your infrastructure and connects to your existing Kafka clusters. It solves problems that arise when organizations scale Kafka across many teams and domains. Companies can implement organization-wide constraints via the proxy.

Gateway lets Platform teams set and enforce rules on data going in and out of Kafka. It sits between applications and Kafka. We released it at Kafka Summit London 2023 (our review).

What Gateway does:

• Regulations and compliance: Encryption and data masking (language-independent), managed through policy repositories.
• Data Security: Granular RBAC on all Kafka operations and resources, with auditing.
• Federated Governance: Logical clustering, multi-tenancy, federation of multiple clusters as one.
• Safeguarding: Enforces technical and data rules so Kafka client configurations meet company standards.
• Plug-and-Play: Gateway uses "Interceptors" that any developer can build in Java to integrate custom logic directly into the proxy.

Tell us what you need: product@conduktor.io

We dogfooded Gateway at scale, hosting thousands of free multi-tenant Kafka clusters in our Cloud. We gave users a free Kafka sandbox so they could skip local Docker setup (and avoid common Kafka-on-Windows problems). This significantly improved Gateway's robustness.

Gateway and Console work together. Since Conduktor 1.19, Console acts as a configuration center for your Gateways. You can manage all interceptors, implement encryption, set up safeguards. More integrations are coming.

Where Conduktor Fits in the Ecosystem

We complement your existing Kafka infrastructure. We are not a Kafka infrastructure provider.

Whether you use AWS MSK, Confluent, Redpanda, Aiven, or another vendor, Conduktor fits on top.

Data streaming is the future of how organizations work with data. Companies must stay agile and responsive. Our mission: revolutionize how businesses interact with real-time data.

We give platform teams tools for implementing organizational standards. We give developers self-service and self-discovery. No more platform team bottleneck. Projects ship faster. Data stays secure.

If Kafka is a car's engine, Conduktor is the dashboard and gear shift: we help you get the most out of your engine.

Many open-source Kafka UI products exist. We're moving toward a more inclusive free model for basic needs. This sets teams on the path to success sooner, with flexibility to add security and governance at their own pace.

2024 Priorities

2023 was a breakthrough year. We learned from the market and positioned ourselves for enterprise challenges.

In 2024, we're doubling down on managing and securing data at scale. We're embracing Data Mesh principles and helping developers stay autonomous. Our goal: provide maximum value to your Kafka infrastructure by streamlining rules and processes.

What we build addresses challenges large organizations face daily with real-time data.

What We Discontinued

We also failed. 2023 required focus and efficiency for every company.

• Conduktor Cloud didn't work. No VPC peering, no local agent support, multi-tenancy only. Customers wanted on-premise over SaaS for security and privacy reasons.
• Conduktor Testing aimed to make Kafka easily testable with end-to-end testing and scenario generation. It didn't gain market traction, possibly due to inadequate advocacy or market readiness. We deprecated it despite customer support and engagement.