What KPD does

The design and development journey

Most tools help with one step. KPD carries the same design from a whiteboard conversation to code you can apply, so nothing is retyped and nothing drifts.

1
Shape

Agree the business picture before anyone draws a topic.
- Domain map
- Data products
- DPCF template
2
Design

Draw the pipeline with real Confluent Cloud parts.
- Canvas
- Quick add
- Workshop mode
3
Govern

Schemas, contracts, naming and tags on every topic.
- Schemas
- Data contracts
- Naming standard
4
Validate

Problems show as you work, not at deploy time.
- Live checks
- Validate project
- Terraform go / no-go
5
Hand over

The picture, the document and the code, from one design.
- PNG
- Design document
- Terraform

Design-time capabilities

A designer built for the way pipelines are actually agreed: in workshops, against standards, with decisions written down.

A canvas that knows Kafka

Drag topics, connectors, Flink statements and clients on, wire them up, and watch data flow along the connections. Groups, notes and shading keep big designs readable.

Quick add and find

Press / and type what you want, or use the + beside an output for what usually comes next. Find in project searches every asset, with filters, and highlights matches on the canvas.

Workshop mode

Design in front of a client: the chrome disappears, problem badges hide, and Follow the data walks one record through the pipeline with a caption at each hop.

Checked as you type

28 checks run as you draw, from Kafka topic-name rules to cycles and ports, each with a severity and a Problems tab per asset. Validate project checks everything at once.

Decisions and assumptions

Log decisions, assumptions and dependencies against the assets they affect, with who, when and next actions. They travel into every export.

Domain map and data products

Draw the business domains, link pipelines to them, and mark the data products, with the DPCF project template to start from.

Built for Confluent Cloud

Every asset on the palette is a real Confluent Cloud part with its real settings, so a design is something you can build, not just a drawing.

Environments and clusters Basic, Standard, Enterprise, Dedicated and Freight clusters, with their environment, ids, endpoints, cloud and region; dev, test and prod.

Topics Partitions, retention and cleanup, key and value schemas, dead letter topics, and existing topics you don’t own.

Schema Registry Avro, JSON Schema and Protobuf, validated as you type, with a visual Avro editor and an example message.

Data contracts A rule library for data quality and migration rules, attached to topics, with their dead letter topics.

Stream Governance Stream Catalog tags and business metadata on topics and schemas, personal data tags on fields.

Fully managed connectors All 102, each with its own settings: CDC, JDBC, object storage, warehouses, messaging and SaaS, with error handling.

Flink SQL INSERT INTO, CREATE TABLE AS, materialized tables and statement sets, on compute pools with a CFU check.

More processing UDFs, AI models, streaming agents, Tableflow, ksqlDB clusters and queries, Kafka Streams and other stream apps.

Clients and sharing Producers, consumer groups and share groups, with their client ids, group ids and the account they run as; Stream Share.

Service accounts and RBAC Accounts, role bindings and API keys, with permission advice for what your design appears to need.

In Confluent Cloud every topic is also a Flink table, so KPD wires topics straight into Flink SQL.

Technical exports

One design, every audience: a picture for the steering group, a document for the architects, a spreadsheet for the testers and Terraform for the platform team.

Pictures

PNG

Transparent PNGs with a legend for the colours and connection styles used, and the cluster captioned. Ready for slides and wikis.

Technical export

Markdown · CSV · pipeline JSON · project zip

Every asset with its palette, settings, schemas, tags, metadata and shading; notes and decisions quoted under the assets they belong to.

Detailed design document

13 sections · Markdown → Word

A low-level design generated from the model: domain model, topic catalogue, schemas field by field, processing sequence, security and operations.

Terraform

Confluent provider

A starting point, checked by tofu validate, with a go / no-go first. One root module, a file per pipeline: topics, schemas with contract rules, tags, connectors, Flink statements, compute pools, service accounts and role bindings.

Permission advice

Role bindings

Before Terraform is generated, KPD lists the bindings your running assets seem to need, including write access to a contract rule’s dead letter queue. Nothing is added unless you tick it.

No keys, ever

Sensitive variables

KPD refuses text that looks like a secret. Every credential and connector secret becomes a sensitive Terraform variable.

Every feature, by category

The full list. Each one is explained, with pictures, in the KPD user guide.

Canvas and modelling

Drag-and-drop canvas with 123 asset types
Auto-routed connections
Animated data flow along connections
Auto-place to tidy a pipeline
Pipeline boxes that link one pipeline to another
Groups, notes and shading
Light and dark themes
Undo, redo, copy and paste, zoom and fit

Working with others

Workshop mode for designing in the room
Follow the data: a guided walk through the pipeline
Journey picker for pipelines with several destinations
Decision log with who, when and next actions
Assumptions and dependencies
Shared assets across pipelines

Data products and domains

Domain map with business areas and sub-domains
Anchor statements for each pipeline
Mark assets as data products
DPCF project template
Projects from design documents (LLD to KPD)
Two sample projects to learn from

Schemas

Avro, JSON Schema and Protobuf
Spec-checked schemas
Schema agreement checks between producers and consumers
Create or import schemas
Visual Avro editor with an example message
Validated as you type
Schema folders
Personal data tags on fields

Contracts and governance

Data contract rule library
Rules attached to topics, with dead letter topics
Stream Catalog tags
Business metadata
Topic naming: Kafka rules, the DPCF standard, or your own
Who needs access to a contract’s topics

Processing

Flink SQL statements of every kind
Statement sets writing to several topics
Materialized tables
Compute pools, with a CFU check on capacity
Tableflow
UDFs, AI models and streaming agents
ksqlDB, Kafka Streams and stream apps

Connectors and systems

102 fully managed connectors
CDC, JDBC, object storage, warehouses, messaging, SaaS
Error handling and dead letter queues
Self-managed source and sink connectors
External systems at either end
Existing topics you don’t own
Producers, consumer groups and share groups
Stream Share

Environments and security

Clusters of every type, shared by the project
Environments for dev, test and prod
Service accounts and role bindings
Optional API keys, where you ask for them
Least-privilege permission advice, never applied for you
Dead letter topic writer advice
Keys as CI variables; secrets blocked

Validation

28 checks, live as you draw
Errors, warnings and information
Problems tab for each asset
Validate project, all pipelines at once
Terraform go / no-go
Cycle and port checks
Problem badges on the canvas
Migrating designs from older versions

Exports and documentation

Transparent PNG with a legend
Markdown and CSV tables
Pipeline JSON and project .zip
13-section detailed design (LLD) document
Terraform starting point, checked by tofu validate
Choose what appears in each export
Plain project files you can keep in git

Getting around

Quick add with /
Find in project (⌘K), with filters
Grouped project explorer
Highlight results on the canvas
Keyboard shortcuts for everything common
Your own palettes
Hide assets from a picture

Try it on your own pipeline

KPD runs in your browser in the beta, with nothing to install.

Request access Read the user guide Check your browser

From the first sketch to the Terraform.