KPD Studio user guide
Draw the pipeline. Check the design. Hand over the Terraform.
KPD Studio is a design tool for data pipelines on Apache Kafka® and Confluent Cloud. You drag assets such as connectors, topics, Flink statements and clients onto a canvas, wire them together and describe them. KPD checks the design as you go and turns it into the pictures, documents and Terraform a project team needs.
What KPD is for
KPD sits where a pipeline is designed, before anything is built. It is a rapid documentation tool that also supports rapid prototyping:
- Storyboards: before any technical detail, tell the story of the data in plain words with the business, in Storyboard mode, then link the pipeline to it.
- Design workshops: sketch the flow with the client, from source systems through raw, canonical and product topics to the consumers.
- Design documents: export a clean diagram with a legend, a technical table, or a full low-level design document with every topic, schema, rule and access binding filled in.
- Hand-over: generate a Terraform starting point for the Confluent provider, for the delivery team to check in and run.
Ground rules
KPD never connects to Kafka or Confluent Cloud. It draws, checks and writes Terraform for your team to run.
Environments hold ids, never keys. Every credential in the Terraform is a variable you supply.
Permission advice is offered, never applied. Only the role bindings you choose are generated.
Ports refuse invalid connections, and validation flags naming, schema, security and Terraform problems as you work.
Where to go next
Open KPD in your browser with your beta username. Nothing to install.
A guided tour of every part of KPD on one small pipeline, with a sample to download.
Build and export your first pipeline in about ten minutes.
Sketch a pipeline's story in plain words, then link the real pipeline to it.
Projects, pipelines, assets, ports and shared assets.
Start a project from the Data Product Creation Framework template.
Turn the whole project into an LLD starting point.
Generate infrastructure as code for the Confluent provider.