KPD closed beta Sign inRequest access

Detailed design document

Turn the whole project into a low-level design (LLD) document, with every table KPD can fill in already written.

Export → Export to Detailed Design Template writes the project as a Markdown document, <project>-detailed-design.md. It is downloaded and saved to the project's exports/ folder.

What is in it

The outline follows a typical client low-level design for a data product:

  1. Executive summary
  2. Business context, with the domain model: the domain map as a picture and a table of every domain and sub-domain, its owner, contacts and linked pipelines
  3. Data product objectives and scope: domain, systems of record, environments, consumers
  4. The end-to-end data product lifecycle: sources, raw layer, canonical model, curated products
  5. Logical architecture and flow, with the pipeline diagrams (each headed by its business domain, sub-domain and anchor statement) and a step-by-step processing sequence
  6. Data product definitions: owner, domain, SLA, quality, classification and topic exposure
  7. Topic and interface design: naming standard, topic catalogue, dead letter topics, interface contracts
  8. Schema Registry and evolution, with a field table for every schema
  9. Processing and enrichment design, with each Flink statement's SQL as written (KPD doesn't check it)
  10. Security and access: service accounts and role bindings
  11. Governance: tags, business metadata and data quality rules
  12. Environments and deployment
  13. Design decisions and assumptions

Generated and authored

Everything KPD knows is generated: asset tables, schema fields, quality rules, role bindings, tags and metadata. Everything it can't know (business drivers, SLAs nobody has entered, out-of-scope items) is a ✏️ Author placeholder with a starting point to edit, for example [Why real-time, governed data matters to them.]. Search the document for ✏️ to find what is left to write.

TipFill in governance first. Business metadata such as owner, domain and SLA on a data product topic flows straight into its definition section, so fewer placeholders are left.

Diagrams

Diagrams are PNG files saved next to the document in exports/ and linked by name. The open pipeline is drawn on export as <pipeline>-diagram.png, without notes, decisions and assumptions, so the picture shows only how the pipeline joins together; the decisions keep their own section. Other pipelines use their last such diagram, so open each pipeline and export once, or use Export PNG. The domain map is always drawn fresh, as domain-map-diagram.png: its tab opens for a moment during the export and the pipeline you were on comes back.

Opening it elsewhere