Topic naming standards
KPD checks every topic name against Kafka’s own rules and against your naming standard.
Kafka's rules (errors)
A topic name may only use letters, digits, ., _ and -, and be at most 249 characters. Anything else, such as a space or & copied from a system's display name, is an error, because Kafka can't create it. Two names that differ only by . versus _ get a warning, because they collide in Kafka's metric names.
A topic name can be used once per pipeline: two different topic assets with the same name are an error, because they would be the same Kafka topic described twice. To draw one topic in several places, make it a shared asset. Two pipelines may use the same topic (one writes, the other reads), but the Terraform page won't generate until they describe it the same way.
The house standard (warnings)
By default KPD checks topic names against the Data Product Creation Framework standard:
<org>.<env>.<raw|ent|sys>.<entity>.<function>[.<sub-function>].v<n>[.dlq]
acme.prod.ent.item.merchandising.pricing.v1
incl.prod.raw.customer.bookingdb.v1
incl.prod.ent.customer.cdm.v1.dlq| Part | Meaning |
|---|---|
org | Organisation or business unit code, 2–4 lowercase letters or digits. |
env | dev, qa, preprod or prod. |
raw / ent / sys | Raw: data as produced by a source system, not yet processed. Ent: enterprise data processed by Confluent, such as canonical models and data products. Sys: system-facing and operational data, such as processing dead letter topics. |
entity, function, sub-function | The business entity and its function, for example customer.onboarding.registered. |
v<n>, .dlq | The version; a breaking change publishes a new version beside the old one. .dlq marks a dead letter topic. |
A name that doesn't match is a warning, never an error, so you can design before the standard is agreed.
Your own standard
A project can replace the rule with its own, for a client's standard: in project.kpdproj, set config.conventions.rules.topic to a regular expression with an example and a readable hint. The DPCF template does this for you, pinning <org> to your organisation code. Connector and Flink statement names have house rules too.