KPD closed beta Sign inRequest access

5. Governance: tags and the catalogue

Classification tags on every topic, and catalogue metadata on the two data products.

Part 5 of 10 in the Hello World tutorial · Follow this part in the guided wizard · Printable version

Tags say how sensitive data is; they become Stream Catalog tags in Confluent Cloud, so people can search for them and policies can act on them. Business metadata describes a data product in the catalogue: what it is, who owns it, how good it is. The DPCF template created both for you.

The tags you have

  1. In the explorer, open Stream governance → Tags. There are four classification tiers (Public, Internal, Confidential, Restricted), two overlays for special data (PII personal data, PCI card data) and DataProduct for published products.
The template’s tags, and the DataProduct business metadata with 13 attributes.
The template’s tags, and the DataProduct business metadata with 13 attributes.
  1. Click PII to see what a tag holds: a name, a colour and a description. Click Cancel: there is nothing to change.
A tag’s colour is the dot it shows on the canvas.
A tag’s colour is the dot it shows on the canvas.

Tag every topic

  1. Click the topic hwc.prod.raw.customer.shopdb.v1 and open its Governance tab.
  2. In Add a tag… choose Restricted, then open it again and choose PII. Each becomes a chip; click a chip's × to remove it.
Raw customer data: Restricted, and PII because it has names and emails.
Raw customer data: Restricted, and PII because it has names and emails.
  1. Tag the other topics the same way:
    TopicTagsWhy
    hwc.prod.raw.order.shopdb.v1RestrictedRaw data is never published.
    hwc.prod.raw.customer.shopdb.v1Restricted, PIIRaw, with personal data.
    hwc.prod.ent.order.cdm.v1Confidential, PIICleaned, but still has names and emails.
    hwc.prod.ent.order.history.v1Confidential, PII, DataProductA product, with customer names.
    hwc.prod.ent.order.big-orders.v1Internal, DataProductA product with no personal data.

Catalogue the two data products

  1. Click hwc.prod.ent.order.history.v1 and open Governance.
  2. In Add business metadata… choose DataProduct. Its attributes appear (a red * marks the required ones).
  3. Fill in:
    AttributeCustomer order historyBig orders
    ProductNameCustomer order historyBig orders
    DescriptionEvery store order with the customer who placed it, for the order history screen in the app.Orders of £20 or more, as they happen, so store managers can thank big customers and plan stock.
    ProductIdDP-001DP-002
    StewardPriya ShahPriya Shah
    CustodianData Platform TeamData Platform Team
    ClassificationConfidential, PIIInternal
    TierBronzeBronze
    LifecycleStateCandidateCandidate
    ContractSummarySchema hwc.prod.ent.order.history.v1-value. Each order within a minute of being placed.Schema hwc.prod.ent.order.big-orders.v1-value. Each big order within a minute of being placed.
The DataProduct metadata on the order history topic.
The DataProduct metadata on the order history topic.
  1. Notice Domain, SubDomain, Owner and Contact: you don't type them. They show from domain map, because the pipeline is linked to Sales / Store orders (Part 2). Change the domain map and every product follows.
Inherited from the domain map: Sales, Store orders, Store Orders Team.
Inherited from the domain map: Sales, Store orders, Store Orders Team.
  1. Do the same on hwc.prod.ent.order.big-orders.v1 with the Big orders column.

Back on the canvas, the coloured dots under each topic are its tags, so you can see the classification at a glance.

Tag dots under the topics.
Tag dots under the topics.

Save. Next: 6. Service accounts and role bindings.