Project files and folders

A KPD project is a set of plain, readable files. Export it as a zip to keep a copy, review it or share it.

The project folder

inclined-labs/
├── project.kpdproj     project settings and shared definitions
├── pipelines/*.kpd     one file per pipeline
├── storyboards/        storyboards (.kpdstory), in folders if you like
├── schemas/            Avro (.avsc), JSON Schema (.json) and Protobuf (.proto), in subfolders if you like
├── sql/                Flink SQL kept in its own files (.sql), in subfolders if you like
├── exports/            PNG, Markdown and CSV written by KPD
│   └── terraform/      the Terraform export
└── .gitignore           keeps workspace files out of Git if you put the project in a repository

File	What it holds
`project.kpdproj`	The project's name and client; shared assets; clusters, service accounts and role bindings; compute pools; governance tags and business metadata definitions; contract rules; the shading key; and any project naming rule.
`pipelines/<name>.kpd`	A pipeline: its assets and the properties you set (defaults come from the palette), its connections, layout, groups, notes, decisions and assumptions, and the cluster it runs on.
`storyboards/…/<name>.kpdstory`	A storyboard: its items, links, layout, groups and notes. Step numbers and the story are worked out when it opens, never saved. A pipeline that implements it holds its id in a `storyboard` field, so renaming or moving a storyboard never breaks the link.
`schemas/…`	Schema files, exactly as Schema Registry would take them.
`sql/…/<name>.sql`	Flink SQL that statements keep in its own file, as plain SQL text. A statement holds the file’s path in `kpd.sqlFile`.

All of these are JSON, schema or SQL text with stable ordering, so a change to a design shows as a small, readable difference, and the files can go into Git or a code review if your team keeps designs there.

What a project can hold

A project holds only KPD’s own files, in their folders: the project file, pipelines, storyboards, schemas, SQL files, KPD’s exports (pictures, design documents, technical exports and the Terraform module) and Markdown notes next to the project file. Each must really be what its name says: a .kpd must be KPD’s JSON, a .png a picture, a .md plain text. KPD refuses to save anything else, and an import leaves out any file that doesn’t qualify and tells you which.

That keeps a project to the design, and stops it being used to store or pass on other files, which matters in the beta where every project is shared. The beta also limits an import to 25 MB and 1,000 files once unpacked, and each file to 10 MB. Real projects are far smaller: the Inclined Labs sample, pictures and all, is about 2 MB. On your own machine the limits are 200 MB, 5,000 files and 25 MB, and whoever runs KPD can change them (KPD_IMPORT_MAX_MB, KPD_IMPORT_MAX_FILES, KPD_MAX_FILE_MB). See also the terms of use.

Saving safely

KPD writes each file to a temporary file and renames it into place, so a crash never leaves half a file, and it keeps the previous version as <file>.bak. File and folder names Windows can't store are refused, so a project made on a Mac always opens on Windows.

Where projects live

In the beta, projects are kept in a shared repository on the server. Everyone in the beta can open every project, and the beta keeps a backup of them every night.

Keep client work outBecause everyone in the beta can see every project, don’t put confidential client material in it. Use made-up names, or the sample projects.

Opening, exporting and importing

File → Open project… Ctrl/⌘+O lists the projects in the shared repository.
File → Close project (or right-click the project name at the top of the explorer → Close project) closes the open project, offering to save any unsaved pipelines first, and shows the start screen with your projects listed. KPD then stays on the start screen, even after a refresh, until you open a project. When you reopen it, the tabs you had open come back.
Export project (.zip) on the toolbar downloads the whole project: the project file, its pipelines, schemas with their folders, and exports. Use it to keep a copy of your own, to send a design to someone, or to look at the files.
File → Import project (.zip)… creates a new project from a project zip. If a project of that name already exists, the new one gets a -1, -2… suffix, so nothing is overwritten. Files a project doesn’t hold are left out and listed when it finishes (see What a project can hold). Projects move as whole projects: there is no single-pipeline import or export, because a pipeline on its own leaves its schemas and shared assets behind.
A pipeline whose palettes have changed major version opens read-only; the Problems tab offers Migrate.

Two people, one pipeline

The last save wins. If someone else has a pipeline you have open, KPD shows a card saying so; agree who saves before you both edit. If your save lands on top of someone else’s newer one, you’re both told, and the other person can reopen the pipeline from the explorer to see your version.