Skip to content

ChangeWorkflow

A ChangeWorkflow is the declared series of Stages a change moves through on its way from where it is authored to everywhere it must land, with the conditions that must hold before a change may enter each Stage, before it may be released in one, and before the rollout counts as complete. It is the definition: the pattern. A ChangeOrder is the invocation: one named change moving through it.

The order is a partial order: sequential between Stages, parallel within one. A Stage is a wave. Everything in it may be promoted in any order, or all at once, and the waves are ordered. A fleet of a thousand clusters cannot be a sequence of a thousand steps.

A ChangeWorkflow is a child of a Space, like a Filter or a Tag, and is created, listed, and edited with cub changeworkflow. Keep workflows in a Space of their own: a workflow is not part of the component it governs, and a definition in a base Space would be cloned into every variant by cub variant create.

Stages:
  - Name: dev
    WhereSpace: "Labels.Stage = 'dev'"
  - Name: staging
    WhereSpace: "Labels.Stage = 'staging'"
    Prerequisites: [Validated, Released]
  - Name: prod
    WhereSpace: "Labels.Stage = 'prod'"
    Prerequisites: [Released, Healthy, not-frozen, approved]
    ReleasePrerequisites: [release-approved]
Final:
  Prerequisites: [Released, Healthy]
CustomPrerequisites:
  - Name: not-frozen
    Expression: "cel:!('code-freeze' in Space.Annotations) || Space.Annotations['code-freeze'] != 'true'"
    Description: the space is not under a code freeze
AttestationPrerequisites:
  - Name: approved
    Count: 2
    Description: two approvals of the change in staging, by people who did not write it
  - Name: release-approved
    FromUserIDs: [9b2c0f3e-5d7a-4a61-8f0e-2b6d1c4e7a90]
    MaxAge: 72h

Stages

WhereSpace is a where expression over Spaces, so Stage membership is by selector rather than by an enumerated list: a variant added to a Stage later is covered without editing the definition. The standard Space labels, Stage, Environment, and Region, are what Stages usually select on. An empty WhereSpace selects every Space of the component.

A Stage's membership is an intersection of three terms: its own selector, the component of the Space the ChangeOrder lives in, and that ChangeOrder's InScopeSpaceIDs. Two of the three come from the invocation, so the same definition resolves to different Spaces under different ChangeOrders, which is what lets one workflow be shared by several components, or cloned to give another component the same shape of rollout. The component is appended to every Stage's selector, so a WhereSpace that names Labels.Component itself is refused rather than conjoined: restating it either changes nothing or makes the Stage select no Space at all, and a Stage that promotes into nothing reports nothing wrong.

The definition does not name where the change starts. The base is the Space its ChangeOrder is created in, so the starting point is stated once, by the invocation.

Prerequisites

A Stage names its gates in two lists, and Final in a third.

Prerequisites are entry gates: what must hold to get into a Stage, evaluated over every Space in the Stage before it when a promotion is attempted. The first Stage's are therefore never evaluated. Having taken the change is checked whatever is declared; the prerequisites are checks on top of that. Three are built in:

  • Validated: no Unit of the Space has Validation Errors on the Revision the ChangeOrder's end Tag marks there. It needs no Release, so a Stage with no release Target can still require that the change passed validation there.
  • Released: the Space has published a Release carrying the change.
  • Healthy: the Space's live status, written back by the GitOps operator, reports Synced, Succeeded, and Healthy. It implies Released, since live status only exists downstream of a Release, and a Space with no release Target can never satisfy it.

Prerequisites are reads of current state, never cached from when the change was promoted, so a Stage that has degraded since it was promoted does not open the gate ahead of it.

ReleasePrerequisites gate publishing a Release for the ChangeOrder in one of the Stage's own Spaces, and are evaluated over the Revisions the Release bundles. They may name only attestation prerequisites, since the built-in and custom gates read the state of a Stage after the change has moved through it, which a publish into the Stage cannot answer. A publish is for a ChangeOrder when it names one with cub release publish --revision ChangeOrder:<slug>. A refused publish leaves no Release behind.

Final.Prerequisites are evaluated against the last Stage and say when the rollout is completed, rather than when a hop may happen. No promotion reads them, since there is no hop left to gate once the change is in the last Stage. They are named the same way a Stage's Prerequisites are, and are where "the change is not done until it is healthy in production" or "until someone signs off on production" belongs.

Custom prerequisites

CustomPrerequisites declares gates the built-in ones do not cover. Each has a Name, which is what a Stage or Final lists, a CEL Expression carrying the cel: prefix, and an optional Description of what it checks, which is where the reason lives when a promotion reports the gate by name. The expression is evaluated for each Space of the Stage ahead, with three variables bound to the JSON the API returns for each entity:

  • Space: the Space being evaluated.
  • ChangeOrder: the ChangeOrder being promoted.
  • Release: the earliest published Release of the Space carrying the change, or null when there is none.

Since the entities' Labels and Annotations are within reach, anything outside ConfigHub that can annotate a Space or a Release can gate a rollout, such as a code freeze flag or the result of a load test. Expressions are compiled when the workflow is written, so one that could never evaluate is refused then, not when it would hold up a promotion. An expression that does not produce a boolean fails the gate rather than reading as false.

Attestation prerequisites

AttestationPrerequisites declares the Attestations, such as approvals, that a Stage's Prerequisites, its ReleasePrerequisites, or Final may require. The fields are:

  • Name: what a Stage or Final lists.
  • Type: the Attestation type that counts. Approval by default.
  • Count: how many distinct attesters must record a Pass. 1 by default.
  • FromUserIDs: the users whose Attestations count. Empty means anyone with the Approve permission on the Space. Naming an integration's service account is how "the change record must be approved in ServiceNow" is expressed.
  • AllowAuthors: by default, an attester who wrote any Revision of the change does not count, since separation of duties is what approval exists to provide. true counts them.
  • MaxAge: how old an Attestation may be and still count, as a duration such as 72h.
  • IgnoreFail: by default, an unrevoked Fail from an eligible attester fails the requirement however many passes there are. true counts only passes.
  • Description: what the requirement is for.

FromGroupIDs and DistinctGroups, for requiring approvers from particular groups, are reserved and refused until Attestations record the groups of their attesters.

A requirement is evaluated over a set of Revisions, and holds when it holds for every one of them. In an entry gate, the set is the Revisions the ChangeOrder's end Tag marks in each Space of the Stage ahead, so "prod needs two approvals" means two approvals of the change as it stands in staging. In a release gate, it is the Revisions the Release bundles. For each Revision, the Attestations of Type that cover it, including those of an earlier Revision of the same Unit with the same content, are collected; revoked, expired, too old, ineligible, and author Attestations are discarded; and the distinct users with a Pass are counted.

The authors of a change are the users who wrote its Revisions in the Space: every Revision after the one the ChangeOrder's start Tag marks, through the Revision evaluated. A promotion's Revisions are written by whoever promoted, so a user who promotes a change into a Space cannot then approve it there. Automated Revisions exclude nobody.

Names

Prerequisite names are unique within a workflow across all three kinds, and a declared name may not shadow a built-in one. A Stage or Final naming a prerequisite that is neither built in nor declared is refused when the workflow is written.

How a ChangeOrder picks one up

cub changeorder create --change-workflow <workflow> names the workflow, and the server copies its definition onto the ChangeOrder. Every promotion and release of that ChangeOrder is judged against the copy, so editing the workflow part way through a rollout cannot change the rules a change already started under. The edit applies to ChangeOrders created afterwards. A change that has to move under the new rules is a new ChangeOrder.

The server enforces the gates for every client: cub variant promote, the UI, and the promote API are refused alike when an entry gate does not hold, and a publish is refused when a release gate does not. --force with a reason promotes past failing entry gates, and is recorded on the ChangeOrder.

As a ChangeOrder moves, the server records the Stage it has reached in its Stage, and Completed once the last Stage satisfies Final.

See promoting changes for the workflow in practice.