Component state migrations#

Component state migrations let a component reshape its persisted resource state before the engine compares it with the component’s current registrations. A migration can rename, split, merge, or otherwise reorganize the component and its descendants without treating the change as unrelated deletes and creates.

Migrations change Pulumi’s state, they do not call providers. A managed custom resource and its successor must retain the same physical ID, provider, extension, ownership, and provider lifecycle state.

For example, suppose component Root previously contained OldChild, but its new implementation replaces that child with NewChild. A migration returns the updated subtree [Root, NewChild] and identifies NewChild as OldChild’s successor. The engine can then rewrite references from OldChild to NewChild without deleting and recreating the physical resource.

How migrations run#

Migrations use the engine’s callback system. The callbacks run in the language SDK process, and a resource registration can carry an ordered list of them. While planning that registration, the step generator:

Resolves the prior subtree root using the registration URN and its aliases.
Captures that root and its transitive descendants from Deployment.prev.Resources in snapshot order.
Converts those pkgresource.State values to ResourceV3 checkpoint JSON and invokes each callback in order.
Converts the result back to resource state and validates the returned subtree and successor mappings.
Prepares the complete post-migration prior-resource list, topologically sorts it, persists it, and commits one StateMigrationTransaction.
Retains the successor rewrite for references the program may send after the commit.
Normalizes the registration goal using the saved rewrites.
Resumes planning the registration against the migrated state.

Callback contract#

In simplified form, the Python SDK callback types and a synchronous migration have this shape:

class StateMigrationArgs:
    urn: str
    old_state: list[dict[str, Any]]

class StateMigrationResult:
    new_state: list[dict[str, Any]]
    successors: dict[str, str] | None

def migrate(args: StateMigrationArgs) -> StateMigrationResult | None:
    return StateMigrationResult(
        new_state=[...],
        successors={old_urn: new_urn},
    )

An asynchronous migration may return the same result from an Awaitable.

args.urn is the URN of the current registration. args.old_state contains the prior root and all of its descendants in checkpoint format. ¹ ¹When the engine resolves the prior root through an alias, args.urn identifies the current registration while args.old_state[0]["urn"] keeps the matched prior resource’s URN. The two URNs may therefore differ.

Returning None means that the state is already in the desired shape. Otherwise, new_state replaces that complete prior subtree and successors maps each omitted prior URN to a URN present in new_state. The engine also treats a result that represents the same state as the input as a no-op. No-op migrations skip persistence and commit.

Migration callbacks remain attached to the resource after a stack has migrated, so they must be idempotent. A callback should recognize state in its final shape and return None, so later updates do not repeat the state change.

A callback must only transform the supplied checkpoint data. It must not perform Pulumi runtime operations ² ²Where possible, language SDKs should detect and reject attempts to call back into the resource monitor from a migration callback, producing a useful error instead of allowing the program to deadlock. or wait for unresolved Output values. The resource monitor waits for the callback to return, so work that calls back into the monitor would deadlock.

Any secrets in the supplied checkpoint data are decrypted before the callback runs. The callback must treat those values as sensitive.

Results and successors#

Every resource in the prior subtree must either remain in the result or map to a successor in the result. This makes removal explicit: accidentally omitting a resource cannot silently forget a physical object that Pulumi manages.

Successors also tell the engine how to rewrite references. For example, if OldChild is replaced by NewChild, the callback maps OldChild to NewChild. References to OldChild in parents, dependencies, typed resource references, output dependencies, and other reference-bearing state fields are rewritten to NewChild. Plain strings that happen to contain a URN are not interpreted as references.

Successor mappings compose across callbacks. For example, if one callback maps OldChild to IntermediateChild and the next maps IntermediateChild to NewChild, the engine rewrites references from OldChild directly to NewChild.

Before invoking the next callback, the engine rewrites references within the returned subtree using the accumulated successor mappings. For example, after a callback renames a component, the next callback sees its children parented to the new component URN. The complete callback chain is still persisted as one transaction and intermediate results are not persisted separately.

A migration can also split inline settings out of a managed resource into a separate managed resource:

resource (inline settings) -> resource (settings removed)
                            + settings (same physical ID and provider)

The callback retains resource at its existing URN and returns the additional settings state with the same physical ID, provider, and extension identity. No successor mapping is needed: references to resource still refer to that resource. The callback supplies the settings resource’s inputs, outputs, and dependencies. The migration itself does not create or read a provider resource.

Safety validation#

Before preparing a transaction, the engine validates the callback result as a complete replacement subtree:

It must contain exactly one resource at the top of the replacement subtree: either the current registration (args.urn) or its matched prior resource (args.old_state[0]["urn"]), but not both.
That resource must keep its prior parent, and every other returned resource must be its descendant.
Returned URNs must remain in the registration’s stack and project, and all structural references must resolve in either the returned subtree or the retained prior state.
Provider resource states must be returned unchanged. A checkpoint rewrite cannot update the engine’s live provider registry.
Every returned custom resource must have a retained or explicitly mapped custom predecessor, or match prior custom state in the migrated subtree by non-empty physical ID, provider, and extension identity. This last case allows additional resources produced by a split.
A custom resource and its successor must preserve physical ID, provider and extension identity, External ownership, PendingReplacement, and Taint. Protect and RetainOnDelete are inherited conservatively: a many-to-one successor carries either flag if any predecessor had it.
For an additional custom resource without a retained or explicitly mapped predecessor, ownership and lifecycle flags must agree with every matching prior custom resource. Conflicting matches are rejected, not ignored. Protect and RetainOnDelete must each be set if any match carries that flag.

Two custom resources can share a provider and physical ID while representing different parts of the same object. The engine cannot tell whether such a cross-type merge, rename, or split is correct. The callback must choose the right successors and supply state compatible with each resource’s provider schema. Matching physical identity alone does not establish that the extracted settings state is valid for the new resource type.

When migrations can run#

Migrations are considered during updates and update previews when prior state exists. Migration callbacks do not run during refresh or destroy, even with --run-program. ³ ³Migrations reshape prior state before update planning. Refresh instead reconciles that state with providers, while destroy removes it. --run-program evaluates the program but does not turn either operation into an update. The selected persistence mode must also support state migrations; full-checkpoint snapshot persistence and journal v2 do, while older journal formats do not.

A state-changing result is rejected when the surrounding update cannot safely rewrite the complete checkpoint. This includes:

updates constrained by --target, --replace, --exclude, or --target-snippet
generating or applying a saved update plan
snapshots with pending operations, or migrated subtrees with resources pending deletion from an earlier update
migrations involving resource views
persisted snippets whose references would need to be rewritten

These restrictions apply only when a callback changes state. A permanently attached callback can return None without blocking partial updates, plan use, or recovery of pending state.

Preparing the transaction#

The engine validates and prepares a StateMigrationTransaction before mutating shared deployment state. Suppose Deployment.prev.Resources is:

[Provider, Root, OldChild, Consumer, Tail]

Root is the component root, and OldChild and Tail are its children. Consumer is outside the migrated subtree but is interleaved with its resources: Consumer depends on OldChild, and Tail depends on Consumer. The callback returns [Root, NewChild, Tail] and maps OldChild to NewChild.

The replacement subtree cannot be inserted as one contiguous block. Placing it at the old subtree’s last position would put Consumer before its new dependency NewChild, while placing it at the first position would put Tail before its dependency Consumer. The engine rewrites Consumer, builds the complete candidate snapshot, and topologically sorts all of its resources. The prepared resource values are:

[Provider, Root, NewChild, Consumer′, Tail]

where Consumer′ is a prepared copy of Consumer whose dependency refers to NewChild. If this resource graph cannot be sorted, preparation fails without persisting or committing the migration. Otherwise, the transaction records both the values to persist and the information needed to commit them without replacing live resource pointers:

Field	Value in the example	Purpose
`PriorSubtree`	`[Root, OldChild, Tail]`	Live states from `Deployment.prev.Resources` replaced by the migration
`ResultSubtree`	`[Root, NewChild, Tail]`	Replacement subtree
`SuccessorURNs`	`{OldChild: NewChild}`	Maps removed resources to their successors
`PreparedPriorResources`	`[Provider, Root, NewChild, Consumer′, Tail]`	Topologically sorted replacement for `Deployment.prev.Resources`
`RetainedResourceRewrites`	`{Consumer: Consumer′}`	Prepared values to copy into retained live objects
`currentResourceRewrites`	As needed	Prepared values for earlier `Deployment.news` and `Deployment.reads` states

Deployment.prev.Resources and other deployment bookkeeping may refer to the same *pkgresource.State object. If the engine replaced Consumer in Deployment.prev.Resources with the new Consumer′ object, other structures could still refer to the original Consumer and see its old references. The engine therefore validates and persists Consumer′, then copies its fields into the original Consumer. It sets Deployment.prev.Resources to [Provider, Root, NewChild, Consumer, Tail] and rebuilds the deployment indexes. The persisted and in-memory states have the same values while shared pointer identity is preserved.

Persistence#

During a real update, persistence happens before the engine changes Deployment.prev or any other shared resource state. If persistence fails, the migration returns an error and the live deployment remains unchanged. A preview performs the same preparation and in-memory commit without durable persistence.

Once a migration is persisted, it is not rolled back if a later part of the update fails. The next update starts from the migrated state, so migration callbacks must be idempotent.

The full-checkpoint snapshot manager combines PreparedPriorResources with its current operation state, then saves the result through the usual persistence path.

The journal manager starts with an immutable base snapshot and records ordered operations on top of it. A migration may change both the base and resources produced by earlier journal entries, so journal v2 records one migration entry containing:

Layout: the complete post-migration base snapshot order. Each item refers either to a retained resource by its index in the original base snapshot or to an inserted resource by its index in States. Base resources omitted from the layout are removed.
States: the complete replacement subtree. stateIndex items in Layout refer to resources in this list.
BaseStatePatches: complete replacements for retained base states whose references changed, addressed by their original base-snapshot index.
NewStatePatches: complete replacements for earlier operation results, addressed by operation ID.

These patches are complete resource states, not property-level diffs.

For the example above, the immutable base snapshot assigns indices 0 through 4 to [Provider, Root, OldChild, Consumer, Tail]. The migration entry has this shape:

{
  "kind": "state-migration",
  "states": ["Root", "NewChild", "Tail"],
  "layout": [
    {"baseIndex": 0},
    {"stateIndex": 0},
    {"stateIndex": 1},
    {"baseIndex": 3},
    {"stateIndex": 2}
  ],
  "baseStatePatches": [
    {"index": 3, "state": "Consumer′"}
  ]
}

The strings stand in for complete serialized resource states. Replay applies the patches and walks Layout to reconstruct [Provider, Root, NewChild, Consumer′, Tail] exactly; the omitted base indices for Root, OldChild, and Tail remove the prior subtree.

If the journal entry is acknowledged before a crash, replay recovers the exact prepared result. If persistence fails, there is no migration to commit.

References sent after commit#

Planning and step execution overlap while the program runs. Before a migration, the engine stops accepting source events, lets earlier planning and step chains finish, and blocks new execution while the migration commits. The transaction rewrites all engine-held state before work resumes, so engine-generated continuations do not need separate normalization.

The program may still hold a reference created before the migration and send it to the engine afterward. The deployment keeps a small stateMigrationRewrite with the successor URNs and physical identity needed to normalize such values. ReadResource and RegisterResourceOutputs events are normalized when they enter the engine. Registration goals are normalized later, after applying any migrations carried by that registration, because its own migration may create the required successor mapping.

URNs used for identity lookup are handled separately. A registration’s parent is rewritten before generating its URN, and its aliases are rewritten before looking up prior state. The predecessor URN cannot be reused as the primary URN of a new registration or read during the same deployment. References to that URN now mean the successor, so reusing it would be ambiguous.

Providers and analyzers receive normalized inputs and old state and are expected to derive resource references from those values.

Component state migrations

Contents