GuillaumeHemmen-k8s/coder-sindri-deployment
2
0
Fork 0
Code Issues Pull requests Releases Activity Actions

Migrate the home PVC to kubernetes_persistent_volume_claim_v1 without recreating it #6

Merged
GuillaumeHemmen merged 2 commits from chore/pvc-v1-migration into master 2026-09-30 08:50:43 +00:00
Conversation 2 Commits 2 Files changed 1 +41 -3
claude-bot commented 2026-09-30 08:22:16 +00:00
Member
Copy link

Follow-up to #5: resolves the last deprecation warning (kubernetes_persistent_volume_claim.home) without touching any home volume. WIP until tested on a throwaway workspace.

Why not a plain rename

Renaming the resource type makes Terraform destroy the old PVC and create a new one. The longhorn StorageClass has reclaimPolicy: Delete, so that deletes the home data. A moved block can't help: v3.2.1's MoveResourceState is a no-op stub, so cross-type moves are unsupported.

How it works (step 1 of 2)

On each workspace's next build (start or stop, since the PVC isn't count-gated):

  • removed { lifecycle { destroy = false } } drops the old address from state without deleting the PVC;
  • import adopts the same PVC under kubernetes_persistent_volume_claim_v1.home.

The import is gated on a kubernetes_resources lookup of the PVC, so brand-new workspaces (no PVC yet) and the template-push dry-run just create it normally. Once _v1 is in state, Terraform treats the import as a no-op.

Tested (plan only, never applied)

Terraform 1.14.5 + kubernetes v3.2.1, planned against the real cluster with a local state and real parameter values:

Scenario PVC plan Destroy
Existing workspace (state imported from ghe-perso's real PVC, old type) old: forget; _v1: import + in-place update (wait_until_bound only), same UID 46f76366…, 100Gi 0
Brand-new workspace (random id, empty state) _v1: create, no import 0
Second build after migration (_v1 already in state) _v1: update, no import 0

The deprecation warning is gone from terraform validate.

Still to do: a real build on a throwaway workspace (file in ~ must survive, PVC UID must be unchanged).

Step 2 (separate PR, later)

Once every workspace has had one build on this version (check that the resource type in each workspace's latest build is _v1), delete the data source and the import block. Keep the removed block: if a straggler still had the old address in state and removed were gone too, Terraform would plan to destroy its PVC. With removed kept, the worst case is a harmless "already exists" error on create.

🤖 Generated with Claude Code

Follow-up to #5: resolves the last deprecation warning (`kubernetes_persistent_volume_claim.home`) **without touching any home volume**. WIP until tested on a throwaway workspace. ## Why not a plain rename Renaming the resource type makes Terraform destroy the old PVC and create a new one. The `longhorn` StorageClass has `reclaimPolicy: Delete`, so that deletes the home data. A `moved` block can't help: v3.2.1's `MoveResourceState` is a no-op stub, so cross-type moves are unsupported. ## How it works (step 1 of 2) On each workspace's next build (start **or** stop, since the PVC isn't count-gated): - `removed { lifecycle { destroy = false } }` drops the old address from state **without deleting** the PVC; - `import` adopts the **same** PVC under `kubernetes_persistent_volume_claim_v1.home`. The import is gated on a `kubernetes_resources` lookup of the PVC, so brand-new workspaces (no PVC yet) and the template-push dry-run just create it normally. Once `_v1` is in state, Terraform treats the import as a no-op. ## Tested (plan only, never applied) Terraform 1.14.5 + kubernetes v3.2.1, planned against the real cluster with a local state and real parameter values: | Scenario | PVC plan | Destroy | |---|---|---| | Existing workspace (state imported from ghe-perso's real PVC, old type) | old: `forget`; `_v1`: import + in-place update (`wait_until_bound` only), same UID `46f76366…`, 100Gi | **0** | | Brand-new workspace (random id, empty state) | `_v1`: `create`, no import | 0 | | Second build after migration (`_v1` already in state) | `_v1`: update, no import | 0 | The deprecation warning is gone from `terraform validate`. Still to do: a real build on a throwaway workspace (file in `~` must survive, PVC UID must be unchanged). ## Step 2 (separate PR, later) Once **every** workspace has had one build on this version (check that the resource type in each workspace's latest build is `_v1`), delete the data source and the `import` block. **Keep the `removed` block**: if a straggler still had the old address in state and `removed` were gone too, Terraform would plan to **destroy** its PVC. With `removed` kept, the worst case is a harmless "already exists" error on create. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
The provider cannot move state across resource types, so a plain rename
would destroy and recreate every home PVC (reclaimPolicy Delete). Instead,
on each workspace's next build a removed block (destroy = false) drops the
old address from state and an import block adopts the same PVC under the
_v1 address. The import only runs when the PVC exists, so new workspaces
just create it, and it is a no-op once the _v1 address is in state.

Step 1 of 2: the data source and import block go away once every
workspace has built on this version.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
kubernetes_resources lists CRDs cluster-wide to resolve even built-in
kinds, which the coder service account may not do, so the template push
failed with 'cannot list resource customresourcedefinitions'. The typed
kubernetes_persistent_volume_claim_v1 data source is a plain GET, which
the SA already has, and returns empty rather than an error when the PVC
does not exist.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
claude-bot commented 2026-09-30 08:36:15 +00:00
Author
Member
Copy link

Template push of af361c5 failed at "Detecting persistent resources":

failed to look up GVK [/v1, Kind=PersistentVolumeClaim] among available CRDs:
customresourcedefinitions.apiextensions.k8s.io is forbidden: User "system:serviceaccount:coder:coder"
cannot list resource "customresourcedefinitions" ... at the cluster scope

kubernetes_resources is a manifest-family data source: it lists CRDs cluster-wide even for built-in kinds. My earlier dry-run used a kubeconfig with broader rights than the coder SA, so it didn't catch this. Nothing was applied; the test workspace's PVC is untouched.

Fix (16db2af): the existence check now uses the typed kubernetes_persistent_volume_claim_v1 data source, a plain GET (kubectl auth can-i get pvc -n coder --as=system:serviceaccount:coder:coder → yes) that returns empty on NotFound (data_source_kubernetes_persistent_volume_claim_v1.go:111). The import is gated on the PVC having a UID. No RBAC change.

Re-planned all three scenarios, using the red-reindeer-86 test workspace (PVC UID 83d83509…) as the existing one: old address forget + same UID imported / brand-new workspace create without import / already migrated: no import. 0 to destroy in all three.

Caveat: I couldn't run the plan as the SA (writing an impersonating kubeconfig was blocked), so SA coverage is proven by can-i on the exact calls (GET pvc), not by a full plan as the SA. The template push itself is the real check.

Template push of `af361c5` failed at "Detecting persistent resources": ``` failed to look up GVK [/v1, Kind=PersistentVolumeClaim] among available CRDs: customresourcedefinitions.apiextensions.k8s.io is forbidden: User "system:serviceaccount:coder:coder" cannot list resource "customresourcedefinitions" ... at the cluster scope ``` `kubernetes_resources` is a manifest-family data source: it lists CRDs cluster-wide even for built-in kinds. My earlier dry-run used a kubeconfig with broader rights than the coder SA, so it didn't catch this. Nothing was applied; the test workspace's PVC is untouched. **Fix (`16db2af`):** the existence check now uses the typed `kubernetes_persistent_volume_claim_v1` data source, a plain `GET` (`kubectl auth can-i get pvc -n coder --as=system:serviceaccount:coder:coder` → yes) that returns empty on NotFound (`data_source_kubernetes_persistent_volume_claim_v1.go:111`). The import is gated on the PVC having a UID. No RBAC change. Re-planned all three scenarios, using the `red-reindeer-86` test workspace (PVC UID `83d83509…`) as the existing one: old address `forget` + same UID imported / brand-new workspace `create` without import / already migrated: no import. **0 to destroy** in all three. Caveat: I couldn't run the plan *as* the SA (writing an impersonating kubeconfig was blocked), so SA coverage is proven by `can-i` on the exact calls (GET pvc), not by a full plan as the SA. The template push itself is the real check.
GuillaumeHemmen changed title from WIP: Migrate the home PVC to kubernetes_persistent_volume_claim_v1 without recreating it to Migrate the home PVC to kubernetes_persistent_volume_claim_v1 without recreating it 2026-09-30 08:50:31 +00:00
GuillaumeHemmen deleted branch chore/pvc-v1-migration 2026-09-30 08:50:43 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
GuillaumeHemmen
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
GuillaumeHemmen-k8s/coder-sindri-deployment!6
No description provided.