State Management Lab

7 minute read

Lab Overview

In this comprehensive lab, you’ll implement complete state management for the CacheCloud provider:

What You’ll Build:

  • Resource import functionality
  • Drift detection
  • Schema versioning and migration
  • Private metadata storage

Time: 90 minutes

Prerequisites

Continue with the CacheCloud provider from previous labs.

cd terraform-provider-cachecloud

Step 1: Implement Import

Add ImportState Interface

Update internal/provider/cache_resource.go:

package provider

import (
    "context"
    "fmt"
    
    "github.com/hashicorp/terraform-plugin-framework/path"
    "github.com/hashicorp/terraform-plugin-framework/resource"
    "github.com/hashicorp/terraform-plugin-framework/resource/schema"
    "github.com/hashicorp/terraform-plugin-framework/types"
    "github.com/hashicorp/terraform-plugin-log/tflog"
    
    "github.com/yourorg/terraform-provider-cachecloud/internal/client"
)

var _ resource.ResourceWithImportState = &CacheResource{}

func (r *CacheResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) {
    id := req.ID
    
    tflog.Info(ctx, "Importing cache", map[string]any{"id": id})
    
    // Validate ID format
    if !strings.HasPrefix(id, "cache-") {
        resp.Diagnostics.AddError(
            "Invalid Cache ID",
            fmt.Sprintf(
                "Cache ID must start with 'cache-', got: %s\n\n"+
                "Find cache IDs with: cachecloud-cli list-caches",
                id,
            ),
        )
        return
    }
    
    // Verify cache exists
    cache, err := r.client.GetCache(ctx, id)
    if err != nil {
        if client.IsNotFoundError(err) {
            resp.Diagnostics.AddError(
                "Cache Not Found",
                fmt.Sprintf("No cache found with ID: %s", id),
            )
            return
        }
        
        resp.Diagnostics.AddError(
            "Import Failed",
            fmt.Sprintf("Could not verify cache: %s", err.Error()),
        )
        return
    }
    
    // Check cache is importable
    if cache.Status == "deleting" {
        resp.Diagnostics.AddError(
            "Cannot Import Cache",
            fmt.Sprintf(
                "Cache %s is being deleted and cannot be imported.\n"+
                "Only caches in 'available' state can be imported.",
                id,
            ),
        )
        return
    }
    
    // Set ID - Read will populate the rest
    resource.ImportStatePassthroughID(ctx, path.Root("id"), req, resp)
    
    tflog.Info(ctx, "Successfully imported cache", map[string]any{
        "id":     cache.ID,
        "name":   cache.Name,
        "status": cache.Status,
    })
}

Test Import

Create examples/import/import.tf:

terraform {
  required_providers {
    cachecloud = {
      source = "registry.terraform.io/yourorg/cachecloud"
    }
  }
}

provider "cachecloud" {
  api_key = "test-key"
}

# Configuration for imported resource
resource "cachecloud_cache" "imported" {
  name          = "production-cache"
  engine        = "redis"
  engine_version = "7.0"
  node_type     = "cache.t3.micro"
  num_nodes     = 2
  
  tags = {
    Environment = "production"
  }
}

Test import:

cd examples/import

# Initialize
terraform init

# Import existing cache
terraform import cachecloud_cache.imported cache-12345

# Verify no changes needed
terraform plan

Step 2: Implement Drift Detection

Update Read Operation

Ensure Read refreshes ALL attributes:

func (r *CacheResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
    var state CacheResourceModel
    diags := req.State.Get(ctx, &state)
    resp.Diagnostics.Append(diags...)
    if resp.Diagnostics.HasError() {
        return
    }
    
    id := state.ID.ValueString()
    
    tflog.Debug(ctx, "Reading cache", map[string]any{"id": id})
    
    // Fetch from API
    cache, err := r.client.GetCache(ctx, id)
    
    if err != nil {
        if client.IsNotFoundError(err) {
            // Cache deleted outside Terraform
            tflog.Warn(ctx, "Cache not found, removing from state", map[string]any{
                "id": id,
            })
            resp.State.RemoveResource(ctx)
            return
        }
        
        resp.Diagnostics.AddError(
            "Read Failed",
            fmt.Sprintf("Could not read cache %s: %s", id, err.Error()),
        )
        return
    }
    
    // Update ALL attributes for drift detection
    state.ID = types.StringValue(cache.ID)
    state.Name = types.StringValue(cache.Name)
    state.Engine = types.StringValue(cache.Engine)
    state.EngineVersion = types.StringValue(cache.EngineVersion)
    state.NodeType = types.StringValue(cache.NodeType)
    state.NumNodes = types.Int64Value(int64(cache.NumNodes))
    state.Port = types.Int64Value(int64(cache.Port))
    
    // Update computed attributes
    state.Status = types.StringValue(cache.Status)
    state.Endpoint = types.StringValue(cache.Endpoint)
    state.ARN = types.StringValue(cache.ARN)
    
    // Update tags (important for drift detection!)
    if len(cache.Tags) > 0 {
        tagsMap := make(map[string]attr.Value)
        for k, v := range cache.Tags {
            tagsMap[k] = types.StringValue(v)
        }
        mapValue, diags := types.MapValue(types.StringType, tagsMap)
        resp.Diagnostics.Append(diags...)
        state.Tags = mapValue
    } else {
        state.Tags = types.MapNull(types.StringType)
    }
    
    // Save updated state
    resp.State.Set(ctx, state)
    
    tflog.Debug(ctx, "Successfully read cache", map[string]any{
        "id":        cache.ID,
        "num_nodes": cache.NumNodes,
        "status":    cache.Status,
    })
}

Test Drift Detection

Create examples/drift/main.tf:

provider "cachecloud" {
  api_key = "test-key"
}

resource "cachecloud_cache" "test" {
  name          = "drift-test"
  engine        = "redis"
  engine_version = "7.0"
  node_type     = "cache.t3.micro"
  num_nodes     = 2
  
  tags = {
    Environment = "test"
  }
}

Test drift:

cd examples/drift

# Create resource
terraform apply

# Manually modify (simulate drift)
# For testing, you could update your mock client to return different values

# Detect drift
terraform plan

# Should show:
# ~ resource "cachecloud_cache" "test" {
#     ~ num_nodes = 3 -> 2  # Drift detected!
# }

Step 3: Schema Versioning

Add Schema Version

Update schema to include version:

func (r *CacheResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) {
    resp.Schema = schema.Schema{
        Version: 1,  // ← Start versioning
        
        MarkdownDescription: "Manages a cache cluster",
        
        Attributes: map[string]schema.Attribute{
            "id": schema.StringAttribute{
                Computed:            true,
                MarkdownDescription: "Cache ID",
            },
            // ... rest of attributes
        },
    }
}

Step 4: Implement Schema Migration

Scenario: Migrate Storage Format

V0: Storage as integer (GB)
V1: Storage as string with unit

Define V0 Schema

Create internal/provider/cache_resource_v0.go:

package provider

import (
    "github.com/hashicorp/terraform-plugin-framework/resource/schema"
    "github.com/hashicorp/terraform-plugin-framework/types"
)

var CacheResourceSchemaV0 = schema.Schema{
    Version: 0,
    
    Attributes: map[string]schema.Attribute{
        "id": schema.StringAttribute{
            Computed: true,
        },
        "name": schema.StringAttribute{
            Required: true,
        },
        "storage_gb": schema.Int64Attribute{  // V0: Integer
            Required: true,
        },
    },
}

Implement Migration

Update internal/provider/cache_resource.go:

var _ resource.ResourceWithUpgradeState = &CacheResource{}

func (r *CacheResource) UpgradeState(ctx context.Context) map[int64]resource.StateUpgrader {
    return map[int64]resource.StateUpgrader{
        0: {
            PriorSchema:   &CacheResourceSchemaV0,
            StateUpgrader: upgradeCacheStateV0toV1,
        },
    }
}

func upgradeCacheStateV0toV1(ctx context.Context, req resource.UpgradeStateRequest, resp *resource.UpgradeStateResponse) {
    // Define V0 model
    type CacheResourceModelV0 struct {
        ID        string `tfsdk:"id"`
        Name      string `tfsdk:"name"`
        StorageGB int64  `tfsdk:"storage_gb"`  // V0: Integer
    }
    
    var stateV0 CacheResourceModelV0
    
    // Unmarshal V0 state
    err := json.Unmarshal(req.RawState.JSON, &stateV0)
    if err != nil {
        resp.Diagnostics.AddError(
            "State Upgrade Failed",
            fmt.Sprintf("Could not unmarshal v0 state: %s", err.Error()),
        )
        return
    }
    
    tflog.Info(ctx, "Upgrading cache state from v0 to v1", map[string]any{
        "id":           stateV0.ID,
        "storage_gb_v0": stateV0.StorageGB,
    })
    
    // Convert to V1 format
    type CacheResourceModelV1 struct {
        ID      string `tfsdk:"id"`
        Name    string `tfsdk:"name"`
        Storage string `tfsdk:"storage"`  // V1: String with unit
    }
    
    stateV1 := CacheResourceModelV1{
        ID:      stateV0.ID,
        Name:    stateV0.Name,
        Storage: fmt.Sprintf("%dGB", stateV0.StorageGB),  // 100 → "100GB"
    }
    
    // Set upgraded state
    resp.State.SetAttribute(ctx, path.Root("id"), stateV1.ID)
    resp.State.SetAttribute(ctx, path.Root("name"), stateV1.Name)
    resp.State.SetAttribute(ctx, path.Root("storage"), stateV1.Storage)
    
    tflog.Info(ctx, "State upgraded successfully", map[string]any{
        "id":          stateV1.ID,
        "storage_v1":  stateV1.Storage,
    })
}

Update Current Schema (V1)

func (r *CacheResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) {
    resp.Schema = schema.Schema{
        Version: 1,  // Current version
        
        Attributes: map[string]schema.Attribute{
            "id": schema.StringAttribute{
                Computed: true,
            },
            "name": schema.StringAttribute{
                Required: true,
            },
            "storage": schema.StringAttribute{  // V1: String
                Required:            true,
                MarkdownDescription: "Storage size (e.g., '100GB', '1TB')",
            },
        },
    }
}

Step 5: Private Metadata

Store Private Data

Update Create operation:

func (r *CacheResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
    var plan CacheResourceModel
    req.Plan.Get(ctx, &plan)
    
    // Create cache
    cache, err := r.client.CreateCache(ctx, CreateCacheInput{
        Name:          plan.Name.ValueString(),
        Engine:        plan.Engine.ValueString(),
        EngineVersion: plan.EngineVersion.ValueString(),
        NodeType:      plan.NodeType.ValueString(),
        NumNodes:      int(plan.NumNodes.ValueInt64()),
    })
    
    if err != nil {
        resp.Diagnostics.AddError("Create Failed", err.Error())
        return
    }
    
    // Set public state
    state := CacheResourceModel{
        ID:            types.StringValue(cache.ID),
        Name:          types.StringValue(cache.Name),
        Engine:        types.StringValue(cache.Engine),
        EngineVersion: types.StringValue(cache.EngineVersion),
        NodeType:      types.StringValue(cache.NodeType),
        NumNodes:      types.Int64Value(int64(cache.NumNodes)),
        Port:          types.Int64Value(int64(cache.Port)),
        Status:        types.StringValue(cache.Status),
        Endpoint:      types.StringValue(cache.Endpoint),
        ARN:           types.StringValue(cache.ARN),
    }
    
    resp.State.Set(ctx, state)
    
    // Store private metadata
    privateData := map[string]interface{}{
        "internal_id":  cache.InternalID,
        "created_at":   cache.CreatedAt,
        "api_version":  "v2",
    }
    
    privateJSON, _ := json.Marshal(privateData)
    resp.Private.SetKey(ctx, "cache_metadata", privateJSON)
    
    tflog.Info(ctx, "Created cache with private metadata", map[string]any{
        "id":          cache.ID,
        "internal_id": cache.InternalID,
    })
}

Use Private Data

Update Update operation:

func (r *CacheResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
    var plan, state CacheResourceModel
    req.Plan.Get(ctx, &plan)
    req.State.Get(ctx, &state)
    
    // Retrieve private metadata
    privateJSON, diags := req.Private.GetKey(ctx, "cache_metadata")
    resp.Diagnostics.Append(diags...)
    
    var privateData map[string]interface{}
    if len(privateJSON) > 0 {
        json.Unmarshal(privateJSON, &privateData)
        
        tflog.Debug(ctx, "Retrieved private metadata", map[string]any{
            "internal_id": privateData["internal_id"],
        })
    }
    
    // Update cache...
    
    // Preserve private metadata
    if len(privateJSON) > 0 {
        resp.Private.SetKey(ctx, "cache_metadata", privateJSON)
    }
}

Step 6: Build and Test

Build Provider

go mod tidy
go build -o terraform-provider-cachecloud

Install Locally

# macOS ARM64
mkdir -p ~/.terraform.d/plugins/registry.terraform.io/yourorg/cachecloud/2.0.0/darwin_arm64
cp terraform-provider-cachecloud ~/.terraform.d/plugins/registry.terraform.io/yourorg/cachecloud/2.0.0/darwin_arm64/

Comprehensive Test

Create examples/state-test/main.tf:

terraform {
  required_providers {
    cachecloud = {
      source = "registry.terraform.io/yourorg/cachecloud"
      version = "2.0.0"
    }
  }
}

provider "cachecloud" {
  api_key = "test-key"
}

# Test resource
resource "cachecloud_cache" "test" {
  name          = "state-test"
  engine        = "redis"
  engine_version = "7.0"
  node_type     = "cache.t3.micro"
  num_nodes     = 2
  
  tags = {
    Environment = "test"
    Purpose     = "state-testing"
  }
}

output "cache_id" {
  value = cachecloud_cache.test.id
}

Run tests:

cd examples/state-test

# Test 1: Create
terraform init
terraform apply

# Test 2: Import
terraform import cachecloud_cache.test2 cache-67890

# Test 3: Drift detection
# (Modify via API)
terraform plan  # Should show drift

# Test 4: State operations
terraform state list
terraform state show cachecloud_cache.test

# Test 5: Refresh
terraform apply -refresh-only

Lab Validation

Validation Checklist

  • Import works with valid cache ID
  • Import fails with helpful error for invalid ID
  • Import fails for non-existent cache
  • Read detects all attribute changes (drift)
  • Read removes resource when deleted externally
  • Schema version is set to 1
  • State upgrade from v0 to v1 works (if implemented)
  • Private metadata is stored during create
  • Private metadata is retrieved during update
  • Configuration matches imported state (no drift)
  • State commands work (list, show, mv, rm)

Expected Outputs

Import:

$ terraform import cachecloud_cache.imported cache-12345

cachecloud_cache.imported: Importing...
  ID: cache-12345
cachecloud_cache.imported: Import complete
  ID: cache-12345

$ terraform plan
No changes. Your infrastructure matches the configuration.

Drift Detection:

$ terraform plan

cachecloud_cache.test: Refreshing state... [id=cache-12345]

Terraform detected drift:

  # cachecloud_cache.test has been modified
  ~ resource "cachecloud_cache" "test" {
      ~ num_nodes = 3 -> 2
        # ...
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Challenge Exercises

1. Import with Multiple Formats

Support importing by name or ARN in addition to ID.

2. Validate State on Read

Add validation to detect corrupted state and provide recovery guidance.

3. Multi-Step Migration

Implement migration from v0 → v1 → v2 with multiple schema changes.

4. Private Data Caching

Use private data to cache expensive API responses and skip unnecessary calls.

5. Import Dry-Run

Add a way to validate import ID before actually importing.

Key Takeaways

  • ✅ Always implement ImportState for resources
  • ✅ Validate import IDs with helpful error messages
  • ✅ Read operation must refresh ALL attributes
  • ✅ Remove deleted resources from state
  • ✅ Version schemas for backward compatibility
  • ✅ Implement state upgraders for breaking changes
  • ✅ Use private data for provider-specific metadata
  • ✅ Test import, drift, and migration thoroughly
  • ✅ Log all state operations for debugging
  • ✅ Provide clear documentation for import format

What’s Next?

Test your understanding with practice questions!

Next: State Management Practice →

Previous: ← State Manipulation

Last updated: July 1, 2026

Updated:

You may also enjoy