State Management Practice
State Fundamentals
Question 1: State Structure
What information does Terraform state store for each resource?
View Answer
Terraform state stores: **1. Resource identification:** - Type (e.g., `mycloud_database`) - Name (e.g., `primary`) - Mode (`managed` for resources, `data` for data sources) **2. Instance data:** - Schema version - All attribute values - Sensitive attribute markers - Dependencies **3. Metadata:** - Provider information - Private provider data (base64 encoded) **Example:** ```json { "mode": "managed", "type": "mycloud_database", "name": "primary", "provider": "provider[\"registry.terraform.io/yourorg/mycloud\"]", "instances": [ { "schema_version": 1, "attributes": { "id": "db-12345", "name": "prod-db", "storage_gb": 100 }, "sensitive_attributes": [], "private": "eyJ...", "dependencies": ["mycloud_vpc.main"] } ] } ```Question 2: Schema Versioning
When should you increment the schema version for a resource?
View Answer
**Increment schema version (breaking changes):** - ✅ Renaming attributes - ✅ Changing attribute types - ✅ Removing attributes - ✅ Restructuring nested objects - ✅ Changing attribute semantics (same type, different meaning) **Example requiring version bump:** ```go // Version 0 "storage": schema.Int64Attribute{ Required: true, }, // Version 1 - Type changed! resp.Schema = schema.Schema{ Version: 1, // Must increment Attributes: map[string]schema.Attribute{ "storage": schema.StringAttribute{ // Changed type Required: true, }, }, } ``` **Don't increment (non-breaking changes):** - ❌ Adding new optional attributes - ❌ Adding new computed attributes - ❌ Changing descriptions - ❌ Updating validation rules - ❌ Fixing bugs without changing structure **Example not requiring version bump:** ```go // Version 1 - No increment needed resp.Schema = schema.Schema{ Version: 1, // Same version Attributes: map[string]schema.Attribute{ "storage": schema.StringAttribute{ Required: true, }, "backup_enabled": schema.BoolAttribute{ Optional: true, // New optional - not breaking }, }, } ```Resource Import
Question 3: Basic Import Implementation
Implement import for a database resource that supports importing by ID.
View Solution
```go package provider import ( "context" "fmt" "github.com/hashicorp/terraform-plugin-framework/path" "github.com/hashicorp/terraform-plugin-framework/resource" "github.com/hashicorp/terraform-plugin-log/tflog" ) var _ resource.ResourceWithImportState = &DatabaseResource{} func (r *DatabaseResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) { id := req.ID tflog.Info(ctx, "Importing database", map[string]any{"id": id}) // Validate ID format if !strings.HasPrefix(id, "db-") { resp.Diagnostics.AddError( "Invalid Database ID", fmt.Sprintf( "Database ID must start with 'db-', got: %s\n\n"+ "Find IDs with: mycloud-cli list-databases", id, ), ) return } // Optionally verify existence db, err := r.client.GetDatabase(ctx, id) if err != nil { if client.IsNotFoundError(err) { resp.Diagnostics.AddError( "Database Not Found", fmt.Sprintf("No database exists with ID: %s", id), ) return } resp.Diagnostics.AddError("Import Failed", err.Error()) return } // Set ID - Read will populate rest resource.ImportStatePassthroughID(ctx, path.Root("id"), req, resp) tflog.Info(ctx, "Successfully imported database", map[string]any{ "id": db.ID, "name": db.Name, }) } ``` **Usage:** ```bash terraform import mycloud_database.db db-12345 ```Question 4: Composite ID Import
Implement import for a subnet resource that requires both VPC ID and subnet ID.
View Solution
```go func (r *SubnetResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) { // Import format: "vpc-id/subnet-id" // Example: "vpc-12345/subnet-67890" id := req.ID tflog.Info(ctx, "Importing subnet", map[string]any{"import_id": id}) // Parse composite ID parts := strings.Split(id, "/") if len(parts) != 2 { resp.Diagnostics.AddError( "Invalid Import ID Format", fmt.Sprintf( "Expected format 'vpc-id/subnet-id', got: %s\n\n"+ "Example:\n"+ " terraform import mycloud_subnet.app vpc-12345/subnet-67890\n\n"+ "Find IDs with:\n"+ " mycloud-cli list-subnets", id, ), ) return } vpcID := parts[0] subnetID := parts[1] // Validate formats if !strings.HasPrefix(vpcID, "vpc-") { resp.Diagnostics.AddError( "Invalid VPC ID", fmt.Sprintf("VPC ID must start with 'vpc-', got: %s", vpcID), ) return } if !strings.HasPrefix(subnetID, "subnet-") { resp.Diagnostics.AddError( "Invalid Subnet ID", fmt.Sprintf("Subnet ID must start with 'subnet-', got: %s", subnetID), ) return } // Verify subnet exists subnet, err := r.client.GetSubnet(ctx, vpcID, subnetID) if err != nil { if client.IsNotFoundError(err) { resp.Diagnostics.AddError( "Subnet Not Found", fmt.Sprintf("Subnet %s not found in VPC %s", subnetID, vpcID), ) return } resp.Diagnostics.AddError("Import Failed", err.Error()) return } // Set both IDs resp.State.SetAttribute(ctx, path.Root("vpc_id"), vpcID) resp.State.SetAttribute(ctx, path.Root("id"), subnetID) tflog.Info(ctx, "Successfully imported subnet", map[string]any{ "vpc_id": vpcID, "subnet_id": subnetID, }) } ``` **Usage:** ```bash terraform import mycloud_subnet.app vpc-12345/subnet-67890 ```Drift Detection
Question 5: Detecting Attribute Drift
Implement a Read operation that properly detects drift for a database resource.
View Solution
```go func (r *DatabaseResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) { var state DatabaseResourceModel diags := req.State.Get(ctx, &state) resp.Diagnostics.Append(diags...) if resp.Diagnostics.HasError() { return } id := state.ID.ValueString() tflog.Debug(ctx, "Reading database for drift detection", map[string]any{ "id": id, }) // Fetch current state from API db, err := r.client.GetDatabase(ctx, id) if err != nil { if client.IsNotFoundError(err) { // Resource deleted outside Terraform tflog.Warn(ctx, "Database not found, removing from state", map[string]any{ "id": id, }) resp.State.RemoveResource(ctx) return } resp.Diagnostics.AddError( "Read Failed", fmt.Sprintf("Could not read database %s: %s", id, err.Error()), ) return } // Update ALL managed attributes for drift detection state.Name = types.StringValue(db.Name) state.Engine = types.StringValue(db.Engine) state.EngineVersion = types.StringValue(db.EngineVersion) state.StorageGB = types.Int64Value(int64(db.StorageGB)) state.Port = types.Int64Value(int64(db.Port)) state.MultiAZ = types.BoolValue(db.MultiAZ) state.PubliclyAccessible = types.BoolValue(db.PubliclyAccessible) // Update computed attributes (not drift, just current values) state.Status = types.StringValue(db.Status) state.Endpoint = types.StringValue(db.Endpoint) state.ARN = types.StringValue(db.ARN) // Update tags - important for drift detection! if len(db.Tags) > 0 { tagsMap := make(map[string]attr.Value) for k, v := range db.Tags { tagsMap[k] = types.StringValue(v) } mapValue, diags := types.MapValue(types.StringType, tagsMap) resp.Diagnostics.Append(diags...) state.Tags = mapValue } else { state.Tags = types.MapNull(types.StringType) } // Save updated state resp.State.Set(ctx, state) tflog.Debug(ctx, "Read complete, state updated", map[string]any{ "id": db.ID, "storage_gb": db.StorageGB, "multi_az": db.MultiAZ, }) } ``` **Key points:** - ✅ Update ALL managed attributes - ✅ Remove from state if not found - ✅ Update computed attributes for current state - ✅ Handle tags for drift detection - ✅ Log operations for debuggingQuestion 6: Deleted Resource Handling
What should happen when Read discovers a resource has been deleted?
View Answer
**Remove from state - don't return error:** ```go // ✅ Good: Remove from state if client.IsNotFoundError(err) { tflog.Warn(ctx, "Resource not found, removing from state", map[string]any{ "id": state.ID.ValueString(), }) resp.State.RemoveResource(ctx) return } // ❌ Bad: Return error if client.IsNotFoundError(err) { resp.Diagnostics.AddError( "Resource Not Found", "Resource was deleted", ) return } ``` **Why remove instead of error?** 1. Terraform needs to know resource is gone 2. Planning recreates the resource 3. Matches user expectations for external deletions 4. Enables disaster recovery workflows **User experience:** ```bash $ terraform plan mycloud_database.db: Refreshing state... [id=db-12345] Terraform detected drift: # mycloud_database.db has been deleted - resource "mycloud_database" "db" { - id = "db-12345" -> null } Terraform will perform the following actions: # mycloud_database.db will be created + resource "mycloud_database" "db" { + id = (known after apply) + name = "prod-db" } Plan: 1 to add, 0 to change, 0 to destroy. ```Schema Migrations
Question 7: Simple Type Change Migration
Migrate from integer storage to string with unit.
View Solution
**V0 Schema:** ```go var DatabaseResourceSchemaV0 = schema.Schema{ Version: 0, Attributes: map[string]schema.Attribute{ "id": schema.StringAttribute{ Computed: true, }, "storage": schema.Int64Attribute{ // V0: Integer Required: true, }, }, } ``` **V1 Schema:** ```go func (r *DatabaseResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) { resp.Schema = schema.Schema{ Version: 1, // Incremented Attributes: map[string]schema.Attribute{ "id": schema.StringAttribute{ Computed: true, }, "storage": schema.StringAttribute{ // V1: String Required: true, MarkdownDescription: "Storage size (e.g., '100GB', '1TB')", }, }, } } ``` **Migration:** ```go var _ resource.ResourceWithUpgradeState = &DatabaseResource{} func (r *DatabaseResource) UpgradeState(ctx context.Context) map[int64]resource.StateUpgrader { return map[int64]resource.StateUpgrader{ 0: { PriorSchema: &DatabaseResourceSchemaV0, StateUpgrader: upgradeDatabaseStateV0toV1, }, } } func upgradeDatabaseStateV0toV1(ctx context.Context, req resource.UpgradeStateRequest, resp *resource.UpgradeStateResponse) { type ModelV0 struct { ID string `tfsdk:"id"` Storage int64 `tfsdk:"storage"` // V0 } var stateV0 ModelV0 err := json.Unmarshal(req.RawState.JSON, &stateV0) if err != nil { resp.Diagnostics.AddError( "State Upgrade Failed", fmt.Sprintf("Could not unmarshal v0 state: %s", err.Error()), ) return } tflog.Info(ctx, "Upgrading state from v0 to v1", map[string]any{ "id": stateV0.ID, "storage_v0": stateV0.Storage, }) // Convert: 100 → "100GB" type ModelV1 struct { ID string `tfsdk:"id"` Storage string `tfsdk:"storage"` // V1 } stateV1 := ModelV1{ ID: stateV0.ID, Storage: fmt.Sprintf("%dGB", stateV0.Storage), } // Set upgraded state resp.State.SetAttribute(ctx, path.Root("id"), stateV1.ID) resp.State.SetAttribute(ctx, path.Root("storage"), stateV1.Storage) tflog.Info(ctx, "State upgrade complete", map[string]any{ "id": stateV1.ID, "storage_v1": stateV1.Storage, }) } ```Question 8: Nested Object Migration
Migrate flat attributes to a nested object.
View Solution
**V0: Flat structure** ```go "backup_enabled": schema.BoolAttribute{Required: true} "backup_retention_days": schema.Int64Attribute{Required: true} "backup_window": schema.StringAttribute{Required: true} ``` **V1: Nested object** ```go "backup_config": schema.SingleNestedAttribute{ Required: true, Attributes: map[string]schema.Attribute{ "enabled": schema.BoolAttribute{Required: true}, "retention_days": schema.Int64Attribute{Required: true}, "preferred_window": schema.StringAttribute{Required: true}, }, } ``` **Migration:** ```go func upgradeDatabaseStateV0toV1(ctx context.Context, req resource.UpgradeStateRequest, resp *resource.UpgradeStateResponse) { type ModelV0 struct { ID string `tfsdk:"id"` BackupEnabled bool `tfsdk:"backup_enabled"` BackupRetentionDays int64 `tfsdk:"backup_retention_days"` BackupWindow string `tfsdk:"backup_window"` } var stateV0 ModelV0 json.Unmarshal(req.RawState.JSON, &stateV0) tflog.Info(ctx, "Migrating flat backup fields to nested object") // Create nested backup config backupConfigAttrs := map[string]attr.Value{ "enabled": types.BoolValue(stateV0.BackupEnabled), "retention_days": types.Int64Value(stateV0.BackupRetentionDays), "preferred_window": types.StringValue(stateV0.BackupWindow), } backupConfigObj, diags := types.ObjectValue( map[string]attr.Type{ "enabled": types.BoolType, "retention_days": types.Int64Type, "preferred_window": types.StringType, }, backupConfigAttrs, ) resp.Diagnostics.Append(diags...) // Set state resp.State.SetAttribute(ctx, path.Root("id"), stateV0.ID) resp.State.SetAttribute(ctx, path.Root("backup_config"), backupConfigObj) tflog.Info(ctx, "Migration complete") } ```Private State Data
Question 9: Using Private Data
When and how should you use private state data?
View Answer
**Use private data for:** - Internal tracking IDs not exposed to users - Provisioning tokens or temporary state - API version tracking - Performance optimization (caching) **Don't use for:** - Data that should be in configuration - Data users need to see - Large data sets (keep it small) **Example:** **Storing private data:** ```go func (r *DatabaseResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) { // ... create resource ... // Set public state state := DatabaseResourceModel{ ID: types.StringValue(db.ID), Name: types.StringValue(db.Name), } resp.State.Set(ctx, state) // Store private metadata privateData := map[string]interface{}{ "internal_id": db.InternalID, // Not in schema "provisioning_token": db.ProvisionToken, // Temporary "api_version": "v2", // Tracking "created_timestamp": time.Now().Unix(), // Audit } privateJSON, _ := json.Marshal(privateData) resp.Private.SetKey(ctx, "metadata", privateJSON) tflog.Debug(ctx, "Stored private metadata", map[string]any{ "internal_id": db.InternalID, }) } ``` **Retrieving private data:** ```go func (r *DatabaseResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) { // Retrieve private data privateJSON, diags := req.Private.GetKey(ctx, "metadata") resp.Diagnostics.Append(diags...) var privateData map[string]interface{} if len(privateJSON) > 0 { json.Unmarshal(privateJSON, &privateData) // Use internal ID for more efficient API call if internalID, ok := privateData["internal_id"].(string); ok { db, err := r.client.UpdateByInternalID(ctx, internalID, input) // ... } } // ... continue update ... // Preserve private data resp.Private.SetKey(ctx, "metadata", privateJSON) } ``` **In state file:** ```json { "instances": [ { "attributes": { "id": "db-12345", "name": "prod-db" }, "private": "eyJpbnRlcm5hbF9pZCI6IjEyMzQ1IiwiYXBpX3ZlcnNpb24iOiJ2MiJ9" } ] } ```Real-World Scenarios
Scenario 1: Brownfield Import
How do you handle importing 100 existing databases into Terraform?
View Solution
**1. Implement robust import:** ```go func (r *DatabaseResource) ImportState(ctx context.Context, req resource.ImportStateRequest, resp *resource.ImportStateResponse) { id := req.ID // Validate and verify db, err := r.client.GetDatabase(ctx, id) if err != nil { resp.Diagnostics.AddError("Import Failed", err.Error()) return } resource.ImportStatePassthroughID(ctx, path.Root("id"), req, resp) } ``` **2. Script to generate configurations:** ```bash #!/bin/bash # generate-configs.sh # Get all database IDs DATABASE_IDS=$(mycloud-cli list-database-ids) # Generate configuration cat > databases.tf <<EOF terraform { required_providers { mycloud = { source = "yourorg/mycloud" } } } provider "mycloud" { region = "us-east-1" } EOF # For each database for db_id in $DATABASE_IDS; do # Fetch details DB_INFO=$(mycloud-cli get-database $db_id --format json) NAME=$(echo $DB_INFO | jq -r '.name') ENGINE=$(echo $DB_INFO | jq -r '.engine') STORAGE=$(echo $DB_INFO | jq -r '.storage_gb') # Generate resource block cat >> databases.tf <<EOF resource "mycloud_database" "db_${db_id}" { name = "${NAME}" engine = "${ENGINE}" storage_gb = ${STORAGE} # Other attributes... } EOF done echo "Generated databases.tf with ${#DATABASE_IDS[@]} databases" ``` **3. Import all:** ```bash #!/bin/bash # import-all.sh terraform init # Import each database for db_id in $(mycloud-cli list-database-ids); do echo "Importing $db_id..." terraform import "mycloud_database.db_${db_id}" "$db_id" done echo "Import complete. Run 'terraform plan' to verify." ``` **4. Verify:** ```bash $ terraform plan # Should show: No changes. Your infrastructure matches the configuration. ```Scenario 2: Disaster Recovery
State file corrupted. How do you recover?
