Provider Architecture & GRPC Protocol

6 minute read

Provider Architecture Overview

Understanding how Terraform providers work under the hood is essential for building reliable, performant providers. This chapter explores the plugin architecture, communication protocols, and lifecycle management.

By the end of this chapter, you’ll understand how Terraform Core and providers communicate, what happens during initialization, and why certain design decisions were made.

The Plugin System

Terraform uses a plugin architecture where providers run as separate processes that communicate with Terraform Core via Remote Procedure Calls (RPC).

Why Plugins?

Benefits:

  • ✅ Isolation — Provider crashes don’t crash Terraform
  • ✅ Language flexibility — Providers can be written in any language (though Go is standard)
  • ✅ Independent development — Providers evolve separately from Terraform Core
  • ✅ Security — Process boundaries limit blast radius
  • ✅ Scalability — Multiple providers run concurrently

Architecture Diagram

┌─────────────────────────────────────────┐
│         Terraform CLI / Core            │
│  (Terraform Language Interpreter)       │
└────────────┬────────────────────────────┘
             │ GRPC over
             │ stdin/stdout
             │ (Protocol v6)
┌────────────▼────────────────────────────┐
│       terraform-plugin-go               │
│    (GRPC Protocol Implementation)       │
└────────────┬────────────────────────────┘
             │
┌────────────▼────────────────────────────┐
│   terraform-plugin-framework            │
│      (High-level Provider SDK)          │
└────────────┬────────────────────────────┘
             │
┌────────────▼────────────────────────────┐
│      Your Provider Implementation        │
│    (Resources, Data Sources, Logic)     │
└────────────┬────────────────────────────┘
             │
┌────────────▼────────────────────────────┐
│       External API / Service            │
│    (AWS, Azure, Custom Platform)        │
└─────────────────────────────────────────┘

GRPC Protocol

Terraform uses GRPC (Google Remote Procedure Call) for provider communication.

What is GRPC?

GRPC is a high-performance RPC framework that uses:

  • HTTP/2 for transport
  • Protocol Buffers for serialization
  • Strongly-typed service definitions
  • Bi-directional streaming support

Why GRPC?

Advantages over JSON-RPC (used in older versions):

Feature GRPC JSON-RPC (Legacy)
Performance Faster (binary) Slower (text)
Type Safety Strong Weak
Streaming Yes No
Versioning Built-in Manual
Code Generation Automatic Manual

Protocol Version

Terraform supports multiple protocol versions:

Version Status Notes
Protocol 5 Legacy SDK v2
Protocol 6 Current Plugin Framework (recommended)

Check protocol version:

// In provider implementation
func (p *MyProvider) Metadata(ctx context.Context, req provider.MetadataRequest, resp *provider.MetadataResponse) {
    resp.TypeName = "mycloud"
    resp.Version = p.version
}

Plugin Handshake

When Terraform starts a provider, they perform a handshake to establish compatibility.

Handshake Process

1. Terraform starts provider binary
   └─> terraform-provider-aws

2. Provider writes magic cookie to stdout
   └─> "PLUGIN_PROTOCOL_VERSIONS=6"

3. Terraform reads cookie and validates
   └─> Checks protocol version compatibility

4. If compatible, establish GRPC connection
   └─> Provider serves GRPC on assigned port

5. Terraform sends GetProviderSchema request
   └─> Provider responds with schema

6. Ready for operations
   └─> Create, Read, Update, Delete resources

The handshake includes a “magic cookie” for security:

var handshake = plugin.HandshakeConfig{
    ProtocolVersion:  6,
    MagicCookieKey:   "TF_PLUGIN_MAGIC_COOKIE",
    MagicCookieValue: "d602bf8f470bc67ca7faa0386276bbdd4330efaf76d1a219cb4d6991ca9872b2",
}

Purpose:

  • Prevents accidental execution
  • Ensures only Terraform can invoke the provider
  • Version negotiation

Communication Flow

Let’s trace a complete operation from Terraform to provider.

Example: Creating a Resource

1. User runs: terraform apply

2. Terraform Core:
   ├─> Starts provider binary
   ├─> Performs handshake
   ├─> Calls GetProviderSchema()
   └─> Receives resource schemas

3. Terraform Core:
   ├─> Parses configuration
   ├─> Generates execution plan
   └─> Calls ValidateResourceConfig()

4. Provider validates:
   ├─> Checks required attributes
   ├─> Validates constraints
   └─> Returns diagnostics

5. Terraform Core:
   └─> Calls ApplyResourceChange() with CREATE action

6. Provider:
   ├─> Extracts configuration
   ├─> Calls external API
   ├─> Creates resource
   ├─> Returns new state
   └─> Includes computed attributes

7. Terraform Core:
   ├─> Receives new state
   ├─> Updates state file
   └─> Continues to next resource

GRPC Service Definition

Simplified Protocol Buffer definition:

service Provider {
  rpc GetProviderSchema(GetProviderSchema.Request) returns (GetProviderSchema.Response);
  rpc ValidateProviderConfig(ValidateProviderConfig.Request) returns (ValidateProviderConfig.Response);
  rpc ConfigureProvider(ConfigureProvider.Request) returns (ConfigureProvider.Response);
  
  rpc ReadResource(ReadResource.Request) returns (ReadResource.Response);
  rpc PlanResourceChange(PlanResourceChange.Request) returns (PlanResourceChange.Response);
  rpc ApplyResourceChange(ApplyResourceChange.Request) returns (ApplyResourceChange.Response);
  
  rpc ReadDataSource(ReadDataSource.Request) returns (ReadDataSource.Response);
  
  rpc StopProvider(StopProvider.Request) returns (StopProvider.Response);
}

Provider Lifecycle

A provider goes through distinct lifecycle phases.

Phase 1: Discovery

terraform init

What happens:

  1. Terraform reads required_providers block
  2. Resolves provider source and version
  3. Downloads provider binary
  4. Installs in plugin cache (~/.terraform.d/plugins/)

Phase 2: Initialization

terraform plan

What happens:

  1. Terraform starts provider binary
  2. Handshake and GRPC setup
  3. GetProviderSchema() called
  4. Provider returns schemas for all resources/data sources

Code:

func (p *MyProvider) Schema(ctx context.Context, req provider.SchemaRequest, resp *provider.SchemaResponse) {
    resp.Schema = schema.Schema{
        Attributes: map[string]schema.Attribute{
            "endpoint": schema.StringAttribute{
                Optional: true,
            },
        },
    }
}

Phase 3: Configuration

What happens:

  1. Terraform calls ConfigureProvider()
  2. Passes provider configuration
  3. Provider initializes (e.g., creates API client)
  4. Stores client in provider instance

Code:

func (p *MyProvider) Configure(ctx context.Context, req provider.ConfigureRequest, resp *provider.ConfigureResponse) {
    var config MyProviderModel
    resp.Diagnostics.Append(req.Config.Get(ctx, &config)...)
    
    // Initialize API client
    client := NewAPIClient(config.Endpoint.ValueString())
    
    // Store for use by resources
    resp.ResourceData = client
}

Phase 4: Resource Operations

What happens:

  1. Terraform calls resource-specific RPCs
  2. Provider performs CRUD operations
  3. Returns updated state
  4. Terraform writes to state file

Phase 5: Shutdown

What happens:

  1. All operations complete
  2. Terraform calls StopProvider()
  3. Provider performs cleanup
  4. Process terminates

Binary Distribution Model

Providers are distributed as compiled binaries, not source code.

Binary Naming Convention

terraform-provider-{NAME}_v{VERSION}_{OS}_{ARCH}

Examples:
terraform-provider-aws_v5.0.0_linux_amd64
terraform-provider-aws_v5.0.0_darwin_arm64
terraform-provider-aws_v5.0.0_windows_amd64.exe

Installation Locations

Registry providers:

~/.terraform.d/plugins/
└── registry.terraform.io/
    └── hashicorp/
        └── aws/
            └── 5.0.0/
                └── linux_amd64/
                    └── terraform-provider-aws_v5.0.0

Local providers (development):

~/.terraform.d/plugins/
└── registry.terraform.io/
    └── YOUR_USERNAME/
        └── mycloud/
            └── 0.1.0/
                └── darwin_arm64/
                    └── terraform-provider-mycloud

Platform Support

Standard targets for multi-platform builds:

# .goreleaser.yml
builds:
  - goos:
      - linux
      - darwin
      - windows
    goarch:
      - amd64
      - arm64

Results in 6 binaries:

  • linux_amd64
  • linux_arm64
  • darwin_amd64 (Intel Mac)
  • darwin_arm64 (M1/M2 Mac)
  • windows_amd64
  • windows_arm64

Process Management

Concurrent Providers

Terraform can run multiple provider instances:

provider "aws" {
  alias  = "us-east-1"
  region = "us-east-1"
}

provider "aws" {
  alias  = "us-west-2"
  region = "us-west-2"
}

Result: Two separate provider processes, one per configuration.

Resource Parallelism

Terraform runs operations in parallel by default:

terraform apply -parallelism=10

Default: 10 concurrent operations

Each operation: Separate GRPC call to provider

Provider must: Handle concurrent requests safely

State Locking

Providers don’t handle state locking directly—Terraform Core does.

But providers should:

  • Be stateless between calls
  • Handle concurrent API requests
  • Implement proper synchronization for shared resources

Error Handling

Providers communicate errors through diagnostics.

Diagnostic Types

Error:

resp.Diagnostics.AddError(
    "Failed to Create Resource",
    fmt.Sprintf("API returned 500: %s", err),
)

Warning:

resp.Diagnostics.AddWarning(
    "Deprecated Attribute",
    "The 'legacy_mode' attribute is deprecated. Use 'mode' instead.",
)

Attribute-Specific:

resp.Diagnostics.AddAttributeError(
    path.Root("endpoint"),
    "Invalid Endpoint",
    "Endpoint must start with https://",
)

User-Facing Errors

Terraform displays diagnostics to users:

╷
│ Error: Failed to Create Resource
│ 
│   with mycloud_environment.example,
│   on main.tf line 10, in resource "mycloud_environment" "example":
│   10: resource "mycloud_environment" "example" {
│ 
│ API returned 500: Internal Server Error
╵

Performance Considerations

Provider Startup Cost

Each provider invocation has overhead:

Startup time breakdown:
- Binary execution: 10-50ms
- GRPC handshake: 5-10ms
- Schema transfer: 10-50ms (depends on schema size)

Total: 25-110ms per provider instance

Mitigation:

  • Terraform reuses provider instances within a single run
  • Minimize provider configurations (aliases)

Schema Size

Large schemas slow initialization:

// ❌ Bad: Massive schema
Attributes: map[string]schema.Attribute{
    "field1": schema.StringAttribute{...},
    "field2": schema.StringAttribute{...},
    // ... 10,000+ attributes
}

// ✅ Good: Nested schemas
Attributes: map[string]schema.Attribute{
    "config": schema.SingleNestedAttribute{
        Attributes: map[string]schema.Attribute{
            "field1": schema.StringAttribute{...},
            // Grouped logically
        },
    },
}

API Rate Limiting

Providers should:

  • Implement retries with exponential backoff
  • Respect API rate limits
  • Use bulk operations when available
  • Cache results when appropriate

Security Model

Process Isolation

Benefits:

  • Provider bugs don’t crash Terraform
  • Compromised provider has limited system access
  • Resource limits can be applied per process

Limitations:

  • Provider still runs with user permissions
  • Can access environment variables
  • Can make arbitrary network calls

Credential Handling

Best practices:

// ✅ Good: Sensitive attribute
"api_key": schema.StringAttribute{
    Optional:  true,
    Sensitive: true,  // Prevents logging
},

// ❌ Bad: Plain text in logs
log.Printf("API Key: %s", apiKey)  // DON'T DO THIS

// ✅ Good: Redacted logging
tflog.Debug(ctx, "Authenticating with API", map[string]interface{}{
    "endpoint": endpoint,
    // api_key automatically redacted if marked sensitive
})

State File Security

Providers should:

  • Mark sensitive attributes as Sensitive: true
  • Never log sensitive data
  • Assume state files may be compromised

Debugging Provider Communication

Enable GRPC Logging

TF_LOG=DEBUG terraform apply

Shows:

2024-01-15T10:30:00.123-0700 [DEBUG] provider: starting plugin: path=.terraform/providers/...
2024-01-15T10:30:00.234-0700 [DEBUG] provider: plugin started: address=/var/tmp/plugin123
2024-01-15T10:30:00.345-0700 [DEBUG] provider: GRPC handshake complete

Provider Debug Mode

Run provider with debugger support:

./terraform-provider-mycloud -debug

Output:

Provider started. To attach Terraform:
  TF_REATTACH_PROVIDERS='{"registry.terraform.io/YOUR_USERNAME/mycloud":{"Protocol":"grpc","ProtocolVersion":6,"Pid":12345,"Test":true,"Addr":{"Network":"unix","String":"/tmp/plugin123"}}}'

Use in Terraform:

export TF_REATTACH_PROVIDERS='...'
terraform apply

Key Takeaways

  • ✅ Providers run as separate processes using GRPC
  • ✅ Protocol version 6 is current (Plugin Framework)
  • ✅ Handshake establishes compatibility
  • ✅ Providers are distributed as compiled binaries
  • ✅ Multiple provider instances can run concurrently
  • ✅ Diagnostics communicate errors to users
  • ✅ Process isolation provides security boundaries
  • ✅ Debugging tools reveal GRPC communication

What’s Next?

Now that you understand how providers work, let’s discuss when you should build a custom provider versus using alternatives.

Next: When to Build a Custom Provider →

Previous: ← Plugin Framework vs SDK

Last updated: June 30, 2026

Updated:

You may also enjoy