Terraform Provider Development: Building Custom Providers with the Plugin Framework

Posted: June 25, 2026 | Last updated: June 25, 2026 | 10 minute read

Terraform Provider Development: Building custom Terraform providers with Plugin Framework, Go code, and infrastructure API integration

Picture this: Your company has an internal platform API that provisions dev environments, manages feature flags, and configures service mesh policies. Your infrastructure team manually calls these APIs through curl scripts scattered across Slack threads and runbooks. Someone suggests “just Terraform it,” but there’s no official provider. You’re stuck clicking through UIs, running bash scripts, and praying nobody typos a production environment name.

This is exactly when you need a custom Terraform provider. And thanks to HashiCorp’s Plugin Framework (introduced to replace the older SDK), building one is less painful than you’d think.

What Is a Terraform Provider?

A Terraform provider is a plugin that translates Terraform’s declarative configuration into API calls against a real system—cloud platforms (AWS, Azure), SaaS tools (Datadog, PagerDuty), or your own internal APIs.

Why Build Custom Providers:

Manage internal platforms as code: Treat your homegrown APIs with the same IaC discipline as AWS resources.
Eliminate manual toil: Replace curl scripts and UI clicking with terraform apply.
Enable self-service infrastructure: Let developers provision their own environments through Git pull requests.
Standardize configurations: Prevent drift by enforcing configuration through code review.
Integrate with existing workflows: Use Terraform’s state management, drift detection, and plan/apply flow for custom resources.
Type safety and validation: Provider schemas catch errors at plan time, not runtime.

Provider Architecture Overview

graph LR
    A[Terraform CLI] -->|gRPC| B[Provider Plugin]
    B -->|CRUD Operations| C[Target API]
    B -->|Schema & Validation| D[Resource Definitions]
    D -->|State Management| E[Terraform State]

Key components:

Provider: Configures authentication and connection to the target API
Resources: Entities that can be created, read, updated, deleted (CRUD)
Data Sources: Read-only queries that fetch existing data
Schemas: Type definitions and validation rules for each resource

Setting Up Your Development Environment

Prerequisites

# Install Go 1.21+
brew install go

# Install Terraform 1.5+
brew install terraform

# Verify versions
go version  # go version go1.21.0 darwin/arm64
terraform version  # Terraform v1.5.7

# Create project structure
mkdir terraform-provider-myapi
cd terraform-provider-myapi

# Initialize Go module
go mod init github.com/yourorg/terraform-provider-myapi

Install HashiCorp’s Plugin Framework

// go.mod
module github.com/yourorg/terraform-provider-myapi

go 1.21

require (
    github.com/hashicorp/terraform-plugin-framework v1.4.2
    github.com/hashicorp/terraform-plugin-go v0.19.0
    github.com/hashicorp/terraform-plugin-log v0.9.0
)

Run go mod tidy to download dependencies.

Building Your First Provider

Step 1: Define the Provider Configuration

// internal/provider/provider.go
package provider

import (
    "context"
    "os"

    "github.com/hashicorp/terraform-plugin-framework/datasource"
    "github.com/hashicorp/terraform-plugin-framework/provider"
    "github.com/hashicorp/terraform-plugin-framework/provider/schema"
    "github.com/hashicorp/terraform-plugin-framework/resource"
    "github.com/hashicorp/terraform-plugin-framework/types"
)

// Ensure the implementation satisfies the provider.Provider interface
var _ provider.Provider = &MyAPIProvider{}

type MyAPIProvider struct {
    version string
}

// Provider configuration model
type MyAPIProviderModel struct {
    Endpoint types.String `tfsdk:"endpoint"`
    APIKey   types.String `tfsdk:"api_key"`
}

func New(version string) func() provider.Provider {
    return func() provider.Provider {
        return &MyAPIProvider{
            version: version,
        }
    }
}

func (p *MyAPIProvider) Metadata(ctx context.Context, req provider.MetadataRequest, resp *provider.MetadataResponse) {
    resp.TypeName = "myapi"
    resp.Version = p.version
}

func (p *MyAPIProvider) Schema(ctx context.Context, req provider.SchemaRequest, resp *provider.SchemaResponse) {
    resp.Schema = schema.Schema{
        Attributes: map[string]schema.Attribute{
            "endpoint": schema.StringAttribute{
                MarkdownDescription: "MyAPI endpoint URL",
                Optional:            true,
            },
            "api_key": schema.StringAttribute{
                MarkdownDescription: "API key for authentication",
                Optional:            true,
                Sensitive:           true,
            },
        },
    }
}

func (p *MyAPIProvider) Configure(ctx context.Context, req provider.ConfigureRequest, resp *provider.ConfigureResponse) {
    var config MyAPIProviderModel

    resp.Diagnostics.Append(req.Config.Get(ctx, &config)...)

    if resp.Diagnostics.HasError() {
        return
    }

    // Allow environment variable overrides
    endpoint := os.Getenv("MYAPI_ENDPOINT")
    apiKey := os.Getenv("MYAPI_API_KEY")

    if !config.Endpoint.IsNull() {
        endpoint = config.Endpoint.ValueString()
    }

    if !config.APIKey.IsNull() {
        apiKey = config.APIKey.ValueString()
    }

    if endpoint == "" {
        resp.Diagnostics.AddError(
            "Missing API Endpoint",
            "The provider cannot create the API client without an endpoint.",
        )
    }

    if apiKey == "" {
        resp.Diagnostics.AddError(
            "Missing API Key",
            "The provider cannot create the API client without an API key.",
        )
    }

    if resp.Diagnostics.HasError() {
        return
    }

    // Create API client and store in provider data
    client := &APIClient{
        Endpoint: endpoint,
        APIKey:   apiKey,
    }

    resp.DataSourceData = client
    resp.ResourceData = client
}

func (p *MyAPIProvider) Resources(ctx context.Context) []func() resource.Resource {
    return []func() resource.Resource{
        NewEnvironmentResource,
    }
}

func (p *MyAPIProvider) DataSources(ctx context.Context) []func() datasource.DataSource {
    return []func() datasource.DataSource{}
}

Step 2: Create a Resource

// internal/provider/environment_resource.go
package provider

import (
    "context"
    "fmt"

    "github.com/hashicorp/terraform-plugin-framework/resource"
    "github.com/hashicorp/terraform-plugin-framework/resource/schema"
    "github.com/hashicorp/terraform-plugin-framework/resource/schema/planmodifier"
    "github.com/hashicorp/terraform-plugin-framework/resource/schema/stringplanmodifier"
    "github.com/hashicorp/terraform-plugin-framework/types"
)

var _ resource.Resource = &EnvironmentResource{}

func NewEnvironmentResource() resource.Resource {
    return &EnvironmentResource{}
}

type EnvironmentResource struct {
    client *APIClient
}

type EnvironmentResourceModel struct {
    ID       types.String `tfsdk:"id"`
    Name     types.String `tfsdk:"name"`
    Type     types.String `tfsdk:"type"`
    Region   types.String `tfsdk:"region"`
    Tags     types.Map    `tfsdk:"tags"`
}

func (r *EnvironmentResource) Metadata(ctx context.Context, req resource.MetadataRequest, resp *resource.MetadataResponse) {
    resp.TypeName = req.ProviderTypeName + "_environment"
}

func (r *EnvironmentResource) Schema(ctx context.Context, req resource.SchemaRequest, resp *resource.SchemaResponse) {
    resp.Schema = schema.Schema{
        MarkdownDescription: "Manages development environment resources",
        Attributes: map[string]schema.Attribute{
            "id": schema.StringAttribute{
                Computed:            true,
                MarkdownDescription: "Environment identifier",
                PlanModifiers: []planmodifier.String{
                    stringplanmodifier.UseStateForUnknown(),
                },
            },
            "name": schema.StringAttribute{
                MarkdownDescription: "Environment name",
                Required:            true,
            },
            "type": schema.StringAttribute{
                MarkdownDescription: "Environment type (dev, staging, prod)",
                Required:            true,
            },
            "region": schema.StringAttribute{
                MarkdownDescription: "Deployment region",
                Optional:            true,
                Computed:            true,
            },
            "tags": schema.MapAttribute{
                MarkdownDescription: "Key-value tags",
                Optional:            true,
                ElementType:         types.StringType,
            },
        },
    }
}

func (r *EnvironmentResource) Configure(ctx context.Context, req resource.ConfigureRequest, resp *resource.ConfigureResponse) {
    if req.ProviderData == nil {
        return
    }

    client, ok := req.ProviderData.(*APIClient)
    if !ok {
        resp.Diagnostics.AddError(
            "Unexpected Resource Configure Type",
            fmt.Sprintf("Expected *APIClient, got: %T", req.ProviderData),
        )
        return
    }

    r.client = client
}

func (r *EnvironmentResource) Create(ctx context.Context, req resource.CreateRequest, resp *resource.CreateResponse) {
    var data EnvironmentResourceModel

    resp.Diagnostics.Append(req.Plan.Get(ctx, &data)...)
    if resp.Diagnostics.HasError() {
        return
    }

    // Call your API to create the environment
    env, err := r.client.CreateEnvironment(ctx, EnvironmentRequest{
        Name:   data.Name.ValueString(),
        Type:   data.Type.ValueString(),
        Region: data.Region.ValueString(),
    })

    if err != nil {
        resp.Diagnostics.AddError("API Error", fmt.Sprintf("Failed to create environment: %s", err))
        return
    }

    // Save to state
    data.ID = types.StringValue(env.ID)
    data.Region = types.StringValue(env.Region)

    resp.Diagnostics.Append(resp.State.Set(ctx, &data)...)
}

func (r *EnvironmentResource) Read(ctx context.Context, req resource.ReadRequest, resp *resource.ReadResponse) {
    var data EnvironmentResourceModel

    resp.Diagnostics.Append(req.State.Get(ctx, &data)...)
    if resp.Diagnostics.HasError() {
        return
    }

    env, err := r.client.GetEnvironment(ctx, data.ID.ValueString())
    if err != nil {
        resp.Diagnostics.AddError("API Error", fmt.Sprintf("Failed to read environment: %s", err))
        return
    }

    data.Name = types.StringValue(env.Name)
    data.Type = types.StringValue(env.Type)
    data.Region = types.StringValue(env.Region)

    resp.Diagnostics.Append(resp.State.Set(ctx, &data)...)
}

func (r *EnvironmentResource) Update(ctx context.Context, req resource.UpdateRequest, resp *resource.UpdateResponse) {
    var data EnvironmentResourceModel

    resp.Diagnostics.Append(req.Plan.Get(ctx, &data)...)
    if resp.Diagnostics.HasError() {
        return
    }

    _, err := r.client.UpdateEnvironment(ctx, data.ID.ValueString(), EnvironmentRequest{
        Name:   data.Name.ValueString(),
        Type:   data.Type.ValueString(),
        Region: data.Region.ValueString(),
    })

    if err != nil {
        resp.Diagnostics.AddError("API Error", fmt.Sprintf("Failed to update environment: %s", err))
        return
    }

    resp.Diagnostics.Append(resp.State.Set(ctx, &data)...)
}

func (r *EnvironmentResource) Delete(ctx context.Context, req resource.DeleteRequest, resp *resource.DeleteResponse) {
    var data EnvironmentResourceModel

    resp.Diagnostics.Append(req.State.Get(ctx, &data)...)
    if resp.Diagnostics.HasError() {
        return
    }

    err := r.client.DeleteEnvironment(ctx, data.ID.ValueString())
    if err != nil {
        resp.Diagnostics.AddError("API Error", fmt.Sprintf("Failed to delete environment: %s", err))
        return
    }
}

Step 3: Implement the API Client

// internal/provider/client.go
package provider

import (
    "bytes"
    "context"
    "encoding/json"
    "fmt"
    "io"
    "net/http"
)

type APIClient struct {
    Endpoint   string
    APIKey     string
    HTTPClient *http.Client
}

type EnvironmentRequest struct {
    Name   string `json:"name"`
    Type   string `json:"type"`
    Region string `json:"region"`
}

type Environment struct {
    ID     string `json:"id"`
    Name   string `json:"name"`
    Type   string `json:"type"`
    Region string `json:"region"`
}

func (c *APIClient) CreateEnvironment(ctx context.Context, req EnvironmentRequest) (*Environment, error) {
    body, _ := json.Marshal(req)
    
    httpReq, err := http.NewRequestWithContext(ctx, "POST", c.Endpoint+"/environments", bytes.NewBuffer(body))
    if err != nil {
        return nil, err
    }

    httpReq.Header.Set("Authorization", "Bearer "+c.APIKey)
    httpReq.Header.Set("Content-Type", "application/json")

    resp, err := c.HTTPClient.Do(httpReq)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusCreated {
        bodyBytes, _ := io.ReadAll(resp.Body)
        return nil, fmt.Errorf("API returned %d: %s", resp.StatusCode, string(bodyBytes))
    }

    var env Environment
    if err := json.NewDecoder(resp.Body).Decode(&env); err != nil {
        return nil, err
    }

    return &env, nil
}

func (c *APIClient) GetEnvironment(ctx context.Context, id string) (*Environment, error) {
    httpReq, err := http.NewRequestWithContext(ctx, "GET", c.Endpoint+"/environments/"+id, nil)
    if err != nil {
        return nil, err
    }

    httpReq.Header.Set("Authorization", "Bearer "+c.APIKey)

    resp, err := c.HTTPClient.Do(httpReq)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    if resp.StatusCode == http.StatusNotFound {
        return nil, fmt.Errorf("environment %s not found", id)
    }

    var env Environment
    if err := json.NewDecoder(resp.Body).Decode(&env); err != nil {
        return nil, err
    }

    return &env, nil
}

func (c *APIClient) UpdateEnvironment(ctx context.Context, id string, req EnvironmentRequest) (*Environment, error) {
    body, _ := json.Marshal(req)
    
    httpReq, err := http.NewRequestWithContext(ctx, "PUT", c.Endpoint+"/environments/"+id, bytes.NewBuffer(body))
    if err != nil {
        return nil, err
    }

    httpReq.Header.Set("Authorization", "Bearer "+c.APIKey)
    httpReq.Header.Set("Content-Type", "application/json")

    resp, err := c.HTTPClient.Do(httpReq)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()

    var env Environment
    if err := json.NewDecoder(resp.Body).Decode(&env); err != nil {
        return nil, err
    }

    return &env, nil
}

func (c *APIClient) DeleteEnvironment(ctx context.Context, id string) error {
    httpReq, err := http.NewRequestWithContext(ctx, "DELETE", c.Endpoint+"/environments/"+id, nil)
    if err != nil {
        return err
    }

    httpReq.Header.Set("Authorization", "Bearer "+c.APIKey)

    resp, err := c.HTTPClient.Do(httpReq)
    if err != nil {
        return err
    }
    defer resp.Body.Close()

    if resp.StatusCode != http.StatusNoContent && resp.StatusCode != http.StatusOK {
        return fmt.Errorf("failed to delete environment, status: %d", resp.StatusCode)
    }

    return nil
}

Step 4: Create the Main Entry Point

// main.go
package main

import (
    "context"
    "flag"
    "log"

    "github.com/hashicorp/terraform-plugin-framework/providerserver"
    "github.com/yourorg/terraform-provider-myapi/internal/provider"
)

var version string = "dev"

func main() {
    var debug bool

    flag.BoolVar(&debug, "debug", false, "set to true to run the provider with support for debuggers")
    flag.Parse()

    opts := providerserver.ServeOpts{
        Address: "registry.terraform.io/yourorg/myapi",
        Debug:   debug,
    }

    err := providerserver.Serve(context.Background(), provider.New(version), opts)
    if err != nil {
        log.Fatal(err.Error())
    }
}

Real-World Example: Complete Configuration

# main.tf
terraform {
  required_providers {
    myapi = {
      source  = "yourorg/myapi"
      version = "~> 1.0"
    }
  }
}

provider "myapi" {
  endpoint = "https://api.company.internal"
  api_key  = var.api_key  # Or use MYAPI_API_KEY env var
}

resource "myapi_environment" "dev" {
  name   = "feature-123-dev"
  type   = "development"
  region = "us-west-2"
  
  tags = {
    team    = "platform"
    project = "feature-123"
    cost_center = "engineering"
  }
}

resource "myapi_environment" "staging" {
  name   = "feature-123-staging"
  type   = "staging"
  region = "us-west-2"
  
  tags = {
    team    = "platform"
    project = "feature-123"
  }
}

output "dev_environment_id" {
  value = myapi_environment.dev.id
}

Usage:

# Initialize and install the provider
terraform init

# Preview changes
terraform plan

# Apply configuration
terraform apply

# Output: dev_environment_id = "env-abc123"

# Clean up
terraform destroy

Best Practices

Use context-aware HTTP clients — Respect Terraform’s context cancellation for long-running API calls. Always use http.NewRequestWithContext(ctx, ...).
Implement proper error handling — Return detailed error messages with diagnostics. Users should understand WHY the API call failed, not just that it failed.
Add comprehensive schema validation — Use Validators to catch invalid configurations at plan time: stringvalidator.OneOf("dev", "staging", "prod").
Handle API idempotency — Ensure Create operations handle “already exists” responses gracefully. Update operations should be idempotent.
Implement retries with exponential backoff — APIs fail. Wrap HTTP calls with retry logic for transient errors (429, 503).
Use plan modifiers wisely — For computed attributes that shouldn’t force replacement, use UseStateForUnknown(). For attributes that require resource recreation, use RequiresReplace().
Write acceptance tests — Use Terraform’s testing framework to validate CRUD operations against a test API or mock server.
Version your provider — Follow semantic versioning. Breaking schema changes require major version bumps.

Conclusion

Building custom Terraform providers unlocks IaC for your internal platforms, eliminating manual API calls and bringing consistency to infrastructure provisioning. The Plugin Framework’s type-safe approach catches errors at compile time, and its declarative nature makes provider development far more approachable than the old SDK.

Key Takeaways:

Custom providers extend Terraform to manage any API, internal or external
HashiCorp’s Plugin Framework uses Go with strong typing and schema validation
Provider implements Configure, Resources, and DataSources methods
Resources need Create, Read, Update, Delete (CRUD) operations
Use context-aware HTTP clients and proper error handling for production readiness

Start with a single resource for your most-used internal API. As your team adopts it, expand to cover more resources. The infrastructure-as-code mindset spreads faster when you meet developers where they are—managing everything through terraform apply.

Frequently Asked Questions

Q: Do I need to publish my provider to the Terraform Registry?

No. For internal use, you can install providers locally or host them in a private registry. Place the compiled binary in ~/.terraform.d/plugins/registry.terraform.io/yourorg/myapi/1.0.0/<OS_ARCH>/ and Terraform will find it during terraform init. For team distribution, use Terraform Cloud’s private registry or a self-hosted artifact repository.

Q: How do I handle authentication with multiple methods (API keys, OAuth, service accounts)?

Add multiple optional authentication attributes to your provider schema and implement fallback logic in the Configure method. Example: check for api_key first, then oauth_token, then service_account_path. Document the precedence order clearly. You can also support multiple provider aliases for different authentication methods simultaneously.

Q: What’s the difference between the Plugin Framework and the older SDK?

The Plugin Framework (v1.0+, released 2022) uses protocol version 6 with better type safety, clearer error messages, and first-class support for nested attributes and plan modifiers. The older SDK uses protocol version 5 and relies heavily on map[string]interface{} which loses compile-time type checking. New providers should use the Plugin Framework; the SDK is in maintenance mode.

Q: How do I test my provider without calling the real API?

Use Go’s httptest package to create a mock HTTP server that returns canned responses. In your tests, configure the provider to point at the mock server endpoint instead of production. For more comprehensive testing, use Terraform’s acceptance testing framework with the TF_ACC=1 environment variable to run full CRUD lifecycle tests against a sandbox API environment.

Q: Can I use a custom provider with Terraform Cloud or Spacelift?

Yes. Terraform Cloud supports custom providers if you publish them to a private registry (either Terraform Cloud’s registry or your own). Upload the provider binary and configure your workspace to use it. For self-hosted Terraform Enterprise, you have even more flexibility with airgapped installations. Spacelift supports custom providers through its worker pool configuration—just ensure the provider binary is available in the worker’s plugin directory.

Share on

Twitter Facebook LinkedIn

Sushil Verma