Module 9: API Client Integration

5 minute read

Module Overview

Your Terraform provider is only as reliable as the API client it uses. This module teaches you to build production-grade API clients that handle authentication, retries, rate limiting, and errors gracefully.

What You’ll Learn:

  • API client architecture patterns
  • HTTP client configuration and best practices
  • Authentication methods (API keys, OAuth, tokens)
  • Comprehensive error handling
  • Rate limiting and exponential backoff
  • Testing API clients
  • Real-world integration patterns

Time: 4-5 hours

Why API Client Design Matters

The API client is the bridge between your provider and the cloud service:

┌─────────────────────────────────────────────────────┐
│            Provider Architecture                     │
├─────────────────────────────────────────────────────┤
│                                                      │
│  User Config → Provider → API Client → Cloud API   │
│                    ↑           ↑                     │
│                    │           │                     │
│           Schema/Resources   HTTP/Auth/Retry        │
│           State Management   Error Handling         │
│                                                      │
└─────────────────────────────────────────────────────┘

A good API client:

  • Handles authentication automatically
  • Retries transient failures
  • Respects rate limits
  • Provides clear error messages
  • Logs requests for debugging
  • Supports testing with mocks

A poor API client:

  • Fails on transient errors
  • Exposes raw HTTP errors
  • Doesn’t handle rate limits
  • Hard to test
  • No logging or debugging support

API Client Responsibilities

1. Authentication

Handle all authentication methods:

type Client struct {
    httpClient *http.Client
    apiKey     string
    baseURL    string
}

func (c *Client) makeRequest(ctx context.Context, method, path string, body interface{}) (*Response, error) {
    // Add authentication
    req.Header.Set("Authorization", "Bearer "+c.apiKey)
    
    // Make request
    return c.do(req)
}

2. Request Construction

Build correct requests:

func (c *Client) CreateDatabase(ctx context.Context, input CreateDatabaseInput) (*Database, error) {
    // Construct request
    body := map[string]interface{}{
        "name":       input.Name,
        "engine":     input.Engine,
        "storage_gb": input.StorageGB,
    }
    
    // POST /api/v2/databases
    resp, err := c.post(ctx, "/databases", body)
    
    // Parse response
    var db Database
    json.Unmarshal(resp.Body, &db)
    
    return &db, nil
}

3. Error Handling

Translate API errors to meaningful messages:

func (c *Client) do(req *http.Request) (*Response, error) {
    resp, err := c.httpClient.Do(req)
    
    if resp.StatusCode == 404 {
        return nil, ErrNotFound
    }
    
    if resp.StatusCode == 429 {
        return nil, ErrRateLimited
    }
    
    if resp.StatusCode >= 500 {
        return nil, fmt.Errorf("API error: %s", resp.Status)
    }
    
    return resp, nil
}

4. Retry Logic

Retry transient failures:

func (c *Client) do(req *http.Request) (*Response, error) {
    var resp *http.Response
    var err error
    
    for attempt := 0; attempt < maxRetries; attempt++ {
        resp, err = c.httpClient.Do(req)
        
        if err == nil && resp.StatusCode < 500 {
            break
        }
        
        // Exponential backoff
        time.Sleep(time.Duration(math.Pow(2, float64(attempt))) * time.Second)
    }
    
    return resp, err
}

5. Rate Limiting

Respect API rate limits:

type Client struct {
    httpClient  *http.Client
    rateLimiter *rate.Limiter
}

func (c *Client) do(req *http.Request) (*Response, error) {
    // Wait for rate limit
    err := c.rateLimiter.Wait(req.Context())
    if err != nil {
        return nil, err
    }
    
    // Make request
    return c.httpClient.Do(req)
}

Common API Client Patterns

Pattern 1: Simple REST Client

type Client struct {
    httpClient *http.Client
    apiKey     string
    baseURL    string
}

func (c *Client) GetDatabase(ctx context.Context, id string) (*Database, error) {
    url := fmt.Sprintf("%s/databases/%s", c.baseURL, id)
    
    req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
    req.Header.Set("Authorization", "Bearer "+c.apiKey)
    
    resp, err := c.httpClient.Do(req)
    if err != nil {
        return nil, err
    }
    defer resp.Body.Close()
    
    var db Database
    json.NewDecoder(resp.Body).Decode(&db)
    
    return &db, nil
}

Pattern 2: SDK-Based Client

import "github.com/cloud-provider/sdk-go"

type Client struct {
    sdk *sdk.Client
}

func NewClient(apiKey string) (*Client, error) {
    sdkClient, err := sdk.NewClient(&sdk.Config{
        APIKey: apiKey,
        Region: "us-east-1",
    })
    
    return &Client{sdk: sdkClient}, nil
}

func (c *Client) GetDatabase(ctx context.Context, id string) (*Database, error) {
    // Use SDK
    output, err := c.sdk.GetDatabase(&sdk.GetDatabaseInput{
        ID: id,
    })
    
    // Convert SDK types to provider types
    db := &Database{
        ID:     *output.ID,
        Name:   *output.Name,
        Engine: *output.Engine,
    }
    
    return db, nil
}

Pattern 3: GraphQL Client

type Client struct {
    graphql *graphql.Client
}

func (c *Client) GetDatabase(ctx context.Context, id string) (*Database, error) {
    query := `
        query GetDatabase($id: ID!) {
            database(id: $id) {
                id
                name
                engine
                status
            }
        }
    `
    
    var response struct {
        Database Database `json:"database"`
    }
    
    err := c.graphql.Query(ctx, query, map[string]interface{}{
        "id": id,
    }, &response)
    
    return &response.Database, err
}

Module Chapters

Chapter 1: API Client Architecture

Design patterns for API clients, interface design, and separation of concerns.

Chapter 2: HTTP Client Configuration

Configure HTTP clients with timeouts, keep-alive, TLS, and custom transports.

Chapter 3: Authentication Patterns

Implement API key, OAuth, JWT, and certificate-based authentication.

Chapter 4: Error Handling

Handle API errors, create custom error types, and provide meaningful messages.

Chapter 5: Rate Limiting & Retries

Implement rate limiting, exponential backoff, and intelligent retry logic.

Chapter 6: API Client Lab

Hands-on: Build a complete API client with authentication, retries, and error handling.

Chapter 7: API Client Practice

Test your knowledge with comprehensive practice scenarios.

Learning Path

Architecture → HTTP Config → Auth → Errors → Retries → Lab
     ↓             ↓           ↓        ↓        ↓       ↓
   Design      Timeouts    API Keys  Custom  Backoff  Practice
   Patterns     TLS        OAuth     Types   Limits

What You’ll Build

By the end of this module, you’ll have a production-ready API client:

Client Interface:

type Client interface {
    GetDatabase(ctx context.Context, id string) (*Database, error)
    CreateDatabase(ctx context.Context, input CreateDatabaseInput) (*Database, error)
    UpdateDatabase(ctx context.Context, id string, input UpdateDatabaseInput) (*Database, error)
    DeleteDatabase(ctx context.Context, id string) error
    ListDatabases(ctx context.Context) ([]Database, error)
}

Implementation with:

  • ✅ Automatic authentication
  • ✅ Exponential backoff retries
  • ✅ Rate limiting
  • ✅ Custom error types
  • ✅ Request/response logging
  • ✅ Context cancellation support
  • ✅ Testable with mocks
  • ✅ TLS configuration

Key Concepts

HTTP Client: Configurable client for making HTTP requests
Authentication: Proving identity to API (API key, OAuth, JWT)
Rate Limiting: Controlling request frequency
Exponential Backoff: Increasing wait time between retries
Circuit Breaker: Preventing cascading failures
Idempotency: Safe to retry operations
Context: Cancellation and timeout management

Best Practices Preview

Throughout this module, you’ll learn:

  • ✅ Use context for cancellation and timeouts
  • ✅ Implement exponential backoff for retries
  • ✅ Respect API rate limits
  • ✅ Create custom error types
  • ✅ Log requests for debugging
  • ✅ Make clients testable with interfaces
  • ✅ Configure reasonable timeouts
  • ✅ Handle authentication transparently
  • ✅ Support user-agent headers
  • ✅ Use connection pooling

Common Pitfalls to Avoid

  • ❌ No retry logic for transient failures
  • ❌ Exposing raw HTTP errors to users
  • ❌ Not handling rate limits
  • ❌ Infinite retries without backoff
  • ❌ No timeout configuration
  • ❌ Ignoring context cancellation
  • ❌ Hard-coding credentials
  • ❌ Not logging for debugging
  • ❌ Untestable API clients
  • ❌ Not reusing HTTP connections

Real-World Examples

AWS SDK Pattern:

import "github.com/aws/aws-sdk-go-v2/service/rds"

type Client struct {
    rds *rds.Client
}

func (c *Client) GetDatabase(ctx context.Context, id string) (*Database, error) {
    output, err := c.rds.DescribeDBInstances(ctx, &rds.DescribeDBInstancesInput{
        DBInstanceIdentifier: &id,
    })
    
    // Convert AWS types to provider types
    return convertAWSDatabase(output.DBInstances[0]), nil
}

Custom REST API:

type Client struct {
    httpClient *http.Client
    baseURL    string
    apiKey     string
    retrier    *retrier.Retrier
}

func (c *Client) CreateDatabase(ctx context.Context, input CreateDatabaseInput) (*Database, error) {
    var db Database
    
    err := c.retrier.Do(ctx, func() error {
        resp, err := c.post(ctx, "/databases", input)
        if err != nil {
            return err
        }
        
        return json.Unmarshal(resp, &db)
    })
    
    return &db, err
}

Prerequisites

Before starting this module, you should have completed:

  • ✅ Module 6: Resource Implementation
  • ✅ Module 8: State Management

Go knowledge needed:

  • HTTP client usage
  • Context handling
  • Error handling
  • Interfaces and testing

Get Started

Ready to build a robust API client?

Start with API Client Architecture →

Previous: ← State Management Practice

Last updated: July 2, 2026

Updated:

You may also enjoy