GCP for DevOps — Cloud Monitoring

Cloud Monitoring

Cloud Monitoring collects metrics, events, and metadata from GCP services, hosted uptime probes, and custom applications. It provides dashboards, alerting, and uptime monitoring out of the box.

How Monitoring Works

Monitoring Data Flow:
┌──────────────┐    ┌──────────────────┐    ┌──────────────┐
│ GCP Services  │───▶│                  │───▶│ Dashboards    │
│ (auto-metrics)│    │                  │    └──────────────┘
├──────────────┤    │  Cloud            │    ┌──────────────┐
│ Ops Agent     │───▶│  Monitoring      │───▶│ Alerting      │
│ (VM metrics)  │    │                  │    └──────────────┘
├──────────────┤    │  Metrics         │    ┌──────────────┐
│ Custom Metrics│───▶│  Store           │───▶│ MQL Queries   │
│ (app code)    │    │                  │    └──────────────┘
├──────────────┤    │                  │    ┌──────────────┐
│ Uptime Checks │───▶│                  │───▶│ SLO / SLI     │
└──────────────┘    └──────────────────┘    └──────────────┘

Built-in Metrics

GCP services automatically emit metrics. Common metric types:

Service	Metric Example	Description
Compute Engine	compute.googleapis.com/instance/cpu/utilization	CPU usage 0–1
Cloud Run	run.googleapis.com/request_count	Request count
GKE	kubernetes.io/container/cpu/core_usage_time	Container CPU
Cloud SQL	cloudsql.googleapis.com/database/cpu/utilization	DB CPU
Cloud Functions	cloudfunctions.googleapis.com/function/execution_count	Invocations

Install the Ops Agent

The Ops Agent collects system and application metrics/logs from VMs:

# Install Ops Agent on a VM
$ gcloud compute ssh my-vm --command="curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh && sudo bash add-google-cloud-ops-agent-repo.sh --also-install"

# Verify agent status
$ gcloud compute ssh my-vm --command="sudo systemctl status google-cloud-ops-agent"

Custom Metrics

from google.cloud import monitoring_v3
import time

client = monitoring_v3.MetricServiceClient()
project_name = f"projects/my-project"

# Create a custom metric descriptor
descriptor = monitoring_v3.MetricDescriptor()
descriptor.type = "custom.googleapis.com/app/request_latency"
descriptor.metric_kind = monitoring_v3.MetricDescriptor.MetricKind.GAUGE
descriptor.value_type = monitoring_v3.MetricDescriptor.ValueType.DOUBLE
descriptor.unit = "ms"
descriptor.description = "Application request latency"

client.create_metric_descriptor(
    name=project_name, metric_descriptor=descriptor
)

# Write a data point
series = monitoring_v3.TimeSeries()
series.metric.type = "custom.googleapis.com/app/request_latency"
series.resource.type = "global"

now = time.time()
interval = monitoring_v3.TimeInterval(
    {"end_time": {"seconds": int(now)}}
)
point = monitoring_v3.Point(
    {"interval": interval, "value": {"double_value": 42.5}}
)
series.points = [point]

client.create_time_series(
    request={"name": project_name, "time_series": [series]}
)

Dashboards

# Create a dashboard from JSON
$ gcloud monitoring dashboards create \
    --config-from-file=dashboard.json

# List dashboards
$ gcloud monitoring dashboards list

# Describe a dashboard
$ gcloud monitoring dashboards describe DASHBOARD_ID

Example dashboard JSON:

{
  "displayName": "DevOps Overview",
  "gridLayout": {
    "columns": 2,
    "widgets": [
      {
        "title": "VM CPU Utilization",
        "xyChart": {
          "dataSets": [{
            "timeSeriesQuery": {
              "timeSeriesFilter": {
                "filter": "metric.type=\"compute.googleapis.com/instance/cpu/utilization\"",
                "aggregation": {
                  "alignmentPeriod": "60s",
                  "perSeriesAligner": "ALIGN_MEAN"
                }
              }
            },
            "plotType": "LINE"
          }]
        }
      },
      {
        "title": "Cloud Run Request Count",
        "xyChart": {
          "dataSets": [{
            "timeSeriesQuery": {
              "timeSeriesFilter": {
                "filter": "metric.type=\"run.googleapis.com/request_count\"",
                "aggregation": {
                  "alignmentPeriod": "60s",
                  "perSeriesAligner": "ALIGN_RATE"
                }
              }
            },
            "plotType": "STACKED_BAR"
          }]
        }
      }
    ]
  }
}

Uptime Checks

# Create an HTTP uptime check
$ gcloud monitoring uptime create my-app-check \
    --display-name="My App Health Check" \
    --resource-type=uptime-url \
    --hostname=myapp.example.com \
    --path=/health \
    --check-interval=60s \
    --timeout=10s

# List uptime checks
$ gcloud monitoring uptime list-configs

Monitoring Query Language (MQL)

# Average CPU across all VMs
fetch gce_instance
| metric 'compute.googleapis.com/instance/cpu/utilization'
| group_by 1m, [value_utilization_mean: mean(value.utilization)]
| every 1m

# Cloud Run request rate by status code
fetch cloud_run_revision
| metric 'run.googleapis.com/request_count'
| align rate(1m)
| group_by [resource.service_name, metric.response_code_class],
    [value_request_count_aggregate: aggregate(value.request_count)]

Common Mistakes

Mistake	Why It’s Wrong	Fix
No Ops Agent on VMs	Missing memory/disk metrics	Install Ops Agent on all VMs
Too many custom metrics	High cost	Use built-in metrics where possible
No uptime checks	Unaware of outages	Create checks for all critical endpoints
Dashboard without filters	Noisy data	Add project/label filters
Ignoring metric alignment	Inaccurate aggregations	Set proper alignment period

Quick Reference

Command	Description
gcloud monitoring dashboards create	Create dashboard
gcloud monitoring dashboards list	List dashboards
gcloud monitoring uptime create	Create uptime check
gcloud monitoring uptime list-configs	List uptime checks
gcloud monitoring metrics-scopes list	List metrics scopes

Next → Alerting Policies