Skip to content

Expert · CLI tools and Go in cloud and DevOps · topic 42 of 46 · 35 min

Go in cloud and DevOps: Kubernetes, Terraform and Prometheus

Explanation

Most cloud-native infrastructure is written in Go: Docker's engine and containerd, Kubernetes, etcd, Terraform, Prometheus, Helm and many more. Platform and DevOps engineers use Go to write CLIs, Kubernetes controllers (with client-go), Terraform providers and Prometheus exporters. Your Go services fit in with health checks, metrics and Kubernetes manifests.

The cloud-native stack speaks Go

ToolWhat it doesWritten in Go
Docker Engine, containerdBuild and run containersYes
Kubernetes (kube-apiserver, kubelet, kubectl)Run containers across many machinesYes
etcdConsistent key-value store that holds Kubernetes stateYes
TerraformInfrastructure as code: create cloud resources from config filesYes (and its providers)
PrometheusCollect and query metrics, trigger alertsYes
Helm, Argo CDPackage and deploy apps on KubernetesYes
Consul, Vault, CoreDNS, Traefik, CaddyService discovery, secrets, DNS, proxiesYes

Why did this happen? Infrastructure tools must run on many systems (single binary, easy cross-compiling), handle thousands of network connections (goroutines, a strong net/http), start quickly and use little memory, and be maintained by large open-source communities (a simple, readable language). Go fits all of these.

What platform and DevOps engineers write in Go

  • Internal CLIs: deploy helpers, log searchers, certificate checkers.
  • Kubernetes controllers and operators: programs that watch the Kubernetes API and act, for example "when a Database resource is created, provision it". They use client-go, and often controller-runtime or Kubebuilder.
  • Terraform providers: plugins that teach Terraform to manage a new API.
  • Prometheus exporters: small services that expose metrics from a system that does not have them.
  • Glue services: webhooks, schedulers, small APIs between systems.

Making your service cloud-ready

go
mux.HandleFunc("GET /healthz", func(w http.ResponseWriter, r *http.Request) {
	w.WriteHeader(http.StatusOK) // "the process is alive"
})
mux.HandleFunc("GET /readyz", func(w http.ResponseWriter, r *http.Request) {
	ctx, cancel := context.WithTimeout(r.Context(), time.Second)
	defer cancel()
	if err := db.PingContext(ctx); err != nil {
		http.Error(w, "db not ready", http.StatusServiceUnavailable)
		return
	}
	w.WriteHeader(http.StatusOK) // "send me traffic"
})
Kubernetes uses a liveness probe to restart a stuck process and a readiness probe to decide whether to send it traffic.

Metrics with Prometheus

go
import (
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promauto"
	"github.com/prometheus/client_golang/prometheus/promhttp"
)

var ordersTotal = promauto.NewCounterVec(prometheus.CounterOpts{
	Name: "orders_total",
	Help: "Orders placed, by result.",
}, []string{"result"})

// in a handler:
ordersTotal.WithLabelValues("success").Inc()

// expose for Prometheus to scrape:
mux.Handle("GET /metrics", promhttp.Handler())
Prometheus scrapes /metrics every few seconds. Grafana dashboards and alerts are built on these numbers.
Careful: Never use unbounded values (user IDs, order IDs, full URLs) as metric labels. Each distinct value creates a new time series and can overload Prometheus. Use a few fixed values like success/failed.

A Kubernetes Deployment for a Go service

yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: expense-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: expense-api
  template:
    metadata:
      labels:
        app: expense-api
    spec:
      containers:
        - name: api
          image: registry.example.com/expense-api:1.0.0
          ports:
            - containerPort: 8080
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: expense-db
                  key: url
            - name: GOMEMLIMIT
              value: "100MiB"
          livenessProbe:
            httpGet: { path: /healthz, port: 8080 }
          readinessProbe:
            httpGet: { path: /readyz, port: 8080 }
          resources:
            requests: { cpu: 100m, memory: 64Mi }
            limits: { memory: 128Mi }
Three replicas, the database URL from a Kubernetes Secret, probes on the health endpoints, and modest resources: Go services are usually light.

Talking to Kubernetes from Go

go
// sketch using k8s.io/client-go (needs a kubeconfig or in-cluster config)
pods, err := clientset.CoreV1().Pods("default").List(ctx, metav1.ListOptions{})
if err != nil {
	return err
}
for _, p := range pods.Items {
	fmt.Println(p.Name, p.Status.Phase)
}
kubectl itself uses the same client libraries. Setting up the clientset takes a few more lines; see the client-go examples.
Tip: A strong DevOps/SRE profile combines Go with Linux, Docker, Kubernetes, Terraform and one cloud (AWS, Azure or GCP). Writing one small exporter or operator for your portfolio is a great talking point in interviews.

Need help?

Stuck on a word or an idea? Search this course, or every course, and jump straight to the explanation or a worked question.

Try:

Watch the lesson

Go in cloud and DevOps: Kubernetes, Terraform and Prometheus · 1/7

Go in cloud and DevOps

Much of the cloud runs on Go. Let us see why, and what you can build.

Prefer a video by a teacher? Find free videos on “golang for devops kubernetes client-go prometheus exporter tutorial” on YouTube ↗

In real life: A tiny exporter for UPI success rate

A fintech's SRE team wants a Grafana alert when the UPI success rate drops. The payment database has the data but no metrics. An SRE writes a 100-line Go exporter: every 30 seconds it counts successful and failed payments from the last 5 minutes and exposes them as Prometheus gauges. It ships as a 10-20 MB distroless image with one Deployment, and an alert fires when the success ratio drops below a threshold.

go
var successRate = promauto.NewGauge(prometheus.GaugeOpts{
	Name: "upi_success_ratio_5m",
	Help: "Share of UPI payments that succeeded in the last 5 minutes.",
})

func poll(ctx context.Context, db *sql.DB) {
	t := time.NewTicker(30 * time.Second)
	defer t.Stop()
	for {
		var ok, total float64
		err := db.QueryRowContext(ctx, `
			SELECT count(*) FILTER (WHERE status = 'SUCCESS'), count(*)
			FROM payments WHERE created_at > now() - interval '5 minutes'`).Scan(&ok, &total)
		if err == nil && total > 0 {
			successRate.Set(ok / total)
		}
		select {
		case <-ctx.Done():
			return
		case <-t.C:
		}
	}
}
A for-select loop with a ticker and context, a PostgreSQL query, and one gauge. Everything from this course in one small tool.

Practice (0/4 done)

Task 1 · Question

Which of these is NOT written in Go?

Task 2 · Question

What is the difference between a liveness and a readiness probe?

Task 3 · Question

Which is a BAD Prometheus label?

Task 4 · Practice

Add /healthz, /readyz (pinging the database) and /metrics to your expense API. Count created expenses with a Prometheus counter labelled by category (a small fixed list). Write a Kubernetes Deployment YAML with 2 replicas, probes, resources and DATABASE_URL from a Secret. If you have kind or minikube, apply it.

Quick quiz

Question 1

1.What is client-go?

Question 2

2.What does a Kubernetes operator do?

Question 3

3.Why do ops teams like Go tools?

Question 4

4.How does Prometheus get metrics from a Go service?

Read more (free)

Open a question to see the answer and why. The first ones are this topic's own quiz, so try the quiz above before you peek.

What is client-go?

Answer: The Go client library for talking to the Kubernetes API

Controllers, operators and kubectl use it.

What does a Kubernetes operator do?

Answer: Watches the Kubernetes API and automatically manages an application or resource

Operators encode operational knowledge as a control loop.

Why do ops teams like Go tools?

Answer: One static binary per platform, quick start, low memory

Easy distribution and low resource use matter for infrastructure.

How does Prometheus get metrics from a Go service?

Answer: Prometheus scrapes an HTTP endpoint such as /metrics

Prometheus pulls metrics over HTTP on a schedule.

Why do DevOps teams like Go programs?Related

Answer: They ship as one binary that is easy to copy and containerise

A single static binary is easy to deploy, version and put in a small container image.

From What Go is and why people use it

Which metric type fits "number of items waiting in the email queue right now"?Related

Answer: Gauge

The queue length goes up and down, so it is a gauge.

From Prometheus and Grafana

What is an artefact in a DevOps pipeline?Related

Answer: A file produced by the build, such as a .jar or container image, that is stored and deployed

Artefacts are build outputs. Storing them means you deploy exactly what was tested.

From The DevOps toolchain and your learning roadmap

To earn Microsoft's DevOps Engineer Expert certification, what do you need besides passing AZ-400?Related

Answer: AZ-104 or AZ-204

The expert certification needs AZ-400 plus either the Administrator (AZ-104) or Developer (AZ-204) associate certification.

From DevOps certifications: which, when and how

Pass the quiz (60% or more) or mark this topic as done when you are ready.

Your progress is saved on this device only.

Next steps

  1. This is the last topic of “CLI tools and Go in cloud and DevOps”. The module exam comes next.
  2. Module 9 exam: CLI tools and Go in cloud and DevOps · 12 questions · 20 min, without help. 4 topics of this module still to finish first.
  3. Then the next module, Careers and interviews. 46 topics left in the course before the final exam.

← Previous: Docker images for Go services