Benchmarks and fuzz tests

Measure, do not guess

Developers often guess wrongly about what is slow. A benchmark runs a piece of code many times and reports the average time per run. Go's testing package has benchmarks built in, so you can compare two versions of a function with real numbers.

// in a _test.go file
func BenchmarkJoinPlus(b *testing.B) {
	for i := 0; i < b.N; i++ {
		s := ""
		for j := 0; j < 100; j++ {
			s += "x"
		}
	}
}

func BenchmarkJoinBuilder(b *testing.B) {
	for i := 0; i < b.N; i++ {
		var sb strings.Builder
		for j := 0; j < 100; j++ {
			sb.WriteString("x")
		}
		_ = sb.String()
	}
}

go test -bench=. -benchmem
# BenchmarkJoinPlus-8       <runs>   <time> ns/op   <bytes> B/op   <many> allocs/op
# BenchmarkJoinBuilder-8    <runs>   <time> ns/op   <bytes> B/op   <few> allocs/op
go test -run='^$' -bench=Builder   # skip normal tests, only matching benchmarks

The newer b.Loop style (Go 1.24+)

Go 1.24 added b.Loop(). You write for b.Loop() { ... } instead of the b.N loop. It handles timing set-up for you and stops the compiler from optimising away the work being measured. Older code and older Go versions use the b.N loop, so you should recognise both.

func BenchmarkWithGST(b *testing.B) {
	for b.Loop() {
		WithGST(10000)
	}
}

Reading benchmark results honestly

• Run on a quiet machine; other programs add noise.
• Run several times (-count=10) and compare with the benchstat tool (golang.org/x/perf/cmd/benchstat) rather than one run.
• Benchmark realistic input sizes. Code that is fast for 10 items may be slow for 10 lakh.
• A function that is 2x faster but runs once a day does not matter. Optimise what profiling shows is hot (the pprof profiler is the next step).

Fuzzing: let the computer find your bugs

Your table tests contain cases you thought of. Fuzzing (added in Go 1.18) generates thousands of inputs you did not think of: odd Unicode, empty strings, huge numbers. You give a few seed inputs and a property that must always hold. When the fuzzer finds an input that fails or panics, it saves it under testdata/fuzz/ so it becomes a permanent regression test.

func Reverse(s string) string {
	r := []rune(s)
	for i, j := 0, len(r)-1; i < j; i, j = i+1, j-1 {
		r[i], r[j] = r[j], r[i]
	}
	return string(r)
}

func FuzzReverse(f *testing.F) {
	f.Add("hello")   // seed corpus
	f.Add("नमस्ते")
	f.Fuzz(func(t *testing.T, s string) {
		twice := Reverse(Reverse(s))
		if utf8.ValidString(s) && twice != s {
			t.Errorf("Reverse twice changed %q into %q", s, twice)
		}
	})
}

go test                         # runs seeds only, like a normal test
go test -fuzz=FuzzReverse -fuzztime=30s   # generate new inputs for 30 seconds