Skip to content

Expert · Standard library essentials · topic 34 of 46 · 40 min

Files, time and structured logs

Explanation

os.ReadFile and os.WriteFile handle small files in one call; bufio.Scanner reads big files line by line. The time package measures durations and formats dates with a reference layout, "2006-01-02 15:04:05". log/slog (Go 1.21+) writes structured logs as key-value pairs or JSON, which production log tools can search.

Reading and writing files

go
data, err := os.ReadFile("menu.txt")        // whole file as []byte
if err != nil {
	if errors.Is(err, os.ErrNotExist) {
		fmt.Println("no menu yet")
	}
	return err
}
fmt.Println(string(data))

err = os.WriteFile("report.txt", []byte("Total: 4200\n"), 0o644)
For small files, one call each. 0o644 is the Unix permission: owner can read/write, others can read.
go
f, err := os.Open("orders.csv")
if err != nil {
	return err
}
defer f.Close()

sc := bufio.NewScanner(f)
line := 0
for sc.Scan() {          // one line per loop, without the newline
	line++
	fmt.Println(line, sc.Text())
}
if err := sc.Err(); err != nil {
	return err
}
bufio.Scanner reads big files without loading them into memory. Lines longer than 64 KB need sc.Buffer to raise the limit.
  • os.Create(name) makes (or empties) a file for writing; wrap it in bufio.NewWriter for many small writes, and call Flush at the end.
  • filepath.Join("data", "2026", "jan.csv") builds paths with the right separator for Windows or Linux.
  • os.MkdirAll("data/2026", 0o755) creates folders, including missing parents.
  • io.Copy(dst, src) streams from any Reader to any Writer: file to HTTP response, HTTP body to file.
  • encoding/csv reads and writes CSV files correctly, including quoted commas.
  • //go:embed (Go 1.16+) bakes files such as templates into your binary at build time.

Time and durations

go
start := time.Now()
// ... work ...
elapsed := time.Since(start)                  // a time.Duration
fmt.Println(elapsed.Milliseconds(), "ms")

d := 90 * time.Minute
fmt.Println(d)                                // 1h30m0s
tomorrow := time.Now().Add(24 * time.Hour)
deadline := time.Now().AddDate(0, 0, 7)       // 7 days later
fmt.Println(tomorrow.Before(deadline))        // true
time.Duration is an int64 count of nanoseconds with nice printing. Durations are always typed: time.Sleep(2) sleeps 2 nanoseconds, not 2 seconds!

Formatting dates: the reference time

Go does not use YYYY-MM-DD codes. Instead you write how one special moment would look: Monday, January 2, 2006 at 3:04:05 PM, time zone MST (UTC-7). The numbers count up: month 1, day 2, hour 3 (15 in 24-hour form), minute 4, second 5, year 2006, zone -0700.

go
t := time.Date(2026, time.August, 15, 9, 30, 0, 0, time.UTC)
fmt.Println(t.Format("2006-01-02"))           // 2026-08-15
fmt.Println(t.Format("02/01/2006 15:04"))     // 15/08/2026 09:30  (Indian style date)
fmt.Println(t.Format("Mon, 02 Jan 2006"))     // Sat, 15 Aug 2026
fmt.Println(t.Format(time.RFC3339))           // 2026-08-15T09:30:00Z

d, err := time.Parse("2006-01-02", "2026-01-26")
fmt.Println(d.Weekday(), err)                 // Monday <nil>
Writing "YYYY-MM-DD" as a layout prints those letters literally; use the reference numbers.

Time zones

go
ist, err := time.LoadLocation("Asia/Kolkata")
if err != nil {
	ist = time.FixedZone("IST", 5*60*60+30*60) // +05:30 fallback
}
fmt.Println(time.Now().In(ist).Format("15:04 MST"))
LoadLocation needs the time-zone database. Tiny containers (scratch) may not have it: add import _ "time/tzdata" to embed it, or use FixedZone for IST.
Careful: Store and send times in UTC (or with an explicit offset, like RFC 3339), and convert to IST only when showing them to users. Mixing local times from different servers is a classic source of "the report is off by 5.5 hours" bugs.

Structured logging with log/slog

The old log package prints free text: log.Printf("payment %s done", id). That is hard to search across millions of lines. log/slog (Go 1.21+) writes structured logs: a message plus key-value pairs, as text or JSON, with levels (Debug, Info, Warn, Error). Log platforms (Loki, ELK, CloudWatch and others) can then filter by order_id or level.

go
slog.Info("payment captured", "order_id", "ORD-77", "paise", 49900)
// 2026/08/15 09:30:00 INFO payment captured order_id=ORD-77 paise=49900

logger := slog.New(slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelDebug}))
slog.SetDefault(logger)
slog.Warn("bank slow", "bank", "BankA", "latency_ms", 2300)
// {"time":"...","level":"WARN","msg":"bank slow","bank":"BankA","latency_ms":2300}

reqLog := logger.With("request_id", "a1b2")   // attach fields to every line
reqLog.Error("charge failed", "err", err)
Pass key, value, key, value after the message. With adds fields once for a whole request.
Careful: Never log secrets: passwords, OTPs, full card numbers, Aadhaar numbers, API keys or tokens. Mask them (last 4 digits) or leave them out. Logs are read by many people and kept for a long time.
Tip: log.Fatal and log.Fatalf print and then call os.Exit(1), so deferred calls do not run. In servers, prefer logging the error and shutting down gracefully (module 8).

Need help?

Stuck on a word or an idea? Search this course, or every course, and jump straight to the explanation or a worked question.

Try:

Watch the lesson

Files, time and structured logs · 1/7

Files · time · slog

Three everyday tools: files, time, and logs.

Prefer a video by a teacher? Find free videos on “golang read file bufio scanner time format layout slog structured logging” on YouTube ↗

In real life: A daily sales report from a CSV file

A kirana store's billing software exports each day's sales to a CSV file. A small Go program reads it line by line, totals the amount, stamps the report with the date in IST format, and logs a structured summary line that the shop owner's dashboard can pick up.

go
package main

import (
	"bufio"
	"fmt"
	"log/slog"
	"strconv"
	"strings"
	"time"
)

func main() {
	csv := "item,paise\nrice,26500\ndal,16000\noil,18500\n"
	sc := bufio.NewScanner(strings.NewReader(csv)) // a real program would use os.Open
	total := 0
	first := true
	for sc.Scan() {
		if first { // skip the header line
			first = false
			continue
		}
		parts := strings.Split(sc.Text(), ",")
		p, err := strconv.Atoi(parts[1])
		if err != nil {
			slog.Warn("bad line", "line", sc.Text())
			continue
		}
		total += p
	}
	ist := time.FixedZone("IST", 5*60*60+30*60)
	day := time.Date(2026, 8, 15, 21, 0, 0, 0, ist)
	fmt.Printf("Sales on %s: Rs %d.%02d\n", day.Format("02-01-2006"), total/100, total%100)
	slog.Info("daily report", "date", day.Format("2006-01-02"), "total_paise", total)
}
Prints Sales on 15-08-2026: Rs 610.00, then a log line with date=2026-08-15 total_paise=61000.

Practice (0/4 done)

Task 1 · Question

Which layout formats 15 August 2026 as 2026-08-15?

Task 2 · Question

What does fmt.Println(90 * time.Minute) print?

Task 3 · Question

time.Sleep(3) is called. How long does it sleep?

Task 4 · Practice

Write a program that appends a line "2026-08-15 09:30 IST checked in" (using the current time in IST) to a file attendance.log, then reads the whole file back and prints how many lines it has. Log the result with slog.Info including the file name and line count.

Quick quiz

Question 1

1.How do you check whether os.ReadFile failed because the file does not exist?

Question 2

2.Which reads a 2 GB log file without loading it all into memory?

Question 3

3.What is log/slog for?

Question 4

4.Which of these should never appear in logs?

Read more (free)

Open a question to see the answer and why. The first ones are this topic's own quiz, so try the quiz above before you peek.

How do you check whether os.ReadFile failed because the file does not exist?

Answer: errors.Is(err, os.ErrNotExist)

os.ErrNotExist (also fs.ErrNotExist) is the sentinel error.

Which reads a 2 GB log file without loading it all into memory?

Answer: bufio.Scanner over an opened file

Scanner reads in chunks, one line at a time.

What is log/slog for?

Answer: Structured logs with levels and key-value pairs

slog arrived in Go 1.21 for structured logging.

Which of these should never appear in logs?

Answer: A user's OTP or password

Secrets in logs are a serious security leak.

Why use a stream for a 5 GB log file?Related

Answer: It processes chunks without loading the whole file into memory

Streams keep memory use small.

From Files and a tiny server in Node

A payment retry succeeded on the second attempt. Which level fits best?Related

Answer: WARN

Something odd happened but the system coped, which is what WARN is for.

From Logging: structured, safe and centralised

Why add a requestId to every log line?Related

Answer: To find every line about one request quickly

Searching one request id shows the full story of that request across all log lines.

From Logging and monitoring

Two Date objects for the same moment: a === b is…Related

Answer: false

=== compares object identity; compare getTime() values instead.

From Dates and times

Pass the quiz (60% or more) or mark this topic as done when you are ready.

Your progress is saved on this device only.

Next: REST API design and JSON handlers →

Next steps

  1. This is the last topic of “Standard library essentials”. The module exam comes next.
  2. Module 7 exam: Standard library essentials · 14 questions · 25 min, without help. 4 topics of this module still to finish first.
  3. Then the next module, Building a REST API with a database. 46 topics left in the course before the final exam.

← Previous: JSON with encoding/json