Expert · Building a REST API with a database · topic 37 of 46 · 45 min
Databases with database/sql
Explanation
The pieces
- database/sql: the standard package with the API you call (DB, Rows, Tx).
- A driver: a third-party package that knows one database's protocol, registered with a blank import. Common ones: github.com/jackc/pgx/v5/stdlib (PostgreSQL, driver name "pgx"), github.com/go-sql-driver/mysql (MySQL, "mysql"), modernc.org/sqlite (SQLite in pure Go, "sqlite").
- *sql.DB: not one connection but a pool of them, safe for all goroutines. Create it once at start-up and share it.
import (
"database/sql"
_ "github.com/jackc/pgx/v5/stdlib" // registers the "pgx" driver
)
func openDB(ctx context.Context, dsn string) (*sql.DB, error) {
db, err := sql.Open("pgx", dsn) // checks arguments only; no connection yet!
if err != nil {
return nil, err
}
db.SetMaxOpenConns(25)
db.SetMaxIdleConns(25)
db.SetConnMaxLifetime(5 * time.Minute)
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
if err := db.PingContext(ctx); err != nil { // now it really connects
db.Close()
return nil, fmt.Errorf("connect to db: %w", err)
}
return db, nil
}One row: QueryRowContext
func (s *PGStore) Get(ctx context.Context, id int64) (Expense, error) {
var e Expense
err := s.db.QueryRowContext(ctx,
"SELECT id, title, amount_paise, category, created_at FROM expenses WHERE id = $1", id,
).Scan(&e.ID, &e.Title, &e.Paise, &e.Category, &e.Created)
if errors.Is(err, sql.ErrNoRows) {
return Expense{}, ErrNotFound
}
if err != nil {
return Expense{}, fmt.Errorf("get expense %d: %w", id, err)
}
return e, nil
}Many rows: QueryContext
func (s *PGStore) List(ctx context.Context, limit, offset int) ([]Expense, error) {
rows, err := s.db.QueryContext(ctx,
"SELECT id, title, amount_paise, category, created_at FROM expenses ORDER BY id LIMIT $1 OFFSET $2",
limit, offset)
if err != nil {
return nil, err
}
defer rows.Close() // returns the connection to the pool
list := []Expense{}
for rows.Next() {
var e Expense
if err := rows.Scan(&e.ID, &e.Title, &e.Paise, &e.Category, &e.Created); err != nil {
return nil, err
}
list = append(list, e)
}
return list, rows.Err() // errors that stopped the loop early
}Changing data: ExecContext and RETURNING
// PostgreSQL: get the new id with RETURNING
err := s.db.QueryRowContext(ctx,
"INSERT INTO expenses (title, amount_paise, category) VALUES ($1, $2, $3) RETURNING id, created_at",
e.Title, e.Paise, e.Category,
).Scan(&e.ID, &e.Created)
// No rows needed back: ExecContext
res, err := s.db.ExecContext(ctx, "DELETE FROM expenses WHERE id = $1", id)
if err != nil {
return err
}
n, _ := res.RowsAffected()
if n == 0 {
return ErrNotFound
}Placeholders stop SQL injection
// NEVER do this:
q := fmt.Sprintf("SELECT * FROM users WHERE email = '%s'", email)
// email = "x' OR '1'='1" turns the query into "... WHERE email = 'x' OR '1'='1'"
// ALWAYS do this:
row := db.QueryRowContext(ctx, "SELECT id, name FROM users WHERE email = $1", email)| Database | Placeholder style |
|---|---|
| PostgreSQL (pgx, lib/pq) | $1, $2, $3 |
| MySQL (go-sql-driver/mysql) | ? |
| SQLite (modernc.org/sqlite, mattn/go-sqlite3) | ? (numbered forms also exist) |
Transactions
func transfer(ctx context.Context, db *sql.DB, from, to, paise int64) error {
tx, err := db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback() // does nothing if Commit already succeeded
if _, err := tx.ExecContext(ctx, "UPDATE wallets SET balance = balance - $1 WHERE id = $2 AND balance >= $1", paise, from); err != nil {
return err
}
if _, err := tx.ExecContext(ctx, "UPDATE wallets SET balance = balance + $1 WHERE id = $2", paise, to); err != nil {
return err
}
return tx.Commit()
}NULLs and tools
A NULL column cannot be scanned into a plain string. Use sql.NullString (and NullInt64, NullTime...) or a pointer like *string. Many teams also use sqlc, which generates type-safe Go code from your SQL queries, or pgx's own native API for PostgreSQL-specific features. ORMs such as GORM exist too; learn plain database/sql first so you understand what they do.
Need help?
Stuck on a word or an idea? Search this course, or every course, and jump straight to the explanation or a worked question.
Try:
Watch the lesson
Databases with database/sql · 1/7
Go talks to SQL databases through database slash sql, plus a driver for your database.
Prefer a video by a teacher? Find free videos on “golang database/sql postgres pgx tutorial query exec transactions” on YouTube ↗
In real life: Saving expenses in PostgreSQL
The budgeting app moves from the in-memory store to PostgreSQL. Only a new PGStore type is written; the handlers keep using the Store interface. Each method uses context-aware calls, so if the Android user closes the app mid-request, the query is cancelled too. Placeholders keep a malicious expense title like "x'); DROP TABLE expenses;--" harmless.
CREATE TABLE expenses (
id BIGSERIAL PRIMARY KEY,
title TEXT NOT NULL,
amount_paise BIGINT NOT NULL CHECK (amount_paise > 0),
category TEXT NOT NULL DEFAULT 'other',
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);Practice (0/4 done)
sql.Open returned no error. What do you know?
Which query is safe from SQL injection (PostgreSQL)?
QueryRowContext(...).Scan(...) finds no matching row. What error do you get?
Using SQLite (modernc.org/sqlite, driver name "sqlite") so you need no server: create a students table (id, name, marks), insert three students with ExecContext and ? placeholders, then query students with marks above 75 and print them. Close rows and check rows.Err().
Quick quiz
Read more (free)
Related help
Before this, learn
This topic builds on these. If something here feels new, open them first.
Related topics
- Configuration, project structure and graceful shutdownBuilding a REST API with a database · same module
- REST API design and JSON handlersBuilding a REST API with a database · same module
- SQL injection and how to stop itBackend Development · Backend security
- JDBC: talking to a databaseJava Programming (Core to Spring Boot) · JDBC, Maven, Gradle and JUnit
- The SQL behind each endpointFull-Stack Development · Building the backend
- What is a database?Backend Development · Databases and SQL basics
Questions people ask about this
Open a question to see the answer and why. The first ones are this topic's own quiz, so try the quiz above before you peek.
What is *sql.DB?
Answer: A pool of connections, safe for concurrent use
Create one at start-up and share it.
What happens if you forget rows.Close()?
Answer: Connections leak and the pool can run out, making requests hang
Open Rows hold a connection.
How do you get the new id after an INSERT in PostgreSQL?
Answer: INSERT ... RETURNING id with QueryRowContext and Scan
PostgreSQL uses RETURNING; LastInsertId is a MySQL-style feature.
Why defer tx.Rollback() right after BeginTx?
Answer: So any early return rolls back; after Commit it does nothing
Rollback after a successful Commit just returns sql.ErrTxDone, which is safe to ignore.
A column can be NULL. What can you scan it into?
Answer: sql.NullString or *string
Plain strings cannot hold NULL; use a Null type or a pointer.
What causes SQL injection?Related
Answer: Joining user input directly into SQL text
When user text is glued into the query, it can change the query itself. That is injection.
Why use a transaction for a money transfer?Related
Answer: So the debit and credit succeed or fail together
Atomicity: all or nothing.
Why use placeholders (? or $1) instead of joining strings?Related
Answer: The database treats the values only as data, which stops SQL injection
Values sent separately can never change the SQL command itself, so attackers cannot inject SQL.
Which of these is a SQL database?Related
Answer: PostgreSQL
PostgreSQL stores data in tables and uses SQL. MongoDB is a NoSQL document database.
From What is a database?
Pass the quiz (60% or more) or mark this topic as done when you are ready.
Your progress is saved on this device only.
Next steps
- 1 more topic in “Building a REST API with a database” before its exam: Configuration, project structure and graceful shutdown.
- Module 8 exam: Building a REST API with a database · 14 questions · 25 min, without help. 4 topics of this module still to finish first.
- Then the next module, CLI tools and Go in cloud and DevOps. 46 topics left in the course before the final exam.