Skip to content

Intermediate: combining and changing data · Changing data, transactions and ACID · topic 39 of 62 · 30 min

Upserts: insert or update

Explanation

An upsert inserts a row if its key is new and updates it if the key already exists, in one statement. SQLite and PostgreSQL write INSERT ... ON CONFLICT (key) DO UPDATE SET ..., using excluded.column for the new values. MySQL uses ON DUPLICATE KEY UPDATE, and SQL Server uses MERGE. INSERT OR IGNORE skips duplicates, and REPLACE deletes and re-inserts, which can lose data.

The problem

A warehouse receives a shipment: 10 bags of basmati rice and 24 bottles of mustard oil. Rice is already in the inventory table; mustard oil is new. A plain INSERT of rice fails with "UNIQUE constraint failed: inventory.sku", and a plain UPDATE of oil changes nothing because the row does not exist. You could first SELECT, then decide in app code, but between your SELECT and your write another request may insert the same row. An upsert does the check and the write in one atomic statement.

ON CONFLICT ... DO UPDATE (SQLite, PostgreSQL)

sql
INSERT INTO inventory (sku, name, qty, updated)
VALUES ('RICE5', 'Basmati Rice 5kg', 10, '2025-04-05'),
       ('OIL1',  'Mustard Oil 1L',   24, '2025-04-05')
ON CONFLICT (sku) DO UPDATE
SET qty     = inventory.qty + excluded.qty,
    updated = excluded.updated;
-- DAL1   | Toor Dal 1kg     | 75 | 2025-04-01
-- OIL1   | Mustard Oil 1L   | 24 | 2025-04-05   (inserted)
-- RICE5  | Basmati Rice 5kg | 50 | 2025-04-05   (40 + 10, updated)
-- TIFFIN | Steel Tiffin Box | 12 | 2025-04-01
excluded is the row you tried to insert; inventory.qty is the row already stored.
  • ON CONFLICT (sku) names the column(s) of a PRIMARY KEY or UNIQUE constraint. Without such a constraint there is no conflict to catch.
  • excluded.qty means "the qty from the VALUES I tried to insert". Writing qty = excluded.qty replaces the old number; qty = inventory.qty + excluded.qty adds to it.
  • ON CONFLICT DO NOTHING skips the duplicate row silently.
  • SQLite has supported this since version 3.24 and PostgreSQL since 9.5.

MySQL: ON DUPLICATE KEY UPDATE

sql
-- MySQL 8.0.19 and later
INSERT INTO inventory (sku, name, qty) VALUES ('RICE5', 'Basmati Rice 5kg', 10) AS new
ON DUPLICATE KEY UPDATE qty = inventory.qty + new.qty;
Older MySQL code writes VALUES(qty) instead of new.qty; that form is deprecated. It fires on any PRIMARY KEY or UNIQUE clash.

SQL Server: MERGE

sql
-- SQL Server (Oracle and PostgreSQL 15+ also have MERGE)
MERGE INTO inventory AS t
USING (VALUES ('RICE5', 'Basmati Rice 5kg', 10)) AS s (sku, name, qty)
ON t.sku = s.sku
WHEN MATCHED THEN UPDATE SET t.qty = t.qty + s.qty
WHEN NOT MATCHED THEN INSERT (sku, name, qty) VALUES (s.sku, s.name, s.qty);
MERGE compares a source with the target and says what to do for matched and unmatched rows. It must end with a semicolon in SQL Server; under heavy concurrent load, read the docs on locking hints.

INSERT OR IGNORE and REPLACE (SQLite)

SQLite also has shortcut forms. INSERT OR IGNORE skips any row that would break a UNIQUE, PRIMARY KEY, NOT NULL or CHECK rule and carries on with the rest; it is handy for importing a list that may contain people you already have. MySQL's INSERT IGNORE is similar. Because it hides every such error, check that you only expect duplicates.

sql
INSERT OR IGNORE INTO subscribers (email, name, source)
VALUES ('asha@example.com', 'Asha V', 'csv'),       -- already there: skipped
       ('meena@example.com', 'Meena Joshi', 'csv');  -- new: inserted

REPLACE INTO (or INSERT OR REPLACE) looks like an upsert but is not. On a conflict it DELETES the old row and INSERTS the new one. Every column you did not give is set to its default or NULL, the old values are lost, delete triggers fire, and with ON DELETE CASCADE foreign keys the child rows of the old row can be deleted too. MySQL's REPLACE behaves the same way.

sql
REPLACE INTO inventory (sku, name, qty) VALUES ('DAL1', 'Toor Dal 1kg', 80);
-- DAL1 | Toor Dal 1kg | 80 | NULL     <- updated was '2025-04-01', now lost

Common mistakes

  • Using ON CONFLICT on a column with no UNIQUE or PRIMARY KEY constraint. Add the constraint first.
  • Writing qty = excluded.qty when you meant to add. Decide: replace (excluded.qty) or add (table.qty + excluded.qty).
  • Using REPLACE as an upsert and silently wiping other columns. Use ON CONFLICT DO UPDATE.
  • Using INSERT OR IGNORE and hiding real errors such as a NULL in a NOT NULL column. Check the row count afterwards.
  • Doing SELECT-then-INSERT in app code. Two requests can both see "not there" and both insert. Let the database decide with an upsert.

Need help?

Stuck on a word or an idea? Search this course, or every course, and jump straight to the explanation or a worked question.

Try:

Watch the lesson

Upserts: insert or update · 1/7

Upsert = insert or update

Sometimes a row may or may not exist yet. An upsert inserts it if it is new and updates it if it is already there.

Prefer a video by a teacher? Find free videos on “sql upsert on conflict do update on duplicate key update merge” on YouTube ↗

In real life: Syncing a product catalogue every night

Marketplaces and quick-commerce apps often receive a nightly file of products and stock from each seller. Most rows already exist and only the stock or price changed; a few are new. The import job loads the file and runs one upsert per batch: new SKUs are inserted, known SKUs are updated, all in a single statement per batch instead of a SELECT before every row.

sql
INSERT INTO inventory (sku, name, qty, updated)
VALUES ('DAL1', 'Toor Dal 1kg', 70, '2025-04-06'),
       ('JAR2', 'Steel Jar 2L', 6,  '2025-04-06')
ON CONFLICT (sku) DO UPDATE
SET qty = excluded.qty, updated = excluded.updated;
Here the file has the counted stock, so qty is replaced: DAL1 becomes 70, JAR2 is added.

Practice (0/3 done)

Task 1 · Code · SQL

A shipment arrives on '2025-04-05': 10 more of SKU 'RICE5' ('Basmati Rice 5kg') and 24 of a new SKU 'OIL1' ('Mustard Oil 1L'). Write ONE upsert that adds the quantity to existing SKUs (and sets updated to the new date) and inserts new ones. Then return sku, qty and updated from inventory, sorted by sku.

Task 2 · Code · SQL

Import these subscribers, skipping anyone whose email already exists, with INSERT OR IGNORE: ('asha@example.com', 'Asha V', 'csv'), ('meena@example.com', 'Meena Joshi', 'csv'), ('ravi@example.com', 'Ravi K', 'csv'), ('karan@example.com', 'Karan Mehta', 'csv'). Then return email, name and source of all subscribers, sorted by email.

Task 3 · Question

inventory has the row ('DAL1', 'Toor Dal 1kg', 75, updated '2025-04-01'). You run REPLACE INTO inventory (sku, name, qty) VALUES ('DAL1', 'Toor Dal 1kg', 80). What is the row afterwards?

Quick quiz

Question 1

1.In ON CONFLICT (sku) DO UPDATE SET qty = excluded.qty, what is excluded.qty?

Question 2

2.What does ON CONFLICT need in order to work?

Question 3

3.Which is the MySQL way to write an upsert?

Question 4

4.Why is "SELECT to check, then INSERT if missing" in app code risky?

Question 5

5.What does INSERT OR IGNORE do with a row whose email already exists?

Read more (free)

Open a question to see the answer and why. The first ones are this topic's own quiz, so try the quiz above before you peek.

In ON CONFLICT (sku) DO UPDATE SET qty = excluded.qty, what is excluded.qty?

Answer: The qty of the row you tried to insert

excluded is the proposed new row; the table name refers to the existing row.

What does ON CONFLICT need in order to work?

Answer: A PRIMARY KEY or UNIQUE constraint on the conflict column(s)

A conflict only exists when a unique rule would be broken.

Which is the MySQL way to write an upsert?

Answer: INSERT ... ON DUPLICATE KEY UPDATE

MySQL uses ON DUPLICATE KEY UPDATE; SQL Server uses MERGE.

Why is "SELECT to check, then INSERT if missing" in app code risky?

Answer: Two requests can both see "missing" and both try to insert

That gap between check and write is a race condition. An upsert is one atomic statement.

What does INSERT OR IGNORE do with a row whose email already exists?

Answer: Skips that row and continues with the others

It quietly skips rows that would break a constraint.

In SQLite, you write INSERT INTO tickets (customer_id, subject, status) VALUES (3, 'Hi', DEFAULT). What happens?Related

Answer: A syntax error: SQLite does not accept DEFAULT in VALUES

Leave the column out instead. PostgreSQL, MySQL and SQL Server do accept DEFAULT there.

From INSERT

What does RETURNING id do at the end of an INSERT?Related

Answer: Gives back the id of the new row

RETURNING sends back values from the changed rows, such as the new id.

From INSERT, UPDATE and DELETE

How do you return only the first 5 rows in SQL Server?Related

Answer: SELECT TOP 5 ...

SQL Server uses TOP, or OFFSET 0 ROWS FETCH NEXT 5 ROWS ONLY with ORDER BY.

From MySQL vs PostgreSQL vs SQLite vs SQL Server

In SQLite, how do you see how many rows the last UPDATE changed?Related

Answer: SELECT changes()

changes() returns the rows changed by the most recent INSERT, UPDATE or DELETE.

From UPDATE and DELETE safely

Pass the quiz (60% or more) or mark this topic as done when you are ready.

Your progress is saved on this device only.

Next steps

  1. 2 more topics in “Changing data, transactions and ACID” before its exam: Transactions: all or nothing, ACID, isolation and locking.
  2. Module 8 exam: Changing data, transactions and ACID · 14 questions · 25 min, without help. 5 topics of this module still to finish first.
  3. Then the next module, Schema design and normalisation. 62 topics left in the course before the final exam.

← Previous: UPDATE and DELETE safely