NoSQL: documents, key-value, wide-column and graph

Why NoSQL appeared

In the 2000s, web companies hit limits of a single relational server: billions of rows, traffic from every country, and data whose shape changed every week. Making one server bigger (scaling up) gets expensive and eventually stops. They needed to spread data across many servers (scaling out), and they wanted to add a field without a migration. New databases were built for that. They were called NoSQL, which today is usually read as "not only SQL".

Two words first. A schema is the agreed structure of the data (tables, columns, types). Flexible schema means each record may have different fields, and the database does not enforce one structure; your application does.

The four families

DynamoDB is usually called key-value, but it also stores document-style items. Many products blur the lines, so think of the families as shapes, not strict boxes.

The CAP theorem in plain words

When data lives on several servers, the network between them can break: this is a partition. The CAP theorem (Eric Brewer's idea, proven in 2002) says: during a partition, a database must choose. Either it stays Consistent (every read sees the latest write, so some requests must wait or fail) or it stays Available (every request gets an answer, which may be stale).

• Misreading 1: "pick any two of three". Partitions happen whether you like it or not, so the real choice is C or A while a partition lasts. When the network is healthy, you can have both.
• Misreading 2: the C in CAP is the C in ACID. It is not. CAP's C means every reader sees the newest value; ACID's C means rules and constraints hold.
• Misreading 3: "NoSQL is AP, SQL is CP". Many systems let you choose per query, for example Cassandra's consistency levels or MongoDB's read and write concerns.

BASE vs ACID

Module 8 taught ACID: Atomic, Consistent, Isolated, Durable. Many early NoSQL systems chose BASE instead: Basically Available, Soft state, Eventually consistent. Eventually consistent means copies may disagree for a short time, but if writes stop, they all end up with the same value. A "like" count that is a second late is fine; a bank balance is not.

The line has blurred. MongoDB supports multi-document ACID transactions since version 4.0 (and across shards since 4.2). DynamoDB has transactions too. And distributed SQL databases such as CockroachDB, YugabyteDB and Google Spanner scale out while keeping SQL and ACID.

Common mistakes

• Choosing NoSQL "because it scales" for an app with ten thousand users. Fix: a single PostgreSQL or MySQL server handles a lot; choose by data shape and access pattern.
• Thinking "schemaless" means "no design". Fix: in document stores you design around the queries you will run; the schema simply lives in your code.
• Assuming NoSQL has no transactions, or that every NoSQL query is eventually consistent. Fix: read the current docs of the product.
• Using a graph database for simple lookups, or a key-value store for reports. Fix: match the family to the questions you ask.