Skip to content

Beginner · Compute and storage: EC2 and S3 · topic 12 of 50 · 35 min

Amazon S3: buckets, objects and storage classes

Explanation

Amazon S3 stores files as objects inside buckets. A bucket has a globally unique name and lives in one region; each object has a key (its full name) and can be up to 5 TB. Storage classes let you pay less for data you read rarely, and lifecycle rules move or delete objects automatically as they age.

Why object storage?

Disks fill up, break, and need someone to look after them. Amazon S3 (Simple Storage Service) is storage you never size: you put files in and pay for what you store and the requests you make. It is designed for 99.999999999 percent durability (eleven nines), by keeping copies across several Availability Zones in the region. That is why so many apps keep photos, invoices, backups and logs there.

Buckets, objects and keys

A bucket is a container for objects. Its name must be unique across all AWS accounts, 3 to 63 characters, lowercase letters, numbers, dots and hyphens. You choose the region when you create it, and the data stays in that region unless you copy it out. An object is a file plus metadata. Its key is its full name inside the bucket, such as invoices/2026/03/inv-001.pdf. A single object can be up to 5 TB. One PUT request can upload up to 5 GB; for larger files (and in practice for anything over about 100 MB) use multipart upload, which the AWS CLI does for you automatically.

S3 has no real folders. The console shows invoices/ as a folder only because many keys start with that prefix. A prefix is just the beginning of a key. This matters: "renaming a folder" really means copying every object to a new key and deleting the old one.

Think of it like this: Think of S3 as a huge godown with one long register. Every box has a label like invoices/2026/03/inv-001.pdf. There are no rooms inside; the slashes in the labels only help people sort and search the register.

Working with the CLI

bash
aws s3 mb s3://asha-invoices-2026 --region ap-south-1
aws s3 cp inv-001.pdf s3://asha-invoices-2026/invoices/2026/03/inv-001.pdf
aws s3 ls s3://asha-invoices-2026/invoices/2026/
aws s3 cp s3://asha-invoices-2026/invoices/2026/03/inv-001.pdf ./download.pdf
aws s3 sync ./site s3://asha-site-bucket --delete        # make the bucket match the folder
aws s3 cp backup.tar.gz s3://asha-backups/db/ --storage-class STANDARD_IA
mb makes a bucket, cp copies, ls lists a prefix, sync uploads only what changed. --delete also removes objects that are gone locally, so use it carefully.

Storage classes

Every object has a storage class. Cheaper classes charge less per GB but add retrieval fees, a minimum storage time or a slower first byte. Choose by how often you read the data and how fast you need it.

ClassWhen to useNotes
S3 StandardData read oftenDefault; multi-AZ; no retrieval fee
S3 Intelligent-TieringUnknown or changing accessMoves objects between tiers for you; small monitoring fee per object
S3 Standard-IARead about once a month, needed fastRetrieval fee; minimum 30 days
S3 One Zone-IALike Standard-IA but data you can recreateStored in one AZ only; minimum 30 days
S3 Glacier Instant RetrievalArchive read about once a quarter, needed in millisecondsMinimum 90 days
S3 Glacier Flexible RetrievalArchive; wait minutes to hours is fineRestore before reading; minimum 90 days
S3 Glacier Deep ArchiveKept for years, read almost neverCheapest; standard restore within 12 hours; minimum 180 days

S3 Express One Zone is a special high-performance class in a single AZ, using "directory buckets", for workloads that need single-digit millisecond access, such as machine learning training. Most apps do not need it.

Versioning

With versioning turned on, overwriting an object keeps the old version, and deleting adds a "delete marker" instead of removing the data. You can bring back an older version after a mistake or a ransomware attack. Once enabled, versioning can be suspended but not switched off. Old versions are billed too, so pair versioning with a lifecycle rule that expires them.

Lifecycle rules

A lifecycle rule moves objects to cheaper classes or deletes them as they age, with no code. This rule moves logs/ to Standard-IA after 30 days and to Glacier Flexible Retrieval after 90, deletes them after a year, and removes old versions 30 days after they are replaced.

json
{
  "Rules": [
    {
      "ID": "age-out-logs",
      "Filter": { "Prefix": "logs/" },
      "Status": "Enabled",
      "Transitions": [
        { "Days": 30, "StorageClass": "STANDARD_IA" },
        { "Days": 90, "StorageClass": "GLACIER" }
      ],
      "Expiration": { "Days": 365 },
      "NoncurrentVersionExpiration": { "NoncurrentDays": 30 }
    }
  ]
}
Apply it with: aws s3api put-bucket-lifecycle-configuration --bucket asha-logs --lifecycle-configuration file://lifecycle.json

Replication

Replication copies new objects to another bucket automatically: Cross-Region Replication (CRR) for disaster recovery or lower latency in another region, and Same-Region Replication (SRR) for copies in another account. Versioning must be on in both buckets. Objects that existed before the rule are not copied unless you run S3 Batch Replication.

Careful: Common mistakes: (1) Using uppercase or underscores in bucket names; only lowercase, numbers, dots and hyphens are allowed. (2) Uploading millions of tiny files to Glacier classes, where per-object overheads and minimum sizes make it cost more, not less; bundle them into archives first. (3) Turning on versioning without a rule for old versions, so the bill keeps growing. (4) Running sync with --delete against the wrong bucket. (5) Expecting a "folder rename" to be instant.
Tip: Put dates in keys, such as logs/year=2026/month=03/day=05/. Lifecycle rules, Athena queries and humans can then find a day's data by prefix without listing everything.

Need help?

Stuck on a word or an idea? Search this course, or every course, and jump straight to the explanation or a worked question.

Try:

Watch the lesson

Amazon S3: buckets, objects and storage classes · 1/8

S3 = object storage Buckets hold objects

Amazon S3 stores files as objects in buckets. It is built for eleven nines of durability by keeping copies across several zones.

Prefer a video by a teacher? Find free videos on “amazon s3 tutorial storage classes lifecycle rules versioning” on YouTube ↗

In real life: GST invoices for a Flipkart seller

A seller on an online marketplace in Jaipur generates about 3,000 invoice PDFs a month. They are stored in one S3 bucket in ap-south-1 under invoices/year/month/. Invoices from the current quarter stay in S3 Standard because the accountant opens them often. A lifecycle rule moves them to Glacier Instant Retrieval after 90 days, since a tax officer might ask for any of them at short notice, and to Deep Archive after two years. Indian GST rules require keeping records for several years, so check the current requirement with an accountant before setting the expiry.

bash
aws s3 cp inv-2026-03-0412.pdf s3://seller-invoices-ap-south-1/invoices/2026/03/
aws s3 ls s3://seller-invoices-ap-south-1/invoices/2026/ --recursive --summarize
--summarize prints the number of objects and the total size.

Practice (0/4 done)

Task 1 · Question

You delete the "folder" reports/ in the S3 console. What actually happens?

Task 2 · Code · Javascript

Write groupByPrefix(objects, depth). Each object is { key, size }. Take the "folder" parts of the key (everything before the last slash), keep the first depth parts, and join them with "/" plus a trailing "/". Objects with no folder go under "". Return an object mapping each prefix to { count, bytes }. Example: with depth 1, "invoices/2026/03/inv-001.pdf" belongs to "invoices/".

Task 3 · Code · Python

Write partition_key(prefix, day, filename) that builds a date-partitioned S3 key like "logs/year=2026/month=03/day=05/app.log". day is a datetime.date. Month and day always have two digits. Remove any trailing slash from prefix so you never get "//".

Task 4 · Code · Python

Write choose_class(access, recreatable=False, wait_hours=0) that returns an S3 storage class name as used in the API. access is "frequent", "unknown", "monthly", "quarterly" or "yearly". Rules: frequent gives "STANDARD"; unknown gives "INTELLIGENT_TIERING"; monthly gives "ONEZONE_IA" if the data can be recreated, else "STANDARD_IA"; quarterly gives "GLACIER_IR"; yearly gives "DEEP_ARCHIVE" if you can wait 12 hours or more, else "GLACIER" (Flexible Retrieval).

Quick quiz

Question 1

1.What is the largest single object S3 can store?

Question 2

2.Which storage class fits data with unknown or changing access patterns?

Question 3

3.Versioning is enabled and you delete an object without giving a version ID. What happens?

Question 4

4.Which command uploads only new or changed files from a local folder to a bucket?

Question 5

5.What must be true before you set up S3 Cross-Region Replication?

Read more (free)

Open a question to see the answer and why. The first ones are this topic's own quiz, so try the quiz above before you peek.

What is the largest single object S3 can store?

Answer: 5 TB

An object can be up to 5 TB. A single PUT is limited to 5 GB, so larger objects use multipart upload.

Which storage class fits data with unknown or changing access patterns?

Answer: S3 Intelligent-Tiering

Intelligent-Tiering watches access and moves each object to the cheapest suitable tier for a small monitoring fee.

Versioning is enabled and you delete an object without giving a version ID. What happens?

Answer: A delete marker is added and older versions remain

With versioning, a simple delete adds a delete marker; you can remove the marker to bring the object back.

Which command uploads only new or changed files from a local folder to a bucket?

Answer: aws s3 sync

sync compares source and destination and copies only what is different.

What must be true before you set up S3 Cross-Region Replication?

Answer: Versioning is enabled on both buckets

Replication requires versioning on the source and the destination bucket.

What should the database store for an uploaded file in S3?Related

Answer: The storage key, size, type and owner

The bytes live in object storage; the database keeps the key and details to find and describe them.

From Handling file uploads safely

Which storage loses its data when the instance is stopped?Related

Answer: Instance store

Instance store disks are physically attached to the host and are wiped on stop or terminate.

From Block and file storage: EBS, instance store and EFS

Why build the storage key yourself instead of using the uploaded file name?Related

Answer: Names can clash and can contain tricks like ../

Two users may upload "photo.jpg", and a name like ../../x can be an attack. A key with user id, note id and a random id is safe and unique.

From Adding image attachments to notes

In which DR strategy is a smaller but fully working copy of the whole system always running in the recovery region?Related

Answer: Warm standby

Warm standby runs a scaled-down full copy that is scaled up during a disaster.

From Backups and disaster recovery

Pass the quiz (60% or more) or mark this topic as done when you are ready.

Your progress is saved on this device only.

Next steps

  1. 1 more topic in “Compute and storage: EC2 and S3” before its exam: Securing and sharing S3 data.
  2. Module 3 exam: Compute and storage, EC2 and S3 · 12 questions · 20 min, without help. 4 topics of this module still to finish first.
  3. Then the next module, Networking and databases: VPC and RDS. 50 topics left in the course before the final exam.

← Previous: Block and file storage: EBS, instance store and EFS