Block and file storage: EBS, instance store and EFS

Three kinds of storage

Before choosing a service, know the three shapes of storage. Block storage gives the operating system a raw disk split into blocks; you format it with a file system such as ext4 or XFS and use it like a laptop drive. File storage is a ready-made shared folder over the network, so many machines see the same files. Object storage keeps whole files as objects with a name (key) and metadata; you put and get them with web requests, but you do not edit a byte in the middle.

EBS volumes

An EBS (Elastic Block Store) volume is a network disk that lives in one Availability Zone (AZ). It can be attached only to instances in that same AZ. It survives stop and start, and it is copied inside the AZ to protect against a single disk failing. Volume types:

• gp3: general purpose SSD and the usual default. It includes a baseline of 3,000 IOPS (input/output operations per second) and 125 MiB/s of throughput at any size, and you can pay to raise them separately.
• io2 (Block Express): provisioned IOPS SSD for busy databases that need steady, very high IOPS and higher durability.
• st1: throughput-optimised HDD for big sequential reads such as log processing. Cannot be a boot volume.
• sc1: cold HDD, the cheapest, for data you rarely read. Cannot be a boot volume.

You can grow a volume or change its type while it is in use. Maximum sizes and IOPS change over time, so check the EBS documentation for the current limits.

Snapshots and encryption

A snapshot is a point-in-time backup of a volume, stored by AWS in S3 behind the scenes (you do not see it in your buckets). Snapshots are incremental: the first one copies all used blocks, and later ones copy only blocks that changed. You can create a new volume from a snapshot in any AZ of the region, or copy the snapshot to another region for disaster recovery. That is also how you move a disk to another AZ.

EBS encryption uses AWS KMS keys. Data at rest, snapshots and data moving between the instance and the volume are all encrypted, with no change to your app. Turn on "EBS encryption by default" in each region you use so nobody forgets.

aws ec2 create-volume --availability-zone ap-south-1a --size 50 --volume-type gp3 --encrypted
aws ec2 attach-volume --volume-id vol-0abc1234 --instance-id i-0abc1234 --device /dev/sdf

# On the instance (Nitro types show the disk as nvme1n1)
lsblk
sudo mkfs -t xfs /dev/nvme1n1      # format ONLY a new, empty volume
sudo mkdir /data && sudo mount /dev/nvme1n1 /data

aws ec2 create-snapshot --volume-id vol-0abc1234 --description "before upgrade"

Instance store: fast but temporary

Some instance types (those with a d in the name, and the i family) come with instance store: NVMe disks physically inside the host. They are very fast and included in the price, but the data is lost when the instance stops, terminates, or the hardware fails. It survives a reboot. Use it only for caches, scratch files and copies of data that also lives somewhere else.

EFS and FSx: shared file systems

Amazon EFS (Elastic File System) is a managed NFS file system for Linux. Many instances, in many AZs, mount it at the same time and see the same files. It grows and shrinks on its own and you pay per GB stored, so you never size it in advance. It has a Regional option that stores data across several AZs and a cheaper One Zone option, plus Infrequent Access and Archive storage classes with lifecycle rules.

sudo dnf install -y amazon-efs-utils
sudo mount -t efs -o tls fs-0abc1234:/ /mnt/shared

Amazon FSx gives you other popular file systems as a managed service: FSx for Windows File Server (SMB, for Windows apps and Active Directory), FSx for Lustre (very fast, for HPC and machine learning), FSx for NetApp ONTAP and FSx for OpenZFS.

How to choose

1. Is it temporary scratch data you can lose? Instance store.
2. Is it accessed by apps over HTTPS, very large, or a backup or archive? S3.
3. Must several servers share the same files? EFS (Linux) or FSx for Windows File Server (Windows).
4. Otherwise, one server needs a disk (boot volume, database)? EBS, usually gp3.