AWS: Storage: EBS (Elastic Block Storage)

EBS Volumes are highly reliable & available block storage volumes (hard drive) that can be attached to a running EC2 instance in the same availability zone. You can build your file system on top of EBS. You can attach multiple EBS volumes to an instance or multiple instances can use an EBS volume. By default, an instance ‘root’ volume is EBS volumes (for example root volume is like HD of laptop and Additional drive is like additional disk or drive). Following are volume types provided by EBS

IOPS is number of input-output operations per second whereas THROUGHPUT is number of bits read-write per second.

	Solid-state drives (SSD)		Hard disk drives (HDD)
Volume type	General Purpose SSD (gp2)	Provisioned IOPS SSD (io1)	Throughput Optimized HDD (st1)	Cold HDD (sc1)
Description	General purpose SSD volume that balances price and performance for a wide variety of workloads	Highest-performance SSD volume for mission-critical low-latency or high-throughput workloads	Low-cost HDD volume designed for frequently accessed, throughput-intensive workloads	Lowest cost HDD volume designed for less frequently accessed workloads
Use cases	Recommended for most workloads System boot volumes Virtual desktops Low-latency interactive apps Development and test environments	Critical business applications that require sustained IOPS performance, or more than 16,000 IOPS or 250 MiB/s of throughput per volume Large database workloads, such as: MongoDB Cassandra Microsoft SQL Server MySQL PostgreSQL Oracle	Streaming workloads requiring consistent, fast throughput at a low price Big data Data warehouses Log processing Cannot be a boot volume	Throughput-oriented storage for large volumes of data that is infrequently accessed Scenarios where the lowest storage cost is important Cannot be a boot volume
API name	gp2	io1	st1	sc1
Volume size	1 GiB - 16 TiB	4 GiB - 16 TiB	500 GiB - 16 TiB	500 GiB - 16 TiB
Max IOPS per volume	16,000 (16 KiB I/O) *	64,000 (16 KiB I/O) †	500 (1 MiB I/O)	250 (1 MiB I/O)
Max throughput per volume	250 MiB/s *	1,000 MiB/s †	500 MiB/s	250 MiB/s
Max IOPS per instance ††	80,000	80,000	80,000	80,000
Max throughput per instance ††	2,375 MB/s	2,375 MB/s	2,375 MB/s	2,375 MB/s
Dominant performance attribute	IOPS	IOPS	MiB/s	MiB/s

When you create an EBS volumes then it is automatically replicated within its availability zone to ensure availability & preventing data loss. 

Delete on Termination: If checked, volume will be lost on termination of EC2 instance, else it will remain as is so that it can be attached to other instances.

Data Encryption: EBS uses AES-256 algorithm encryption & KMS

Snapshot: Snapshots are incremental backups that means you are charged for only the blocks that are changed after recent snapshot. EBS volumes that are restored from encrypted snapshots are automatically encrypted.

General Purpose SSD (gp2) volumes: IO credits & burst performance: IO credit represents the available bandwidth that your gp2 volume can use to burst large amounts of IO when more than baseline performance is needed. Each volume receives an initial I/O credit balance of 5.4 million I/O credits, which is enough to sustain the maximum burst performance of 3000 IOPS for 30 minutes. When the baseline performance is higher than the maximum burst performance, IO credits are never spent.

Burst duration =  (Credit balance) / ((Burst IOPS) – 3 (Volume size in GiB))

Volume size (GiB)	Baseline performance (IOPS)	Burst duration when driving sustained 3,000 IOPS (second)	Seconds to fill empty credit balance when driving no IO
1	100	1802	54,000
100	300	2,000	18,000
250	750	2,400	7,200
334 (Min. size for max throughput)	1002	2703	5389
500	1,500	3,600	3,600
750	2,250	7,200	2,400
1,000	3,000	N/A*	N/A*
5,334 (Min. size for max IOPS)	16,000	N/A*	N/A*
16,384 (16 TiB, max volume size)	16,000	N/A*	N/A*

Provisioned IOPS SSD (io1) volumes: Unlike gp2, which uses bucket & credit model to calculate performance, io1 allows you to specify consistent IOPS rate when you create the volume. The maximum IOPS to provisioned GiB volume size ratio is 50:1, for example, 100 GiB volume can be provisioned with 5000 IOPS. 

Throughput Optimized HDD (st1) volumes: Defines performance in terms of throughput than IOPS. Designed to support frequently accessed data. Like gp2, st1 also uses burst-bucket model for performance. Volume size determines the baseline throughput (rate for accumulating credit) as well as the burst throughput (rate for spending credits). The available throughput on the st1 volume is expressed as 

(volume size) * (credit accumulation rate per TiB) = throughput

Cold HDD (sc1) volumes: Defines performance in terms of throughput than IOPS. Designed to support infrequent access to data with lower cost. Like gp2, sc1 also uses burst-bucket model for performance. Volume size determines the baseline throughput (rate for accumulating credit) as well as the burst throughput (rate for spending credits). The available throughput on the st1 volume is expressed as 

(volume size) * (credit accumulation rate per TiB) = throughput

Snapshots: Snapshots are incremental backups on S3. Snapshot replication is processed in background; hence you can immediately start using the volume. When you request for the data which is not loaded yet, the volume immediately downloads it from S3 and then continues loading rest of the data. 

You can use EBS APIs to directly read the data from snapshots and identify the difference between two snapshots. 

Using Amazon Data Lifecycle Manager to automate creation, retention & deletion of the snapshots.

When you delete a snapshot, only the data referenced exclusively by snapshot is removed. Unique data will not be deleted unless all of the snapshots that reference that data will not be deleted. 

How it works – for below changes total volume is 12 GB whereas snapshot space required is 16 GB

Instance Store: It is temporary block level storage for your instance on the disks that are physically attached to your computer.