Decide on default storage class for EKS clusters

docs/layers/eks/design-decisions/decide-on-default-storage-class.mdx

@@ -14,38 +14,49 @@ When provisioning EKS (Kubernetes) clusters, selecting the appropriate default s
 
 We need to decide between **Amazon EFS (Elastic File System)** and **Amazon EBS (Elastic Block Store)** as the default storage class for our EKS clusters. Historically, **EFS** has been the recommended option, but there is a growing preference for **EBS** for customers.
 
-### Amazon EFS
+## Top Considerations
+
+1. **Low Performance:**
+   - **EFS:** Can be mitigated by paying for additional bandwidth, but has routinely caused outages due to throttling even with low-performance applications. Additionally, poor lock performance makes EFS completely unsuitable for high-performance applications like RDBMS.
+
+2. **Cost:**
+   - **EFS:** At least 3x the cost of EBS since the storage is backed by an EBS volume in each of the 3 Availability Zones (AZs).
+
+3. **Integration:**
+   - Both **EFS** and **EBS** are now fully integrated into Kubernetes with CSI (Container Storage Interface) controllers, facilitating seamless provisioning and management.
+
+## Amazon EFS
 
 **Amazon EFS** provides a scalable, fully managed elastic NFS file system for use with AWS Cloud services and on-premises resources.
 
-#### Pros:
+### Pros:
 - **Scalability:** Automatically scales storage capacity as needed without manual intervention.
 - **Shared Access:** Allows multiple pods to access the same file system concurrently, facilitating shared storage scenarios.
 - **Managed Service:** Fully managed by AWS, reducing operational overhead for maintenance and scaling.
 - **NFS Protocol Support:** Compatible with applications requiring NFS (Network File System) access.
 
-#### Cons:
+### Cons:
 - **Performance:** Generally offers lower performance compared to EBS, with throughput as low as 100 MB/s, which may not meet the demands of high-performance applications.
 - **Cost:** Can be significantly more expensive than EBS, potentially up to 3x the price depending on usage patterns.
 - **Latency:** Higher latency compared to EBS, which may impact performance-sensitive applications.
 
-### Amazon EBS
+## Amazon EBS
 
 **Amazon EBS** provides high-performance block storage volumes for use with Amazon EC2 instances, suitable for a wide range of workloads.
 
-#### Pros:
+### Pros:
 - **Performance:** Offers high IOPS and low latency, making it ideal for performance-critical applications.
 - **Cost-Effective:** Potentially lower costs for specific storage types and usage scenarios.
 - **Integration:** Well-integrated with Kubernetes through the EBS CSI (Container Storage Interface) driver, facilitating seamless provisioning and management.
 - **Snapshot and Backup:** Supports snapshotting for data backup, recovery, and cloning.
 
-#### Cons:
+### Cons:
 - **Single-Attach Limitation:** EBS volumes are typically attached to a single node at a time, limiting shared access across multiple pods.
 - **Data Sharing:** Data cannot be easily shared across multiple instances, requiring additional configurations or solutions for shared access.
 - **Availability Zones:** EBS volumes are confined to a single Availability Zone, which may affect high availability and disaster recovery strategies. Kubernetes has implemented several mitigations for this limitation, and ECS likely has similar solutions.
 - **Scalability:** Requires manual provisioning and management of storage capacity, which can introduce operational complexity.
 
-## Recommendation
+## Decision
 
 **Both Amazon EFS and Amazon EBS are excellent choices for the default storage class in EKS clusters, depending on specific use cases and requirements.**