(AWS) Elastic File System (EFS)

Early in my experience with AWS I discovered EFS and made use of a persistent parallel file system a tool for launching a clone of the LAM Alaska website on demand. I initially chose the us-west-2 Oregon region to operate in in 2017 and didn't use another region until 2024. By that time sucess was defined as the ability to launch a fully functional clone of the LAM Alaska website with date from backups created from the main website within the last 24 hours.

Mount the LAM aws-efs for the region the instance was launched in

I created the aws-efs directory on my github pages website to allow metadata retrieval somewhat like the Instance Metadata Service as implemented an AWS. I use AWS Instance Metadata Service Version 2 (IMDSv2) to retrieve the REGION and then retrieve the EFS DNS name used to mount the volume with NFS from my github pages website.

The cloud-init section to determine the EFS name for the region this instance was launched in:

 - echo
 - echo 'AWS LAM Get EFS for REGION'
 - wget --quiet https://lamurakami.github.io/aws-efs/REGION.bash
 - chmod a+x REGION.bash
 - export REGION=$(./REGION.bash)
 - rm REGION.bash
 - EFS=$(curl -s https://lamurakami.github.io/aws-efs/${REGION})

The availability of this metadata to the cloud-init service during initialization allows me to mount the Elastic File System (EFS) for that region with NFS. The github pages website seems to be high availability and reliable and provides both IPv4 and IPv6 connectivity unlike the main github site where repositories are housed which as of Tuesday, February 20, 2024 does not preovide IPv6 access.

The cloud-init section to mount the AWS VPC Elastic File System with nfs4:

 - echo
 - echo 'AWS LAM Adding nfs4 mount to AWS VPC Elastic File System'
 - mkdir /mnt/efs
 - nfsOpt="_netdev,noresvport,nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 0 0"
 - echo "${EFS}:/ /mnt/efs nfs4 ${nfsOpt}" >> /etc/fstab
 - systemctl daemon-reload
 - mount -a -t nfs

The above mount section has been included in my cloud-init files since I became dependant on it's availability and ease of use but until 2024 when I started expaning to other regions the file system name was hard coded. When a EFS is created within a region a unique file system ID is assigned and a DNS name like <file system ID>.efs.<region>.amazonaws.com is assigned but is only usable in the VPC where the EFS was created and by default is accessable only in that region.

During the process of modifying my scripts to work in any region I reduced the EFS dependency to the ubuntu.tgz and ec2-user.tgz user resource files. The ubuntu.tgz file includes some private resources including aws and duckdns credentials specific to an AWS LAM clone.

Many of the other dependencies were moved from AWS EFS to a private AWS S3 bucket. Accessing the private AWS S3 bucket requires credentials which are in the ubuntu.tgz and ec2-user.tgz user resource files so mounting the EFS is a requirement to accessing the private AWS S3 bucket.

An EFS by default can be accessed only from the region it is created in

By default EC2 instances running in multiple Availability Zones within the same AWS Region can access the Elastic File System (EFS) for that region. An EFS can be created to server only One Availability Zone but by default a mount target is created for each Availability Zone within the region.

A Domain Name Service (DNS) name is assigned for each EFS which resolves to the IP address of the EFS mount target in the same Availability Zone as the EC2 instance querying the DNS. Connectivity and DNS Name resolution is only available on the private IPv4 subnets of the VPC.

I have decided not to use AWS Backup, EFS Replication, lifecycle management or encryption on the filesystems I am creating in additional regions. I am not sure any of these features were available when I created the Oregon EFS years ago.

Create an EFS in a region

Check 172.31.0.0/16 NFS for EFS inbound rule in Security Group
Create EFS security group with NFS for EFS inbound rule
Create EFS - aws-efs - use customize option
Attach EFS security group instead of default on second customize page for the interfaces for each Availability Zone in the region

Populate EFS with the ubuntu.tgz and ec2-user.tgz user resource files

ll -aRh /mnt/efs

sudo chown ubuntu:ubuntu /mnt/efs
sudo chown ec2-user:ec2-user /mnt/efs
sudo chown admin:admin /mnt/efs

mkdir /mnt/efs/aws-lam1-ubuntu

scp -p lam@aws:/mnt/efs/aws-lam1-ubuntu/ubuntu.t* \
/mnt/efs/aws-lam1-ubuntu

mkdir /mnt/efs/Amazon-Linux-2023

scp -p lam@aws:/mnt/efs/Amazon-Linux-2023/ec2-user.t* \
/mnt/efs/Amazon-Linux-2023