EFS mount target unreachable from EC2 instance in private subnet with custom security group

Cloud Platforms Amazon Web Services (AWS)
, , , , , , , ,
1

EC2 instances in our private subnet can’t mount the EFS file system. The mount command times out after several minutes without establishing a connection. The EFS mount target configuration shows as “Available” in the console, but our instances just can’t reach it.

We’ve verified the EC2 instances are in the same VPC as the EFS mount targets, and the subnet alignment looks correct. The mount command we’re using:

sudo mount -t nfs4 -o nfsvers=4.1 fs-12345678.efs.us-east-1.amazonaws.com:/ /mnt/efs
mount.nfs4: Connection timed out

The security group NFS rules seem to be configured - we have port 2049 open. But something is still blocking the connection. The storage outage is affecting our application deployment since we rely on shared EFS storage for configuration files.

What’s the typical culprit when EFS mount targets are unreachable from private subnets?

2

Here’s the complete solution addressing all three configuration areas for EFS connectivity:

1. EFS Mount Target Configuration: You must create an EFS mount target in every Availability Zone where you have EC2 instances that need to access the file system. Mount targets are AZ-specific resources:

# Create mount targets in each AZ
aws efs create-mount-target \
  --file-system-id fs-12345678 \
  --subnet-id subnet-az1-private \
  --security-groups sg-efs-mount-target

aws efs create-mount-target \
  --file-system-id fs-12345678 \
  --subnet-id subnet-az2-private \
  --security-groups sg-efs-mount-target

Each mount target gets a unique DNS name, but you can use the file system DNS name (fs-12345678.efs.region.amazonaws.com) which automatically resolves to the mount target in the same AZ as the mounting instance.

2. Security Group NFS Rules: Configure two security groups with proper NFS access:

EFS Mount Target Security Group:

{
  "IpPermissions": [{
    "IpProtocol": "tcp",
    "FromPort": 2049,
    "ToPort": 2049,
    "UserIdGroupPairs": [{
      "GroupId": "sg-ec2-instances",
      "Description": "Allow NFS from EC2 instances"
    }]
  }]
}

EC2 Instance Security Group: No specific outbound rules needed if you have default allow-all outbound. If restricted:

{
  "IpPermissionsEgress": [{
    "IpProtocol": "tcp",
    "FromPort": 2049,
    "ToPort": 2049,
    "UserIdGroupPairs": [{
      "GroupId": "sg-efs-mount-target"
    }]
  }]
}

3. VPC/Subnet Alignment: Verify your network configuration:

# Pseudocode - Verification steps:
1. Confirm EC2 instances and mount targets in same VPC
2. Check each mount target subnet is in correct AZ
3. Verify route tables for private subnets allow local VPC traffic
4. Ensure NACLs allow TCP 2049 bidirectional (inbound AND outbound)
5. Test DNS resolution: nslookup fs-12345678.efs.us-east-1.amazonaws.com
6. Verify mount target state is "available" in all AZs

Mount Command Best Practices:

# Use recommended mount options
sudo mount -t nfs4 -o nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport fs-12345678.efs.us-east-1.amazonaws.com:/ /mnt/efs

Troubleshooting Steps:

  1. Test connectivity from EC2: `telnet fs-12345678.efs.us-east-1.amazonaws.com 2049
  2. Check mount target network interfaces in EC2 console (under ENIs)
  3. Verify DNS resolution returns IP in same subnet range as EC2 instance
  4. Review VPC Flow Logs for rejected connections on port 2049
  5. Check CloudWatch Logs for EFS mount errors if using EFS mount helper

Common Issues:

  • Cross-AZ mounting without local mount target: Create mount target in every AZ
  • Security group references wrong group: Use security group IDs, not names
  • NACL blocking NFS: NACLs are stateless - need both inbound and outbound rules
  • DNS resolution failing: Check VPC DNS settings (enableDnsHostnames and enableDnsSupport)
  • Subnet route table missing local route: Should have route for VPC CIDR pointing to “local”

Best Practices:

  • Create mount targets in all subnets where instances run
  • Use security group references instead of CIDR blocks for dynamic security
  • Enable EFS encryption at rest for sensitive data
  • Implement lifecycle policies to automatically transition files to Infrequent Access storage class
  • Monitor EFS performance metrics in CloudWatch (BurstCreditBalance, ClientConnections)
  • Use EFS mount helper (amazon-efs-utils) for better error handling and automatic retries
  • Add mount entries to /etc/fstab for automatic mounting on instance reboot
  • Test failover by terminating instances in one AZ and verifying access from remaining AZs

Once you’ve created mount targets in all required AZs and configured security groups with proper NFS rules, your EC2 instances should mount the EFS file system successfully.

3

Good point about AZs. I just checked and we only have one mount target created, but our EC2 instances are spread across three AZs. Could that be the issue? Do we need a mount target in each AZ?

4

Beyond the AZ issue, double-check your NFS security group rules. The rule should allow TCP port 2049 from the source security group (your EC2 instances), not from a CIDR block. Using security group references is more secure and dynamic. Also verify there are no NACLs blocking port 2049 at the subnet level - security groups are stateful but NACLs are stateless.

5

First check - are your EC2 instances’ security group and the EFS mount target’s security group properly configured? The mount target security group needs to allow inbound NFS traffic (TCP port 2049) from the EC2 security group. It’s bidirectional trust, so both groups need to reference each other correctly.

6

Yes, absolutely! Each subnet where you have EC2 instances needs its own EFS mount target in the same AZ. You can’t mount EFS across AZs directly. Create mount targets in all three AZs where your instances run, each with the same security group that allows NFS traffic from your EC2 security group. This is a common oversight when setting up EFS.

7

Also verify your subnet routing. Private subnets need proper route table configuration. If your EC2 instances are in different availability zones than the mount targets, you need mount targets in each AZ where you have instances. EFS mount targets are AZ-specific, not region-wide.