Quarantine an EC2 Instance

If you have identified that one of your Amazon Web Services (AWS) instances has been compromised, you need to contain the threat, restore service, determine, and remediate the root cause. The capabilities in AWS give you some specific options to help with containment. In AWS, there are multiple ways to respond to this incident.

To contain, you may choose to quarantine, stop, and/or terminate the instance.

Method 1 - Quarantine the Instance

By quarantining an instance, you can prevent future harm while leaving the instance running to allow for an investigation into the cause of the compromise.

Considerations

Before performing the steps to quarantine the instance, you need to determine if the instance is part of an Auto Scaling group or attached to a load balancer. If the instance was created by Auto Scaling, you must first detach the instance from the Auto Scaling group to prevent it from being terminated by the Auto Scaling service due to failing health checks once it has been quarantined.

For an instance attached to an Elastic Load Balancer, you will need to deregister it from the Elastic Load Balancer to prevent the Elastic Load Balance from attempting to send it further requests.  If the instance is part of a target group associated with a Network Load Balancer or Application Load Balancer, you will need to delete the instance from the target group. 

Performing the Quarantine

Once the above steps are completed, you can quarantine the compromised instance by changing the security groups associated with all the instance’s Elastic Network Interfaces (ENIs) to a security group that has no allow rules for either ingress or egress and thus will deny all traffic in or out of the instance.

IMPORTANT: Be sure that you create a new security group to apply to the instance. DO NOT modify the existing security group as that will affect other instances. Also be sure to remove the default ‘Allow All’ rule for egress on the new security group used to quarantine the instance.

At this point, the malware will be unable to spread, but it may still be actively working on the system, encrypting data for ransom, searching for sensitive information to exfiltrate, probing the security group for an open outbound port, or possibly self-destructing.

Once the instance has been quarantined, perform forensic analysis to determine the root cause of the compromise. Only security experts should investigate the root cause of a compromise; otherwise, valuable forensic data could be inadvertently lost, or the threat could spread.

When your incident response team is ready to investigate, you can change the security group to allow inbound SSH and RDP from an instance. The statefulness of security groups will allow you to connect to the instance without allowing any egress rules on the affected host.

If you are concerned about the effects of some malware (ransomware) on an instance, you can stop the instance.

Method 2 - Stop the Instance

If you want to stop malware activities once the instance has been isolated, or take a snapshot to send to forensic investigators, you should stop the instance and take an offline Amazon EBS snapshot of the root EBS volume and any other volumes that may contain information needed as part of the investigation.  Forensic investigators can determine the root cause, the impact of the compromise, and the remediation required to prevent it in the future.

If the instance is using an instance-store AMI, you cannot stop the instance. You should consider quarantining the instance with a security group instead to make sure no data is lost or terminating the instance.

Method 3 - Terminate the Instance

If you will not be performing in-depth forensics, you may always terminate the instance. However, until security experts perform a root cause analysis and remediate the issue, this instance and other instances will continue to be compromised.

Once you have contained the threat, you can launch a new instance of the same type or launch a snapshot from before the compromise occurred. Once you have determined a root cause, you should update your AMI to include any needed patches or configuration changes and ensure that the vulnerable AMI is not referenced by any Auto Scaling groups or automation tools.