Container Security Frequently Asked Questions

Alert Logic Managed Detection & Response (MDR) gives you visibility into your containerized workloads, using a combination of container discovery and agent-based data collection. The following list of frequently asked questions is intended to help you better understand and proactively answer questions you may have about using Agent Containers. For more information about specific container types, capabilities, and best practices, refer to Container Security with Alert Logic. 

Frequently Asked Questions

• Where can I obtain the latest versions of the Alert Logic Container image, and where is it documented?
  
  The agent container repositories are documented in this README. The Container agent is updated whenever the standard agent is updated. For all OCI-compliant runtimes (contained, CRI-O, etc.), the yaml files pull the latest agent image from Docker Hub anyway, so on every reboot or reprovision, it will continuously get the latest version.
  
  
• The Alert Logic GitHub page for the Agent Container lists having a Threat Manager appliance as a pre-requisite for agent containers. If we do not intend to analyze network traffic and just want the agent to collect syslogs, do we still need an IDS appliance?
  
  If the only goal is to collect logs, the Alert Logic agent (Container or Standard base-host version) can be deployed without an IDS appliance. 
  
  
• What are the pre-requisites for the Alert Logic Agent Container or Standard Agent to be installed on hosts? Any requirements specific to the host configuration or network connectivity?
  
  The Agent Container must be deployed in Host-Network mode so it will see the base network interfaces of the node/host and use Privileged Mode (i.e. superuser access) to access other containers. The Standard Agent can be installed on the host/node and both the Container and Standard Agents have the same network access requirements for Linux agents. Those requirements are documented here:
  United States Firewall Rules
  United Kingdom and European Union Firewall Rules
  
  
• What exactly are the differences between the Agent Container and the Standard Agent? Why would I use one over the other?
  
  The Agent Container is essentially a regular agent running on a headless nix OS. It does not have access to the node’s filesystem or the filesystem of the other containers, so features that rely on native filesystem access (Agent Based Scanning, File Image Monitoring, Flatfile Collection, etc) will not work. Alternatively, the Standard Agent can be installed on the node/host, provided the host is not read-only/atomic. In this case, the agent still has access to the underlying filesystem, so agent features that rely on that access are still available.
  
  
• If I wanted to install the regular agent on the host/node instead of the Agent Container, which OS are supported?
  
  The base host agent has been tested with the operating systems listed in this document. 
  
  
• Should I use the Agent Container or the Standard Agent to support atomic (read-only) operating systems?
  
  Only the Agent Container can be deployed on “atomic” systems with read-only file systems (Fedora CoreOS, Amazon Bottlerocket, etc.) where packages cannot be installed on the base host and the only method of software deployment is via containers. For all other rpm-based and dpkg-based OS with mutable file systems, either agent type can be used.
  
  
• Does Alert Logic support Istio? If so, how does Istio support work?
  
  

  In the case of Istio our container agent works a little different. Istio injects a proxy (called Envoy) sidecar, running in the same pod as the application container. Envoy typically communicates with the application in clear text over the loopback interface. Normally we would capture traffic going into and out of the pod, on the pod's external virtual network interface, but in the Istio case that is encrypted with mTLS; so if we detect Istio, we instead switch to capturing traffic on the pod's internal loopback interface, between the proxy and the application (which is not encrypted, unless the application itself is doing additional encryption).

  If logs are outputted to stdout/stderr, then we can receive them, including Istio logs themselves.
  
  

• How does the Agent Container batch logs?
  
  The Agent Container performs normal fwrite() and fflush() to store logs on disk for transport. It will accumulate 8,000 messages in a batch or up to 16 MB, whichever is first, before closing the batch for transport. The agent does not lock stdout/stderr container streams when working with the logs.
  
  
• Am I able to scan containers? What types of vulnerabilities would I expect to see?
  
  Because of the minimalistic nature of containers, we won't have the requisite packages to run the types of tests you would see in ABS or Authenticated scans. However, port scans and other network-based tests should still return results in most cases, though their source container may be hard to attribute unless XFF is configured.
  
  
• How do we configure our web server to write logs to stdout/stderr?
  
  Apache2
  1. Edit /etc/httpd/conf/httpd.conf
  2. Add the following two lines:

  ErrorLog /dev/stderr
  TransferLog /dev/stdout

  3. Save file and restart Apache service.
  
  NGINX
  1. Edit /etc/nginx/nginx.conf
  2. Add the following lines to the HTTP object:

  access_log /dev/stdout main;
  error_log /dev/stderr;

  3. Save file and restart NGINX service. 
  
  
• How do we configure X-Forwarded-For headers for our web server?
  
  Tomcat
  1. Edit $CATALINA_HOME/conf/server.xml to include the following:

  <Valve className="org.apache.catalina.valves.RemoteIpValve"
  internalProxies=<insert your internal proxy IPs here, or can use regex>
  remoteIpHeader="X-Forwarded-For"
  remoteIpProxiesHeader="X-Forwarded-By"
  protocolHeader="X-Forwarded-Proto" />

  2. Follow the guidance in Web Log Analytics (WLA) Configuration for configuring log format.
  
  Apache2
  1. Activate the remoteip Apache module that comes with Apache:

  sudo a2enmod remoteip

  2. Navigate to /etc/httpd/conf/apache2.conf and uncomment the following line:

  #LoadModule remoteip_module modules/mod_remoteip.so

  3. Save the file.
  4. Navigate to /etc/apache/conf-available/remoteip.conf and add the following lines:

  RemoteIPHeader X-Forwarded-For
  RemoteIPTrustedProxy <IP address of your load balancer or proxy>

  5. (Optional) If the end-user has an IP in the internal network, you will want to use the following instead:

  RemoteIPHeader X-Forwarded-For
  RemoteIPInternalProxy <IP of internal proxy>

  6. Configure the logging according to Web Log Analytics (WLA) Configuration. 
  
  NGINX
  1. Edit /etc/nginx/nginx.conf.
  2. Add the following line:

  proxy_set_header X-Forwarded-For $remote_addr;

  3. Configure logging according to Web Log Analytics (WLA) Configuration.
  4. Save file and restart NGINX service. 
  
  Note: You can also use the 'real_ip' module if you have compiled your nginx with the --with-http-realip-module option. In this case, you need to modify nginx.conf as follows:

  # Directives for setting real_ip/XFF IP address in log files
  
  set_real_ip_from <IP address of master LB>
  
  set_real_ip_from <IP Address of slave LB>
  
  real_ip_header X-Forwarded-For;

• Does Alert Logic support File Integrity Monitoring (FIM) for containers?
  
  Due to the minimalistic nature of containers, Alert Logic File Integrity Monitoring, Flat-File Ingestion, and Agent-Based Scanning do not function within them.
  
  
• What is the best way to collect logs? Can specific log types be filtered out?
  
  Both the Standard Agent and the Agent Container hook into the container to pull logs from stdout/stderr. At this time, specific types of logs cannot be filtered out; Alert Logic can only turn off container log collection entirely via the syslog policy in the deployment.
  
  
• If I have multiple volumes, how can I make sure the Agent Container ingests from all of them?
  
  You can add multiple volume mounts and volumes in the provided YAML file; simply follow the same format as the defaults and point them to your other log volumes.
  
  
• What is the security value of traffic/logs from containers?
  
  Traffic has the same security value as its non-containerized analog, so the same web-based incidents from web servers, etc., can be expected. Alert Logic does not currently generate many incidents for container-based logs, so their collection is mostly for audit, compliance, and forensic purposes.
  
  
• Why do my containers show up with multiple IP addresses in the console?
  
  Often a node has multiple containers. Alert Logic enumerates these from the container’s metadata, as well as the host’s metadata that has all base host and container IPs merged into a single list.
  
  
• Is deploying the Agent Container going to increase traffic going across my network?
  
  Due to the nature of traffic capture, you could double, triple, or even quadruple the amount of traffic being mirrored across the network, and much of it will be redundant. Alert Logic captures pod-pod traffic, in/out pod eth0, and in/out host eth0, but not intrapod or intrahost traffic. This question is a worthy consideration when deciding whether to use agent-based IDS on container workers, as it can degrade network performance or increase IDS packet drop rate.
  
  Note: In the case of Istio, Alert Logic turns on loopback on the pod and disables the pod's eth0 capture. In this case, Alert Logic does monitor intrapod traffic but not intrahost traffic. 
  
  
• Does Alert Logic support the inclusion of the Agent Container on deployment configurations, since the containers will rapidly be spun up and down?
  
  Only one instance of Agent Container is needed per node/host. The agent will monitor the node instance for other containers going up and down to attach to their network interfaces and log streams. Therefore, node/host deployment templates should include an Alert Logic Agent Container (a daemonset takes care of this on Kubernetes), but individual tasks or pods do not need the agent.
  
  
• Why does the Fargate container agent require fluentbit to operate?
  
  The fluentbit sidecar is used to convert logs from the default format (fluentbit/fluentd) into the format used by the Alert Logic agent, which is syslog over TCP. It is also useful for routing logs to multiple destinations, such as Alert Logic and CloudWatch.