Parsing for Key-Value Logs in Alert Logic Search

Structured logging uses the Key-Value (KV) format for several reasons, including easier parsing, classification, correlation, and conversion. Many systems use KV formatting for one or many of these benefits.

The Alert Logic console enables users to utilize search in order to extract valuable information from their logs. You can use this technique to analyze log messages where Alert Logic doesn't provide a dedicated log parser or not all fields in a given message are parsed.

This can be a useful technique when a particular log message type is highly configurable or varies widely based on the installed version of the software generating the log message. As long as the message conforms to the key-value format, you will be able access their contents using this method.

Confirm Whether Log Messages Are in Key-Value Format

To understand whether you have logs in KV format, you can filter your search results with EXISTS(parsed.kv). The query below will return the most recent log messages in KV format in your account:

SELECT 
  time_recv AS "Time Received", 
  message AS "Message", 
  parsed.kv AS "KV"
FROM logmsgs
WHERE 
  -- Only log messages in Key-Value format
  EXISTS(parsed.kv)
ORDER BY time_recv DESC
LIMIT 1000

To utilize this query immediately, open this query in the Alert Logic console.

Example Queries

#1: Extracting Data from Linux Kernel Audit Logs

One source of KV logs is the Linux kernel auditing system. A sample log might look like the following.

audit: event=getUserRights, user=Undef, id="12345:123456789", source=1.2.3.4

This is a very simple example of an auditd log, and Alert Logic will categorize this log message as type Unix System Call Audit. However, there are many possible audit log messages, based on local configuration. Because Alert Logic can parse the structure of KV logs, you can access data in these messages in many formats.

The above message will have four key-value pairs:

The following is an example of an Expert Mode search query for extracting data from Linux kernel audit logs.

SELECT time_recv AS "Time Received", message AS "Message"
FROM logmsgs
WHERE parsed.kv.event = 'getUserRights'
ORDER BY time_recv DESC
LIMIT 1000

To utilize this query immediately, open this query in the Alert Logic console.

#2: Aggregating Logs by Type

This is an example search query of aggregating logs by event, which enables you to simplify and speed up your search for the specific data you're searching for.

SELECT 
  COUNT(*) AS "Count", 
  LSET(message, 3) as "Messages", 
  parsed.kv.event AS "Event"
FROM logmsgs
WHERE 
  -- Auditd messages start with audit by default
  message CONTAINS 'audit' AND

  -- The event key is present in the message
  parsed.kv.event != null 
GROUP BY Event
ORDER BY Count DESC

To utilize this query immediately, open this query in the Alert Logic console.

Additional Resources

For more information on working with Alert Logic Search, see the following knowledge base articles:

• Aggregating Results in Alert Logic Search
• When should I use a scheduled search or create a correlation alert?
• A Brief Introduction to auditd