Parsed JSON searching is built into the improved log search and allows you to target exact fields from any JSON log, regardless of whether that field has been parsed by Alert Logic. You can find the improved log search within the Alert Logic console – for Alert Logic Cloud Defender, Threat Manager, and Log Manager customers at Search > Log Search or Log Search BETA and for Alert Logic Essentials, Professional, and Enterprise customers at the navigation menu () > Investigate > Search > Log Search.
In some cases, parsers for a particular log may not be as granular as you would like due to countless variations in log types. JSON parsing allows you to be more precise about the values you're looking for.
Example
If you want to know the requestID from a particular AWS log, the traditional content token will come back as follows:
"tableName”: “Data Table 12345”},”responseElements”: null, “additionalEventData”: null, “requestID”:
“AISNSAFKNJ12J32J34568K1J8JASDL3456FNA”, “eventID”: “a12b34c5-1234-5678-def1-12345a6b7cd",
"readOnly": true,"resources": [{"accountId": "11111111111","type": "AWS::DynamoDB::Table","ARN":
"arn:aws:dynamodb:us-west-2:11111111111:table/Data Table 12345"}],"eventType": "AwsApiCall",
"apiVersion": "2012-08-10","managementEvent": true,"recipientAccountId": "11111111111"
This result is as granular as possible with content tokens, which makes it difficult to build specific filters and almost impossible to aggregate on. In comparison, a parsed JSON token will return the following value:
“AISNSAFKNJ12J32J34568K1J8JASDL3456FNA”
This value is much more concise and allows for easy aggregation.
Build a Parsed.JSON Token
In order to parse a JSON log message, you must find the path through the JSON hierarchy and plug this into the parsed.json syntax. Examples include:
-
[parsed.json.xxx]
-
[parsed.json.xxx.xxx.xxx]
The easiest way to decipher the JSON hierarchy is to view a JSON log within the console. Complete the following steps:
- Find an example of the type of log you want to parse within the Alert Logic console.
- Click the
button to view the log in the console, then scroll down the "JSON" section.
- Once you have formatted the JSON, find the field that you want to specify and build your parsed.json token. It will always start with the prefix "parsed.JSON" and you will always need to add the hierarchy to the specific field.
Examples
Unparsed Log
3aws: {
“eventTime”: “ 2019-02-13T15:37:54Z “,
“eventSource”: “ dynamob.amazonaws.com “,
“eventName”: “DescribeTable”,
“eventVersion”: 1.06”,
“awsRegion”: “us-west-2",
“userIdentity”: {
“type”: “AssumedRole”,
“principalId”: “1234567890:AssumeRoleSession”,
“arn”: “arn:aws:sts::11111111111:assumed-role/Role-111/AssumeRoleSession”,
“accessKeyId”: “ASIAXM2656UYIC”,
“userName”: null,
“sessionContext”: {
“sessionIssuer”: {
“type”: “Role”,
“principalId”: “1234567890”,
“arn”: “arn:aws:iam::11111111111:role/Role-111”,
“accountId”: “11111111111”,
“userName”: “Role-111”
},
“attributes”: {
“creationDate:” “2019-02-13T15:07:10Z”,
“mfaAuthenticated”: “false”
}
Simple Parsing
Using the above unparsed log, build a token that targets "eventName".
First level: "eventName"
Completed token: [parsed.JSON.eventName]
Return: DescribeTable
Complex Parsing
Using the above unparsed log, build a token that targets "accountID".
First level: "userIdentity"
Second level: "sessionContext"
Third level: "sessionIssuer"
Fourth level: "accountID"
Completed token: [parsed.JSON.userIdentity.sessionContext.sessionIssuer.accountID]
Return: 11111111111
Use a Parsed JSON Token
Parsed.JSON tokens act just like any other content token in the log search. They can be used in the WHERE field for filtering or in the SELECT field to present specific data.
WHERE Field Example
SELECT Field Example
JSON Parsing Limitations
Some fields in JSON logs live inside of lists. Lists are easy to spot because they are contained within square brackets - []. While log search can grab entire contents of a list, it cannot grab individual fields within a list.
Example
“eventID”: “a12b34c5-1234-5678-def1-12345a6b7cd",
“readOnly”: true,
“resources”: [
{
“accountID”: “1234567890”,
“type”: “AWS:DynamoDB::Table”,
“ARN”: “arn:aws:dynamodb:us-west-2:1234567890:table/prod-1A2B3C4D5E6F”
}
]
Using parsed.json tokens, you can grab all of the fields under the "resources" list:
Token: [parsed.json.resources]
Return: [{“accountID”: “1234567890”, “type”: “AWS:DynamoDB::Table”, “ARN”: “arn:aws:dynamodb:us-west-2:1234567890:table/prod-1A2B3C4D5E6F” }]
However, attempting to pull a single field out of those brackets will result in an undefined return:
Token: [parsed.json.resources.type]
Return: undefined
Comments
0 comments
Please sign in to leave a comment.