To send an external request, the request headers should contain the following Headers:
Content-Type: application/JSON
Authorization: Bearer {{Alerts, Rules, and Tags API key}}
Alerts, Rules, and Tags API key can be found under Data Flow –> API Keys.
NOTE – Only users with Admin privileges have access to the Rules API and keys.
This guide will help you use our Rules API to create, read, update or delete parsing rules and rule groups via API. Parsing rules are organized inside Rule Groups. Each group has a name and a set of parsing rules with a logical AND/OR relationship between them.
Logs are processed according to the order of the Rule Groups (top to bottom) and then by the order of rules within the Rule Group and according to the logical operators between them (AND/OR).
Group Name
Sub-Group / Condition
Rule Name
My group 1
My rule 1-1-a
OR
My rule 1-1-b
AND
My rule 1-2-a
OR
My rule 1-2-b
My group 2
My rule 2-1-a
Create a Rule
To create a parsing rule, there is a need to define a rule group and the parsing rule itself:
Method: POST URL: https://api.<domain>/api/v1/external/rule/rule-set Body (the following fields are required/optional):
"name": (string)(mandatory) – the name of the rules group,
"description": (string)(optional – default is an empty string) – the description for the rules group,
"enabled": (boolean)(optional) – indicates rather the group is active or not when created – if fields is missing or value is empty – default is True,
"creator": (string)(optional – default is “Coralogix external API”) – the name of the group creator,
"order": (number)(optional) – the rule group order number. If it is not provided, the rule group will be added at the end of the list of rule groups. If the group with the provided order number already exists, new group will be added one below and the order will be changed to +1 automatically,
"ruleMatchers":(array of objects)(mandatory) – the conditions which the group of rules works on – application/subsystem/severity of logs. Each object in the array should contain:
"field": (string) can be any of the following – applicationName/ subsystemName/ severity,
"constraint": (string)- the value for the chosen field (if the field is severity, the value should be one: debug, verbose, info, warning, error, critical)
"rulesGroups":(array of objects)(optional) – It contains two fields:
"order": (number) – it is the number of the sub-group. If rules are under two different sub-groups then it means there is AND condition between them
"rules": (array of object) – each object is a rule
The rules object keys:
"name": (string)(mandatory) – the name of rule,
"description": (string)(optional – default is an empty string) – the description for the rule,
"enabled": (boolean)(optional) – indicates rather the rule is active or not when created – if fields is missing or value is empty – default is True,
"type": (string)(mandatory) – type of the rule – (block, extract, parse, jsonextract, replace, timestampextract, removefields),
"sourceField": (string)(optional) – If not stated in the body then the default source field will be the log text. If you want to run the rule against any internal log JSON field use Text.field_name.field_name2…
"destinationField":(string)(optional) – If not stated in the body then the default destination field will be the log text. If you want to run the rule against any internal log JSON field use Text.field_name.field_name2…
"order": (number)(optional) – if empty then the rule is added at the end
"rule": (string)(mandatory – except in “removefields” and “timestampextract” – for more info look at restrictions) – the regex of the rule (must be a regex that can be compiled)
Rule Type
Description / Restrictions
BLOCK, EXTRACT
API request body cannot have: “ReplaceNewVal” OR “DestinationField”
PARSE
API request body cannot have: “ReplaceNewVal”
JSONEXTRACT
API request body cannot have: “ReplaceNewVal”. API request body must have: “DestinationField”:(string) – need to be one of: “category”, “className”, “methodName”, “severity”, “threadId”
REPLACE
API request body must have: “ReplaceNewVal”
REMOVEFIELDS
API request body must have: “rule”:(string) – comma-separated list of json fields you would like to remove
TIMESTAMPEXTRACT
API request body cannot have: “rule” or “destinationField” fields. API request body must have:”timeFormat”(string) – you can see our suggestion for each type of standard, normally you will need to change the format to match your time field string pattern with the while comply to the standard “formatStandard”:(string) = need to be one of: “javasdf”, “golang”, “strftime”, “secondsts”, “millits”, “microts”, “nanots”). API request body should contain a meaningful value for “sourceField”: (string) – a value is not specified the default source files will be the text field and the rule will not work.You should set the value as Text.<json_key_with_time_field>)
STRINGIFY JSON FIELD
Convert JSON object to JSON string. Find out more here.
PARSE JSON FIELD
Convert JSON string to JSON object. Find out more here.
Examples
A group with one parsing rule
{
"name": "My test group with one rule - parse",
"description": "It is an example how to create a group with a parsing rule",
"enabled": true,
"order": 1,
"creator": "[email protected]",
"ruleMatchers": [],
"rulesGroups": [
{
"order": 1,
"rules": [
{
"name": "Test parse rule",
"description": "This is the first parsing rule",
"enabled": true,
"type": "parse",
"sourceField": "text",
"destinationField": "text",
"order": 1,
"rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$"
}
]
}
]
}
A group with two parsing rules and an AND condition between them
{
"name": "My test group with one rule per each sub-group",
"description": "It is an example how to create a group with two rules with AND condition between them ",
"enabled": true,
"order": 1,
"creator": "[email protected]",
"ruleMatchers": [],
"rulesGroups": [
{
"order": 1,
"rules": [
{
"name": "Test parse rule",
"description": "This is the first parsing rule",
"enabled": true,
"type": "parse",
"sourceField": "text",
"destinationField": "text",
"order": 1,
"rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$"
}
]
},
{
"order": 2,
"rules": [
{
"name": "Test replace rule",
"description": "This is the first replace rule",
"enabled": true,
"type": "replace",
"sourceField": "text.message",
"destinationField": "text.message",
"order": 1,
"rule": "host",
"replaceNewVal": "hostname"
}
]
}
]
}
A group with 3 parsing rules
{
"name": "My test group with three rules",
"description": "It is an example how to create a group with 3 rules",
"enabled": true,
"order": 5,
"creator": "[email protected]",
"ruleMatchers": [],
"rulesGroups": [
{
"order": 1,
"rules": [
{
"name": "Test parse rule",
"description": "This is the first parsing rule",
"enabled": true,
"type": "parse",
"sourceField": "text",
"destinationField": "text",
"order": 1,
"rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$"
}
]
},
{
"order": 2,
"rules": [
{
"name": "Test replace rule 1",
"description": "This is the first replace rule",
"enabled": true,
"type": "replace",
"sourceField": "text.message",
"destinationField": "text.message",
"order": 1,
"rule": "host",
"replaceNewVal": "hostname"
},
{
"name": "Test replace rule 2",
"description": "This is the second replace rule",
"enabled": true,
"type": "replace",
"sourceField": "text.message",
"destinationField": "text.message",
"order": 2,
"rule": "server",
"replaceNewVal": "hostname"
}
]
}
]
}
Get All Rules
To get all parsing rules:
Method: GET URL: https://api.<domain>/api/v1/external/rules
In response, you will get the "companyRulesData" array with groups and rules.
Example
{
"companyRulesData": [
{
"id": "20538afe-12d4-4032-a2d9-42b14e0036a1",
"name": "My test group with one rule - parse",
"description": "It is an example how to create a group with a parsing rule",
"order": 1,
"enabled": true,
"creator": "[email protected]",
"updatedAt": "2022-11-03 08:32:53",
"createdAt": "2022-11-03 08:32:53",
"ruleMatchers": null,
"rulesGroups": [
{
"id": "562cbdf3-5021-42ab-a85a-23c0e80db62b",
"order": 1,
"rules": [
{
"id": "f3ddb109-c80d-4a46-8aa9-98f2984fb2f7",
"name": "Test parse rule",
"description": "This is the first parsing rule",
"enabled": true,
"ruleMatchers": null,
"rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$",
"sourceField": "text",
"destinationField": "text",
"replaceNewVal": "",
"type": "parse",
"order": 1,
"keepBlockedLogs": false,
"deleteSource": false,
"escapedValue": false,
"overrideDest": false
}
]
}
],
"hidden": false
},
{
"id": "94b0a95b-994e-42ef-9c22-48d58b2e339e",
"name": "My test group with one rule per each sub-group",
"description": "It is an example how to create a group with two rules with AND condition between them ",
"order": 2,
"enabled": true,
"creator": "[email protected]",
"updatedAt": "2022-11-03 08:52:51",
"createdAt": "2022-11-03 08:52:39",
"ruleMatchers": null,
"rulesGroups": [
{
"id": "4dfde3e7-2791-4b34-aa0a-3edafe719ddb",
"order": 1,
"rules": [
{
"id": "5802ec65-dae5-4e20-bd1d-37dbf15e2630",
"name": "Test parse rule",
"description": "This is the first parsing rule",
"enabled": true,
"ruleMatchers": null,
"rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$",
"sourceField": "text",
"destinationField": "text",
"replaceNewVal": "",
"type": "parse",
"order": 1,
"keepBlockedLogs": false,
"deleteSource": false,
"escapedValue": false,
"overrideDest": false
}
]
},
{
"id": "ca69375b-a869-4c6f-96eb-14a4e6c8a935",
"order": 2,
"rules": [
{
"id": "62e46153-a56a-4322-a1be-c4db871afaec",
"name": "Test replace rule",
"description": "This is the first replace rule",
"enabled": true,
"ruleMatchers": null,
"rule": "host",
"sourceField": "text.message",
"destinationField": "text.message",
"replaceNewVal": "hostname",
"type": "replace",
"order": 1,
"keepBlockedLogs": false,
"deleteSource": false,
"escapedValue": false,
"overrideDest": false
}
]
}
],
"hidden": false
}
]
}
As you can see, each group contains the id field which is the GROUPID.
The GROUPID can be also obtained from the Parsing Rules page.
Go to Data Flow → Parsing Rules.
Click on the rule group you would like to edit.
Look at the URL in the browser address bar.
Get a Rule Group
Query for parsing rules from the specific group.
Method: GET URL: https://api.<domain>/api/v1/external/rule/rule-set/<GROUPID>
{
"id": "20538afe-12d4-4032-a2d9-42b14e0036a1",
"name": "My test group with one rule - parse",
"description": "It is an example how to create a group with a parsing rule",
"order": 1,
"enabled": true,
"creator": "[email protected]",
"updatedAt": "2022-11-03 08:32:53",
"createdAt": "2022-11-03 08:32:53",
"ruleMatchers": null,
"rulesGroups": [
{
"id": "562cbdf3-5021-42ab-a85a-23c0e80db62b",
"name": "",
"order": 1,
"enabled": true,
"rules": [
{
"id": "f3ddb109-c80d-4a46-8aa9-98f2984fb2f7",
"name": "Test parse rule",
"description": "This is the first parsing rule",
"enabled": true,
"ruleMatchers": null,
"rule": "^(?P<timestamp>\\\\d{2}-\\\\d{2}-\\\\d{4}\\\\s*\\\\d{2}:\\\\d{2}:\\\\d{2}\\\\.\\\\d+)\\\\s+(?P<message>.*)$",
"sourceField": "text",
"destinationField": "text",
"replaceNewVal": "",
"type": "parse",
"order": 1,
"keepBlockedLogs": false,
"deleteSource": false,
"escapedValue": false,
"overrideDest": false
}
],
"type": "user"
}
]
}
Modify Rules
Modify or update a group with one parsing rule.
Method: PUT URL: https://api.<domain>/api/v1/external/rule/rule-set
To update any rule just perform GET with a GROUPID, modify all rules in the group from the response, and send it with PUT request.
Note: The parameter ruleMatchers at the group section is returned by the GET request. It needs to be changed to the empty array [] before sending the request.
Note: It is not possible to delete a sub-group of rules.
Export Multiple Rules and Groups
If there is a need to copy all rules from one team to another follow these steps:
Export all groups and rules as described above in the section “Get all rules”.
Create and send a new request with: a) the Rules API key of the target team, b) URL: https://api.<domain>/api/v1/external/rules/export c) The body should contain the JSON object you got from executing the GET request in step 1. Copy and paste the output of the GET request to the body of the POST request without changing anything. A message stating “Group and Rules transformed successfully” will be prompted once the transfer is complete.
NOTE: When moving rules from one account to another keep in mind that the rule groups you imported are concatenated after the existing rules groups, we do not override existing groups by default. If you want to override the existing groups you should first remove them entirely before importing new rule groups.
Legacy REST API Methods
The REST API methods below are the old ways to manage parsing rules. They do not support the order of rules and fields names may be different. They are still supported.
Groups API
Create a Group
Create a group which will keep your rules.
Method: POST URL: https://api.<domain>/api/v1/external/group Body (the following fields are required/optional):
"name": (string)(mandatory) – the name of the rules group,
"description": (string)(optional – default is an empty string) – the description for the rules group,
"enabled": (boolean)(optional) – indicates rather the group is active or not when created – if fields is missing or value is empty – default is True,
"creator": (string)(optional – default is “Coralogix external API”) – the name of the group creator,
"ruleMatchers":(array of objects)(mandatory) – the conditions which the group of rules work on – application/subsystem/severity of logs. Each object in the array should contain:
"field": (string) can be any of the following – applicationName/ subsystemName/ severity,
"constraint": (string)- the value for the chosen field (if the field is severity, the value should be one: debug, verbose, info, warning, error, critical)
Example request:
{
"name": "My group 1",
"description": "This is a test group",
"enabled": true,
"creator": "[email protected]",
"ruleMatchers": [{
"field": "applicationName",
"constraint": "test_app"
}, {
"field": "subsystemName",
"constraint": "test_sub"
}, {
"field": "severity",
"constraint": "warning"
}, {
"field": "severity",
"constraint": "error"
}, {
"field": "severity",
"constraint": "critical"
}]
}
Example response:
The rule was created at the bottom of all rule groups.
The first step is to create a group which will keep our rules.
Method: GET URL: https://api.<domain>/api/v1/external/group/GROUPID Body: Empty
Update a Group
The first step is to create a group that will keep our rules.
Method: PUT URL: https://api.<domain>/api/v1/external/group/GROUPID Body (the following fields are required/optional):
"name": (string)(mandatory) – the name of the rules group,
"Description": (string)(optional – default is an empty string) – the description for the rules group,
"enabled": (boolean)(optional) – indicates rather the group will be set to active or not when updating,
"ruleMatchers":(array of objects)(mandatory) – the conditions which the group of rules work on – application/subsystem/severity of logs. Each object in the array should contain:
"field": (string) can be any of the following – applicationName/ subsystemName/ severity,
"constraint": (string)- the value for the chosen field (if the field is severity, the value should be one: debug, verbose, info, warning, error, critical)
Method: POST URL: https://api.<domain>/api/v1/external/rule/GROUPID Body (the following fields are required/optional):
"type": (string)(mandatory) – type of the rule – (block, extract, parse, jsonextract, replace, timestampextract, removefields),
"description": (string)(optional – default is an empty string) – the description for the rule,
"enabled": (boolean)(optional) – indicates rather the rule is active or not when created – if fields is missing or value is empty – default is True,
"name": (string)(mandatory) – the name of the rules group,
"rule": (string)(mandatory – except in “removefields” and “timestampextract” – for more info look at restrictions) – the regex of the rule (must be a regex that can be compiled),
"sourceField": (string)(optional) – If not stated in the body then the default source field will be the log text. If you want to run the rule against any internal log JSON field use text.field_name.field_name2…
"destinationField":(string)(optional) – If not stated in the body then the default destination field will be the log text. If you want to run the rule against any internal log JSON field use text.field_name.field_name2…
Rule Type
Description / Restrictions
BLOCK, EXTRACT
API request body cannot have: “ReplaceNewVal” OR “DestinationField”
PARSE
API request body cannot have: “ReplaceNewVal”
JSONEXTRACT
API request body cannot have: “ReplaceNewVal”. API request body must have: “DestinationField”:(string) – need to be one of: “category”, “className”, “methodName”, “severity”, “threadId”
REPLACE
API request body must have: “ReplaceNewVal”
REMOVEFIELDS
API request body must have: “rule”:(string) – comma-separated list of json fields you would like to remove
TIMESTAMPEXTRACT
API request body cannot have: “rule” or “destinationField” fields. API request body must have:”timeFormat”(string) – you can see our suggestion for each type of standard, normally you will need to change the format to match your time field string pattern with the while comply to the standard “formatStandard”:(string) = need to be one of: “javasdf”, “golang”, “strftime”, “secondsts”, “millits”, “microts”, “nanots”). API request body should contain a meaningful value for “sourceField”: (string) – a value is not specified the default source files will be the text field and the rule will not work.You should set the value as Text.<json_key_with_time_field>)
STRINGIFY JSON FIELD
Convert JSON object to JSON string. Find out more here.
PARSE JSON FIELD
Convert JSON string to JSON object. Find out more here.
Read a Rule
Method: GET URL: https://api.<domain>/api/v1/external/rule/RULEID/group/GROUPID Body: Empty
Update a Rule
Method: PUT URL: https://api.<domain>/api/v1/external/rule/RULEID/group/GROUPID Body (the following fields are required/optional):
"type": (string)(mandatory) – type of the rule – (block, extract, parse, jsonextract, replace, timestampextract, removefields),
"description": (string)(optional – default is an empty string) – the description for the rule,
"enabled": (boolean)(optional) – indicates rather the rule is active or not when created – if fields is missing or value is empty – default is True,
"name": (string)(mandatory) – the name of the rules group,
"rule": (string)(mandatory – except in “removefields” and “timestampextract” – for more info look at restrictions) – the regex of the rule (must be a regex that can be compiled),
"sourceField": (string)(optional) – If not stated in the body then the default source field will be the log text. If you want to run the rule against any internal log JSON field use text.field_name.field_name2…
"destinationField":(string)(optional) – If not stated in the body then the default destination field will be the log text. If you want to run the rule against any internal log JSON field use text.field_name.field_name2…
Note: It is possible to modify only fields which you want to edit but keep in mind that if you do a change which impact other fields then those fields need to be also updated.
Rule Type
Description / Restrictions
BLOCK, EXTRACT
API request body cannot have: “ReplaceNewVal” OR “DestinationField”
PARSE
API request body cannot have: “ReplaceNewVal”
JSONEXTRACT
API request body cannot have: “ReplaceNewVal”. API request body must have: “DestinationField”:(string) – need to be one of: “category”, “className”, “methodName”, “severity”, “threadId”
REPLACE
API request body must have: “ReplaceNewVal”
REMOVEFIELDS
API request body must have: “rule”:(string) – comma-separated list of json fields you would like to remove
TIMESTAMPEXTRACT
API request body cannot have: “rule” or “destinationField” fields. API request body must have:”timeFormat”(string) – you can see our suggestion for each type of standard, normally you will need to change the format to match your time field string pattern with the while comply to the standard “formatStandard”:(string) = need to be one of: “javasdf”, “golang”, “strftime”, “secondsts”, “millits”, “microts”, “nanots”). API request body should contain a meaningful value for “sourceField”: (string) – a value is not specified the default source files will be the text field and the rule will not work.You should set the value as Text.<json_key_with_time_field>)
STRINGIFY JSON FIELD
Convert JSON object to JSON string. Find out more here.
PARSE JSON FIELD
Convert JSON string to JSON object. Find out more here.
Oh no!We didn’t find anything for “”,
