Direct Archive Query HTTP API
Use our Direct Archive Query HTTP API to run DataPrime or Lucene queries of your indexed and archived logs without accessing your Coralogix UI.
API Reference
The Direct Archive Query HTTP API is of an HTTP-style method.
Our API has predictable resource-oriented URLs, accepts POST JSON-formatted request bodies, returns Newline Delimited or NDJSON-formatted responses, and uses standard HTTP response codes, authentication, and verbs.
Prerequisites
Requests must be sent using TLS.
Base URL
Use the DataPrime Endpoint that matches your Coralogix domain.
When sending an HTTPS POST, you must present your arguments as JSON.
HTTP Method | POST |
---|---|
Content-Type | JSON |
Authentication
API Key
To use this API you need to create a personal or team API key. It’s recommended to use permission presets, as they are automatically updated with all relevant permissions. Alternatively, you can manually add individual permissions.
Preset | Action | Description |
---|---|---|
DataQuerying | LEGACY-ARCHIVE-QUERIES:EXECUTE LOGS.DATA-API#HIGH:READDATA LOGS.DATA-API#LOW:READDATA METRICS.DATA-API#HIGH:READDATA SPANS.DATA-API#HIGH:READDATA SPANS.DATA-API#LOW:READDATA | Query Data from the Archive Query Frequent Search Logs Query Monitoring & Compliance Logs Query Metrics Query Frequent Search Spans Query Monitoring & Compliance Spans |
All API requests must be made over HTTPS. Calls made over plain HTTP will fail. API requests without authentication will also fail.
Authorization Header
To authenticate, add an Authorization Bearer <cx_api_key>
header to your API request. It contains a Bearer Token, which identifies a single user, bot user, or workspace-application relationship for authentication.
Key | Authorization |
---|---|
Value | Bearer <cx_api_key> |
Requests
Host
Use the DataPrime endpoint that matches your Coralogix domain.
All requests must be made over HTTPS. The API does not support HTTP.
Authorization Header
You must provide an authorization header as described in Authentication.
Formatting Your Request
For complete guidance on how to structure requests and responses, view this swagger.md, a human-readable document derived from a swagger.json file.
We recommend using OpenAPI Editor or a similar technology for support.
Request Body
When submitting data to a resource via POST
, you must submit your payload in JSON.
You are required to include a query
in the API body.
All other fields are optional and form the metadata object.
Body Examples
JSON Object
The following example consists of a JSON object that represents the request
Minimal DataPrime Query
DataPrime with Lucene Operator
{
"query": "source logs | filter log_obj.subsystem_name == 'foo' | lucene 'coralogix.metadata.applicationName:bar'"
}
Lucene Query
{
"query": "coralogix.metadata.applicationName:bar'",
"metadata": {
"syntax": "QUERY_SYNTAX_LUCENE"
}
}
Full DataPrime Query
{
"query": "source logs | limit 100",
"metadata": {
"tier": "TIER_FREQUENT_SEARCH",
"syntax": "QUERY_SYNTAX_DATAPRIME",
"startDate": "2023-05-29T11:20:00.00Z",
"endDate": "2023-05-29T11:30:00.00Z",
"defaultSource": "logs"
}
}
cURL
Basic Usage
curl --location 'https://ng-api-http.<span class="domain-value"></span>/api/v1/dataprime/query' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <cx_api_key>' \
--data '{
"query": "source logs | limit 10"
}'
With Metadata
curl --location 'https://ng-api-http.<span class="domain-value"></span>/api/v1/dataprime/query' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <cx_api_key>' \
--data '{
"query": "source logs | limit 10",
"metadata": {
"tier": "TIER_FREQUENT_SEARCH",
"syntax": "QUERY_SYNTAX_DATAPRIME",
"startDate": "2023-05-29T11:20:00.00Z",
"endDate": "2023-05-29T11:30:00.00Z"
}
}'
Field | Description | Example |
---|---|---|
startDate | The start date and time for the query. |
Assumes Coordinated Universal Time (UTC). If the time zone of your timestamp differs, convert it to UTC or offset the timestamp to your time zone by navigating to Account settings > Preferences > Change Your Time Zone Settings. | For example, to start the query at 11:20 local time:
In San Fransisco (Pacific Daylight Time): 2023-05-29T11:20:00.00-07:00
In India (India Standard Time): 2023-05-29T11:20:00.00+05:30 | | endDate
| The end date and time for the query.
Assumes Coordinated Universal Time (UTC). If the time zone of your timestamp differs, convert it to UTC or offset the timestamp to your time zone by navigating to Account settings > Preferences > Change Your Time Zone Settings. | | | tier
| Specifies the tier of data to query. Options are TIER_FREQUENT_SEARCH
or TIER_ARCHIVE
. | TIER_FREQUENT_SEARCH
| | limit
| The maximum number of results to return. The default is 2000, with a maximum of 12,000. | 2000
| | syntax
| The query syntax to use. Options are QUERY_SYNTAX_DATAPRIME
or QUERY_SYNTAX_LUCENE
. | QUERY_SYNTAX_LUCENE
|
With defaultSource
curl --location 'https://ng-api-http.<span class="domain-value"></span>/api/v1/dataprime/query' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer <cx_api_key>' \
--data '{
"query": "limit 10",
"metadata": {
"tier": "TIER_FREQUENT_SEARCH",
"syntax": "QUERY_SYNTAX_DATAPRIME",
"defaultSource": "logs"
}
}'
Responses
The Content-Type
representation header is used to indicate the original media type of the resource (before any content encoding is applied for sending).
In responses, a Content-Type
header provides the client with the actual content type of the returned content.
Correct results are returned in batches. Each batch may contain multiple rows of results, which are returned as Newline Delimited JSON or ndjson format.
Example Response
The following is a generic example response. Responses may vary per query and user data.
{
"result": {
"results": [
{
"metadata": [
{
"key": "logid",
"value": "c3ca5343-88dc-4807-a8f3-82832274afb7"
}
],
"labels": [
{
"key": "applicationname",
"value": "staging"
}
],
"userData": "{ ... \\"log_obj\\":{\\"level\\":\\"INFO\\",\\"message\\":\\"some log message\\" ... }, ...}"
}
]
}
}
Status Codes
Here are some of the most common status codes to expect.
Status Code | Description |
---|---|
200 | No Error |
400 | Bad Request |
403 | Forbidden |
Failed Requests
The general format guidelines are displayed when the accompanying status code is returned.
Warnings
The Warning
response contains information about possible problems with the status of the message. More than one Warning
header may appear in a response.
Warning
responses can be applied to any message, before or after query results.
Warning Type | Description |
---|---|
CompileWarning | Warning of potential compilation error in your query. In the event of a compilation failure, you will receive an error response. |
TimeRangeWarning | When the time frame for your query has been built incorrectly or exceeds internal limits |
NumberOfResultsLimitWarning | When the number of query results exceeds internal limits |
BytesScannedLimitWarning | When the number of bytes returned in query results exceeds internal limits |
DeprecationWarning | When a value in your query is changed or deprecated incorrectly |
Limitations
Coralogix places certain limitations on query responses. Warnings are returned when a limit is breached.
Results Returned
The number of results returned in rows is limited as follows:
50k for S3 Archive queries
12k for OpenSearch queries
Bytes Scanned Limit
The number of bytes scanned (for high-tier data) is limited as follows:
- 100MB for for OpenSearch queries
This limitation is placed on fetching 100 MB of high-tier data. It does not limit the scanning within the database storage.
Rate Limiting
The number of requests per minute is limited as follows:
- 10 requests per minute
Additional Resources
Documentation | Complete Guidance on Structuring Requests & Responses (Swagger) [DataPrime Syntax./../../../dataprime/introduction/welcome-to-the-dataprime-reference/index.md) Lucene Syntax S3 Archive |
Support
Need help?
Our world-class customer success team is available 24/7 to walk you through your setup and answer any questions that may come up.
Feel free to reach out to us via our in-app chat or by sending us an email at [email protected].