query_service v1

Scroll down for example requests and responses.

Authentication

API Key (apiKey)
- Parameter Name: Authorization, in: header.

Default

com.coralogixapis.dataprime.v1.DataprimeQueryService.Query

Code samples

POST /api/v1/dataprime/query

method to run dataprime text queries

Body parameter

{
  "query": "string",
  "metadata": {
    "tier": "TIER_UNSPECIFIED",
    "syntax": "QUERY_SYNTAX_UNSPECIFIED",
    "limit": 0,
    "startDate": "2019-08-24T14:15:22Z",
    "endDate": "2019-08-24T14:15:22Z",
    "defaultSource": "string",
    "strictFieldsValidation": true,
    "nowDate": "2019-08-24T14:15:22Z"
  },
  "executionProfile": {
    "preset": "EXECUTION_PROFILE_PRESET_UNSPECIFIED"
  }
}

Parameters

Name	In	Type	Required	Description
body	body	com.coralogixapis.dataprime.v1.QueryRequest	true	dataprime text query request

Example responses
200 Response

Responses

Status	Meaning	Description	Schema
200	OK	none	com.coralogixapis.dataprime.v1.QueryResponse

com.coralogixapis.dataprime.v1.DataprimeQueryService.SubmitBackgroundQuery

Code samples

POST /api/v1/dataprime/background-query

Body parameter

{
  "query": "string",
  "syntax": "QUERY_SYNTAX_UNSPECIFIED",
  "name": "string",
  "description": "string",
  "startDate": "2019-08-24T14:15:22Z",
  "endDate": "2019-08-24T14:15:22Z",
  "nowDate": "2019-08-24T14:15:22Z",
  "resultsDestination": {
    "temporaryDestination": {},
    "persistentDestination": {
      "datasets": [
        {
          "dataspace": "string",
          "dataset": "string",
          "writeMode": "DATASET_WRITE_MODE_UNSPECIFIED"
        }
      ],
      "partitioning": {
        "unpartitioned": {},
        "dateHourPartitioned": {
          "dataprimePath": "string"
        }
      }
    }
  },
  "defaultSource": "string"
}

Parameters

Name	In	Type	Required	Description
body	body	com.coralogixapis.dataprime.v1.SubmitBackgroundQueryRequest	true	none

Example responses
200 Response

Responses

Status	Meaning	Description	Schema
200	OK	none	com.coralogixapis.dataprime.v1.SubmitBackgroundQueryResponse

com.coralogixapis.dataprime.v1.DataprimeQueryService.GetBackgroundQueryStatus

Code samples

POST /api/v1/dataprime/background-query/status

Body parameter

{
  "queryId": "string"
}

Parameters

Name	In	Type	Required	Description
body	body	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusRequest	true	none

Example responses
200 Response

Responses

Status	Meaning	Description	Schema
200	OK	none	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse

com.coralogixapis.dataprime.v1.DataprimeQueryService.GetBackgroundQueryData

Code samples

POST /api/v1/dataprime/background-query/data

Body parameter

{
  "queryId": "string"
}

Parameters

Name	In	Type	Required	Description
body	body	com.coralogixapis.dataprime.v1.GetBackgroundQueryDataRequest	true	none

Example responses
200 Response

Responses

Status	Meaning	Description	Schema
200	OK	none	com.coralogixapis.dataprime.v1.GetBackgroundQueryDataResponse

com.coralogixapis.dataprime.v1.DataprimeQueryService.CancelBackgroundQuery

Code samples

POST /api/v1/dataprime/background-query/cancel

Body parameter

{
  "queryId": "string"
}

Parameters

Name	In	Type	Required	Description
body	body	com.coralogixapis.dataprime.v1.CancelBackgroundQueryRequest	true	none

Example responses
200 Response

Responses

Status	Meaning	Description	Schema
200	OK	none	com.coralogixapis.dataprime.v1.CancelBackgroundQueryResponse

Schemas

com.coralogixapis.dataprime.v1.QueryRequest

{
  "query": "string",
  "metadata": {
    "tier": "TIER_UNSPECIFIED",
    "syntax": "QUERY_SYNTAX_UNSPECIFIED",
    "limit": 0,
    "startDate": "2019-08-24T14:15:22Z",
    "endDate": "2019-08-24T14:15:22Z",
    "defaultSource": "string",
    "strictFieldsValidation": true,
    "nowDate": "2019-08-24T14:15:22Z"
  },
  "executionProfile": {
    "preset": "EXECUTION_PROFILE_PRESET_UNSPECIFIED"
  }
}

dataprime text query request

Properties

Name	Type	Required	Restrictions	Description
query	string	true	none	query for which you seek results
metadata	com.coralogixapis.dataprime.v1.Metadata	false	none	configuration of query execution
executionProfile	com.coralogixapis.dataprime.v1.ExecutionProfile	false	none	configuration that controls query execution parameters of the query engine

com.coralogixapis.dataprime.v1.Metadata

{
  "tier": "TIER_UNSPECIFIED",
  "syntax": "QUERY_SYNTAX_UNSPECIFIED",
  "limit": 0,
  "startDate": "2019-08-24T14:15:22Z",
  "endDate": "2019-08-24T14:15:22Z",
  "defaultSource": "string",
  "strictFieldsValidation": true,
  "nowDate": "2019-08-24T14:15:22Z"
}

configuration of query execution

Properties

Name	Type	Required	Restrictions	Description
tier	com.coralogixapis.dataprime.v1.Metadata.Tier	false	none	tier on which query runs, default: TIER_FREQUENT_SEARCH
syntax	com.coralogixapis.dataprime.v1.QuerySyntax	false	none	syntax of the query, default: QUERY_SYNTAX_DATAPRIME
limit	integer(int32)	false	none	limit number of results, default: 2000; max for TIER_FREQUENT_SEARCH: 12000; max for TIER_ARCHIVE: 50000
startDate	string(date-time)	false	none	beginning of the time range for the query, default: end - 15 min or current time - 15 min if end is not defined, example 2023-11-05T13:15:30Z
endDate	string(date-time)	false	none	end of the time range for the query, default: start + 15 min or current time if start is not defined, example 2023-11-05T13:15:30Z
defaultSource	string	false	none	default value for source to be used when source is omitted in a query
strictFieldsValidation	boolean	false	none	prohibit using unknown fields, ones which were not detected in the ingested data, default = false
nowDate	string(date-time)	false	none	contextual `now` for the query, default: current time, example 2023-11-05T13:15:30Z

com.coralogixapis.dataprime.v1.QuerySyntax

"QUERY_SYNTAX_UNSPECIFIED"

syntax of the query, default: QUERY_SYNTAX_DATAPRIME

Properties

Name	Type	Required	Restrictions	Description
anonymous	string	false	none	syntax of the query, default: QUERY_SYNTAX_DATAPRIME

Enumerated Values

Property	Value
anonymous	QUERY_SYNTAX_UNSPECIFIED
anonymous	QUERY_SYNTAX_LUCENE
anonymous	QUERY_SYNTAX_DATAPRIME
anonymous	QUERY_SYNTAX_LUCENE_UTF8_BASE64
anonymous	QUERY_SYNTAX_DATAPRIME_UTF8_BASE64

com.coralogixapis.dataprime.v1.ExecutionProfile

{
  "preset": "EXECUTION_PROFILE_PRESET_UNSPECIFIED"
}

configuration that controls query execution parameters of the query engine

Properties

Name	Type	Required	Restrictions	Description
preset	com.coralogixapis.dataprime.v1.ExecutionProfile.ExecutionProfilePreset	false	none	null

com.coralogixapis.dataprime.v1.QueryResponse

{
  "error": {
    "message": "string",
    "code": {
      "rateLimitReached": {},
      "resourceExhausted": {}
    }
  },
  "result": {
    "results": [
      {
        "metadata": [
          {
            "key": "string",
            "value": "string"
          }
        ],
        "labels": [
          {
            "key": "string",
            "value": "string"
          }
        ],
        "userData": "string"
      }
    ]
  },
  "warning": {
    "compileWarning": {
      "warningMessage": "string"
    },
    "timeRangeWarning": {
      "warningMessage": "string",
      "startDate": "2019-08-24T14:15:22Z",
      "endDate": "2019-08-24T14:15:22Z"
    },
    "numberOfResultsLimitWarning": {
      "numberOfResultsLimit": 0
    },
    "bytesScannedLimitWarning": {},
    "deprecationWarning": {
      "warningMessage": "string"
    },
    "blocksLimitWarning": {},
    "aggregationBucketsLimitWarning": {
      "aggregationBucketsLimit": 0
    },
    "archiveWarning": {
      "noMetastoreData": {},
      "bucketAccessDenied": {},
      "bucketReadFailed": {},
      "missingData": {}
    },
    "scrollTimeoutWarning": {},
    "fieldCountLimitWarning": {},
    "shuffleFileSizeLimitReachedWarning": {},
    "filesReadLimitWarning": {},
    "sidebarFilterCardinalityLimitWarning": {
      "fields": [
        "string"
      ],
      "cardinalityLimit": "string"
    }
  },
  "statistics": {
    "status": "string",
    "storage": {
      "objectStore": {
        "limits": {
          "scan": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "column": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "shuffleSize": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "filesRead": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          }
        },
        "bytesRead": "string",
        "crossRegionBytesRead": "string"
      },
      "openSearch": {
        "limits": {
          "scan": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "scrollTimeout": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "aggBuckets": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          }
        }
      }
    },
    "e2eDurationMs": "string",
    "outputRowCount": "string"
  },
  "queryId": {
    "queryId": "string"
  }
}

dataprime response for text query, at most one of the fields can be defined in this object

Properties

Name	Type	Required	Restrictions	Description
error	com.coralogixapis.dataprime.v1.DataprimeError	false	none	wrapper for dataprime error
result	com.coralogixapis.dataprime.v1.DataprimeResult	false	none	batch of results
warning	com.coralogixapis.dataprime.v1.DataprimeWarning	false	none	warning message, at most one of the fields can be defined in this object
statistics	com.coralogixapis.dataprime.v1.QueryStatistics	false	none	none
queryId	com.coralogixapis.dataprime.v1.QueryId	false	none	internal identifier of the query. Can be used to simplify investigation of issues

com.coralogixapis.dataprime.v1.DataprimeWarning

{
  "compileWarning": {
    "warningMessage": "string"
  },
  "timeRangeWarning": {
    "warningMessage": "string",
    "startDate": "2019-08-24T14:15:22Z",
    "endDate": "2019-08-24T14:15:22Z"
  },
  "numberOfResultsLimitWarning": {
    "numberOfResultsLimit": 0
  },
  "bytesScannedLimitWarning": {},
  "deprecationWarning": {
    "warningMessage": "string"
  },
  "blocksLimitWarning": {},
  "aggregationBucketsLimitWarning": {
    "aggregationBucketsLimit": 0
  },
  "archiveWarning": {
    "noMetastoreData": {},
    "bucketAccessDenied": {},
    "bucketReadFailed": {},
    "missingData": {}
  },
  "scrollTimeoutWarning": {},
  "fieldCountLimitWarning": {},
  "shuffleFileSizeLimitReachedWarning": {},
  "filesReadLimitWarning": {},
  "sidebarFilterCardinalityLimitWarning": {
    "fields": [
      "string"
    ],
    "cardinalityLimit": "string"
  }
}

warning message, at most one of the fields can be defined in this object

Properties

Name	Type	Required	Restrictions	Description
compileWarning	com.coralogixapis.dataprime.v1.CompileWarning	false	none	warning from Dataprime compilation
timeRangeWarning	com.coralogixapis.dataprime.v1.TimeRangeWarning	false	none	warning from applying time range to query
numberOfResultsLimitWarning	com.coralogixapis.dataprime.v1.NumberOfResultsLimitWarning	false	none	warning from applying limit on number of results
bytesScannedLimitWarning	com.coralogixapis.dataprime.v1.BytesScannedLimitWarning	false	none	warning for reaching bytes scanned limit
deprecationWarning	com.coralogixapis.dataprime.v1.DeprecationWarning	false	none	warning about deprecated elements
blocksLimitWarning	com.coralogixapis.dataprime.v1.BlocksLimitWarning	false	none	Warning for when query has reached maximum number of parquet blocks
aggregationBucketsLimitWarning	com.coralogixapis.dataprime.v1.AggregationBucketsLimitWarning	false	none	warning for reaching aggregation buckets limit
archiveWarning	com.coralogixapis.dataprime.v1.ArchiveWarning	false	none	wrapper for archive related warnings, at most one of the fields can be defined in this object
scrollTimeoutWarning	com.coralogixapis.dataprime.v1.ScrollTimeoutWarning	false	none	warning for when OpenSearch scroll timeout is reached
fieldCountLimitWarning	com.coralogixapis.dataprime.v1.FieldCountLimitWarning	false	none	warning for when result contain entries where number of fields is truncated
shuffleFileSizeLimitReachedWarning	com.coralogixapis.dataprime.v1.ShuffleFileSizeLimitReachedWarning	false	none	warning for when shuffle file size limit is reached - e.g. during a join with a large right side
filesReadLimitWarning	com.coralogixapis.dataprime.v1.FilesReadLimitWarning	false	none	warning for when query has reached maximum number of parquet files
sidebarFilterCardinalityLimitWarning	com.coralogixapis.dataprime.v1.SidebarFilterCardinalityLimitWarning	false	none	warning for when a field has reached the cardinality limit

com.coralogixapis.dataprime.v1.CompileWarning

{
  "warningMessage": "string"
}

warning from Dataprime compilation

Properties

Name	Type	Required	Restrictions	Description
warningMessage	string	true	none	none

com.coralogixapis.dataprime.v1.TimeRangeWarning

{
  "warningMessage": "string",
  "startDate": "2019-08-24T14:15:22Z",
  "endDate": "2019-08-24T14:15:22Z"
}

warning from applying time range to query

Properties

Name	Type	Required	Restrictions	Description
warningMessage	string	true	none	none
startDate	string(date-time)	false	none	example 2023-11-05T13:15:30Z
endDate	string(date-time)	false	none	example 2023-11-05T13:15:30Z

com.coralogixapis.dataprime.v1.NumberOfResultsLimitWarning

{
  "numberOfResultsLimit": 0
}

warning from applying limit on number of results

Properties

Name	Type	Required	Restrictions	Description
numberOfResultsLimit	integer(int32)	true	none	none

com.coralogixapis.dataprime.v1.BytesScannedLimitWarning

{}

warning for reaching bytes scanned limit

Properties

None

com.coralogixapis.dataprime.v1.AggregationBucketsLimitWarning

{
  "aggregationBucketsLimit": 0
}

warning for reaching aggregation buckets limit

Properties

Name	Type	Required	Restrictions	Description
aggregationBucketsLimit	integer(int32)	true	none	none

com.coralogixapis.dataprime.v1.DeprecationWarning

{
  "warningMessage": "string"
}

warning about deprecated elements

Properties

Name	Type	Required	Restrictions	Description
warningMessage	string	true	none	none

com.coralogixapis.dataprime.v1.BlocksLimitWarning

{}

Warning for when query has reached maximum number of parquet blocks

Properties

None

com.coralogixapis.dataprime.v1.FilesReadLimitWarning

{}

warning for when query has reached maximum number of parquet files

Properties

None

com.coralogixapis.dataprime.v1.SidebarFilterCardinalityLimitWarning

{
  "fields": [
    "string"
  ],
  "cardinalityLimit": "string"
}

warning for when a field has reached the cardinality limit

Properties

Name	Type	Required	Restrictions	Description
fields	[string]	true	none	none
cardinalityLimit	string(int64)	true	none	none

com.coralogixapis.dataprime.v1.DataprimeResult

{
  "results": [
    {
      "metadata": [
        {
          "key": "string",
          "value": "string"
        }
      ],
      "labels": [
        {
          "key": "string",
          "value": "string"
        }
      ],
      "userData": "string"
    }
  ]
}

batch of results

Properties

Name	Type	Required	Restrictions	Description
results	[com.coralogixapis.dataprime.v1.DataprimeResults]	true	none	[wrapper for dataprime results]

com.coralogixapis.dataprime.v1.DataprimeResults

{
  "metadata": [
    {
      "key": "string",
      "value": "string"
    }
  ],
  "labels": [
    {
      "key": "string",
      "value": "string"
    }
  ],
  "userData": "string"
}

wrapper for dataprime results

Properties

Name	Type	Required	Restrictions	Description
metadata	[com.coralogixapis.dataprime.v1.DataprimeResults.KeyValue]	true	none	none
labels	[com.coralogixapis.dataprime.v1.DataprimeResults.KeyValue]	true	none	none
userData	string	true	none	none

com.coralogixapis.dataprime.v1.DataprimeError

{
  "message": "string",
  "code": {
    "rateLimitReached": {},
    "resourceExhausted": {}
  }
}

wrapper for dataprime error

Properties

Name	Type	Required	Restrictions	Description
message	string	true	none	none
code	com.coralogixapis.dataprime.v1.DataprimeError.Code	false	none	at most one of the fields can be defined in this object

com.coralogixapis.dataprime.v1.QueryId

{
  "queryId": "string"
}

internal identifier of the query. Can be used to simplify investigation of issues

Properties

Name	Type	Required	Restrictions	Description
queryId	string	true	none	none

com.coralogixapis.dataprime.v1.ArchiveWarning

{
  "noMetastoreData": {},
  "bucketAccessDenied": {},
  "bucketReadFailed": {},
  "missingData": {}
}

wrapper for archive related warnings, at most one of the fields can be defined in this object

Properties

Name	Type	Required	Restrictions	Description
noMetastoreData	com.coralogixapis.dataprime.v1.ArchiveWarning.NoMetastoreData	false	none	none
bucketAccessDenied	com.coralogixapis.dataprime.v1.ArchiveWarning.BucketAccessDenied	false	none	none
bucketReadFailed	com.coralogixapis.dataprime.v1.ArchiveWarning.BucketReadFailed	false	none	none
missingData	com.coralogixapis.dataprime.v1.ArchiveWarning.MissingData	false	none	none

com.coralogixapis.dataprime.v1.ScrollTimeoutWarning

{}

warning for when OpenSearch scroll timeout is reached

Properties

None

com.coralogixapis.dataprime.v1.FieldCountLimitWarning

{}

warning for when result contain entries where number of fields is truncated

Properties

None

com.coralogixapis.dataprime.v1.ShuffleFileSizeLimitReachedWarning

{}

warning for when shuffle file size limit is reached - e.g. during a join with a large right side

Properties

None

com.coralogixapis.dataprime.v1.ResponseMetadata

{
  "statistics": {
    "bytesScanned": "string"
  }
}

this is only used in BG queries but will be also used as part of QueryResponse

Properties

Name	Type	Required	Restrictions	Description
statistics	com.coralogixapis.dataprime.v1.ResponseMetadata.Statistics	false	none	none

com.coralogixapis.dataprime.v1.QueryStatistics

{
  "status": "string",
  "storage": {
    "objectStore": {
      "limits": {
        "scan": {
          "reached": true,
          "limit": "string",
          "measuredValue": "string"
        },
        "column": {
          "reached": true,
          "limit": "string",
          "measuredValue": "string"
        },
        "shuffleSize": {
          "reached": true,
          "limit": "string",
          "measuredValue": "string"
        },
        "filesRead": {
          "reached": true,
          "limit": "string",
          "measuredValue": "string"
        }
      },
      "bytesRead": "string",
      "crossRegionBytesRead": "string"
    },
    "openSearch": {
      "limits": {
        "scan": {
          "reached": true,
          "limit": "string",
          "measuredValue": "string"
        },
        "scrollTimeout": {
          "reached": true,
          "limit": "string",
          "measuredValue": "string"
        },
        "aggBuckets": {
          "reached": true,
          "limit": "string",
          "measuredValue": "string"
        }
      }
    }
  },
  "e2eDurationMs": "string",
  "outputRowCount": "string"
}

Properties

Name	Type	Required	Restrictions	Description
status	string	true	none	none
storage	com.coralogixapis.dataprime.v1.StorageStatistics	false	none	at most one of the fields can be defined in this object
e2eDurationMs	string(int64)	true	none	none
outputRowCount	string(int64)	false	none	none

com.coralogixapis.dataprime.v1.StorageStatistics

{
  "objectStore": {
    "limits": {
      "scan": {
        "reached": true,
        "limit": "string",
        "measuredValue": "string"
      },
      "column": {
        "reached": true,
        "limit": "string",
        "measuredValue": "string"
      },
      "shuffleSize": {
        "reached": true,
        "limit": "string",
        "measuredValue": "string"
      },
      "filesRead": {
        "reached": true,
        "limit": "string",
        "measuredValue": "string"
      }
    },
    "bytesRead": "string",
    "crossRegionBytesRead": "string"
  },
  "openSearch": {
    "limits": {
      "scan": {
        "reached": true,
        "limit": "string",
        "measuredValue": "string"
      },
      "scrollTimeout": {
        "reached": true,
        "limit": "string",
        "measuredValue": "string"
      },
      "aggBuckets": {
        "reached": true,
        "limit": "string",
        "measuredValue": "string"
      }
    }
  }
}

at most one of the fields can be defined in this object

Properties

Name	Type	Required	Restrictions	Description
objectStore	com.coralogixapis.dataprime.v1.ObjectStoreStatistics	false	none	none
openSearch	com.coralogixapis.dataprime.v1.OpenSearchStatistics	false	none	none

com.coralogixapis.dataprime.v1.ObjectStoreStatistics

{
  "limits": {
    "scan": {
      "reached": true,
      "limit": "string",
      "measuredValue": "string"
    },
    "column": {
      "reached": true,
      "limit": "string",
      "measuredValue": "string"
    },
    "shuffleSize": {
      "reached": true,
      "limit": "string",
      "measuredValue": "string"
    },
    "filesRead": {
      "reached": true,
      "limit": "string",
      "measuredValue": "string"
    }
  },
  "bytesRead": "string",
  "crossRegionBytesRead": "string"
}

Properties

Name	Type	Required	Restrictions	Description
limits	com.coralogixapis.dataprime.v1.ObjectStoreLimits	true	none	none
bytesRead	string(int64)	true	none	bytes that DQE reads from customer-exposed storage (customers bucket or high-tier storage, not including e.g. cache or staging bucket)
crossRegionBytesRead	string(int64)	true	none	bytes read from a different region than the DQE cluster's region

com.coralogixapis.dataprime.v1.ObjectStoreLimits

{
  "scan": {
    "reached": true,
    "limit": "string",
    "measuredValue": "string"
  },
  "column": {
    "reached": true,
    "limit": "string",
    "measuredValue": "string"
  },
  "shuffleSize": {
    "reached": true,
    "limit": "string",
    "measuredValue": "string"
  },
  "filesRead": {
    "reached": true,
    "limit": "string",
    "measuredValue": "string"
  }
}

Properties

Name	Type	Required	Restrictions	Description
scan	com.coralogixapis.dataprime.v1.LimitStatus	true	none	none
column	com.coralogixapis.dataprime.v1.LimitStatus	true	none	none
shuffleSize	com.coralogixapis.dataprime.v1.LimitStatus	true	none	none
filesRead	com.coralogixapis.dataprime.v1.LimitStatus	true	none	none

com.coralogixapis.dataprime.v1.OpenSearchStatistics

{
  "limits": {
    "scan": {
      "reached": true,
      "limit": "string",
      "measuredValue": "string"
    },
    "scrollTimeout": {
      "reached": true,
      "limit": "string",
      "measuredValue": "string"
    },
    "aggBuckets": {
      "reached": true,
      "limit": "string",
      "measuredValue": "string"
    }
  }
}

Properties

Name	Type	Required	Restrictions	Description
limits	com.coralogixapis.dataprime.v1.OpenSearchLimits	true	none	none

com.coralogixapis.dataprime.v1.OpenSearchLimits

{
  "scan": {
    "reached": true,
    "limit": "string",
    "measuredValue": "string"
  },
  "scrollTimeout": {
    "reached": true,
    "limit": "string",
    "measuredValue": "string"
  },
  "aggBuckets": {
    "reached": true,
    "limit": "string",
    "measuredValue": "string"
  }
}

Properties

Name	Type	Required	Restrictions	Description
scan	com.coralogixapis.dataprime.v1.LimitStatus	true	none	none
scrollTimeout	com.coralogixapis.dataprime.v1.LimitStatus	true	none	none
aggBuckets	com.coralogixapis.dataprime.v1.LimitStatus	true	none	none

com.coralogixapis.dataprime.v1.LimitStatus

{
  "reached": true,
  "limit": "string",
  "measuredValue": "string"
}

Properties

Name	Type	Required	Restrictions	Description
reached	boolean	true	none	none
limit	string(int64)	false	none	threshold from execution config
measuredValue	string(int64)	false	none	measured value from DQE metrics

com.coralogixapis.dataprime.v1.SubmitBackgroundQueryRequest

{
  "query": "string",
  "syntax": "QUERY_SYNTAX_UNSPECIFIED",
  "name": "string",
  "description": "string",
  "startDate": "2019-08-24T14:15:22Z",
  "endDate": "2019-08-24T14:15:22Z",
  "nowDate": "2019-08-24T14:15:22Z",
  "resultsDestination": {
    "temporaryDestination": {},
    "persistentDestination": {
      "datasets": [
        {
          "dataspace": "string",
          "dataset": "string",
          "writeMode": "DATASET_WRITE_MODE_UNSPECIFIED"
        }
      ],
      "partitioning": {
        "unpartitioned": {},
        "dateHourPartitioned": {
          "dataprimePath": "string"
        }
      }
    }
  },
  "defaultSource": "string"
}

Properties

Name	Type	Required	Restrictions	Description
query	string	true	none	none
syntax	com.coralogixapis.dataprime.v1.QuerySyntax	true	none	syntax of the query, default: QUERY_SYNTAX_DATAPRIME
name	string	false	none	Must start with a letter, and must consist only of letters, numbers, underscores, or dots, up to maximum length of 255 characters. Dots cannot appear consecutively, nor at the end. Regex pattern: `^[a-zA-Z][a-zA-Z0-9_.]{0,254}$`. Defaults to auto-generated name.
description	string	false	none	Must be under 2048 characters. Defaults to empty string.
startDate	string(date-time)	false	none	Beginning of the time-range for the query., example 2023-11-05T13:15:30Z
endDate	string(date-time)	false	none	End of the time-range for the query., example 2023-11-05T13:15:30Z
nowDate	string(date-time)	false	none	Contextual `now` for the query, will be used in place of `now()` invocation. Defaults to current time., example 2023-11-05T13:15:30Z
resultsDestination	com.coralogixapis.dataprime.v1.ResultsDestination	false	none	at most one of the fields can be defined in this object
defaultSource	string	false	none	FQN of the default source when omitted in the query. Defaults to "default/logs".

com.coralogixapis.dataprime.v1.ResultsDestination

{
  "temporaryDestination": {},
  "persistentDestination": {
    "datasets": [
      {
        "dataspace": "string",
        "dataset": "string",
        "writeMode": "DATASET_WRITE_MODE_UNSPECIFIED"
      }
    ],
    "partitioning": {
      "unpartitioned": {},
      "dateHourPartitioned": {
        "dataprimePath": "string"
      }
    }
  }
}

at most one of the fields can be defined in this object

Properties

Name	Type	Required	Restrictions	Description
temporaryDestination	com.coralogixapis.dataprime.v1.ResultsDestination.TemporaryDestination	false	none	Results stored in engine-owned `system/engine.resultsets.bg_queries.{unique_id}` dataset for 30 days.
persistentDestination	com.coralogixapis.dataprime.v1.ResultsDestination.PersistentDestination	false	none	Results written to an user-defined dataset in caller's team

com.coralogixapis.dataprime.v1.DatasetDestination

{
  "dataspace": "string",
  "dataset": "string",
  "writeMode": "DATASET_WRITE_MODE_UNSPECIFIED"
}

Properties

Name	Type	Required	Restrictions	Description
dataspace	string	true	none	Specified dataset must exist - this API will not create a new datasets, and must be under `default` dataspace.
dataset	string	true	none	none
writeMode	com.coralogixapis.dataprime.v1.DatasetDestination.DatasetWriteMode	true	none	null

com.coralogixapis.dataprime.v1.Partitioning

{
  "unpartitioned": {},
  "dateHourPartitioned": {
    "dataprimePath": "string"
  }
}

at most one of the fields can be defined in this object

Properties

Name	Type	Required	Restrictions	Description
unpartitioned	com.coralogixapis.dataprime.v1.Partitioning.Unpartitioned	false	none	none
dateHourPartitioned	com.coralogixapis.dataprime.v1.Partitioning.DateHourPartitioned	false	none	Partition results on values of timestamp type with date/hour granularity.

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusRequest

{
  "queryId": "string"
}

Properties

Name	Type	Required	Restrictions	Description
queryId	string	true	none	none

com.coralogixapis.dataprime.v1.GetBackgroundQueryDataRequest

{
  "queryId": "string"
}

Properties

Name	Type	Required	Restrictions	Description
queryId	string	true	none	none

com.coralogixapis.dataprime.v1.CancelBackgroundQueryRequest

{
  "queryId": "string"
}

Properties

Name	Type	Required	Restrictions	Description
queryId	string	true	none	none

com.coralogixapis.dataprime.v1.SubmitBackgroundQueryResponse

{
  "warnings": [
    {
      "compileWarning": {
        "warningMessage": "string"
      },
      "timeRangeWarning": {
        "warningMessage": "string",
        "startDate": "2019-08-24T14:15:22Z",
        "endDate": "2019-08-24T14:15:22Z"
      },
      "numberOfResultsLimitWarning": {
        "numberOfResultsLimit": 0
      },
      "bytesScannedLimitWarning": {},
      "deprecationWarning": {
        "warningMessage": "string"
      },
      "blocksLimitWarning": {},
      "aggregationBucketsLimitWarning": {
        "aggregationBucketsLimit": 0
      },
      "archiveWarning": {
        "noMetastoreData": {},
        "bucketAccessDenied": {},
        "bucketReadFailed": {},
        "missingData": {}
      },
      "scrollTimeoutWarning": {},
      "fieldCountLimitWarning": {},
      "shuffleFileSizeLimitReachedWarning": {},
      "filesReadLimitWarning": {},
      "sidebarFilterCardinalityLimitWarning": {
        "fields": [
          "string"
        ],
        "cardinalityLimit": "string"
      }
    }
  ],
  "queryId": "string"
}

Properties

Name	Type	Required	Restrictions	Description
warnings	[com.coralogixapis.dataprime.v1.DataprimeWarning]	true	none	all warnings that can apprear during query submission
queryId	string	true	none	none

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse

{
  "running": {
    "runningSince": "2019-08-24T14:15:22Z"
  },
  "terminated": {
    "success": {},
    "error": {
      "failed": {
        "reason": "string"
      },
      "timedOut": {}
    },
    "cancelled": {},
    "statistics": {
      "status": "string",
      "storage": {
        "objectStore": {
          "limits": {
            "scan": {
              "reached": true,
              "limit": "string",
              "measuredValue": "string"
            },
            "column": {
              "reached": true,
              "limit": "string",
              "measuredValue": "string"
            },
            "shuffleSize": {
              "reached": true,
              "limit": "string",
              "measuredValue": "string"
            },
            "filesRead": {
              "reached": true,
              "limit": "string",
              "measuredValue": "string"
            }
          },
          "bytesRead": "string",
          "crossRegionBytesRead": "string"
        },
        "openSearch": {
          "limits": {
            "scan": {
              "reached": true,
              "limit": "string",
              "measuredValue": "string"
            },
            "scrollTimeout": {
              "reached": true,
              "limit": "string",
              "measuredValue": "string"
            },
            "aggBuckets": {
              "reached": true,
              "limit": "string",
              "measuredValue": "string"
            }
          }
        }
      },
      "e2eDurationMs": "string",
      "outputRowCount": "string"
    },
    "runningSince": "2019-08-24T14:15:22Z",
    "terminatedAt": "2019-08-24T14:15:22Z"
  },
  "metadata": [
    {
      "statistics": {
        "bytesScanned": "string"
      }
    }
  ],
  "warnings": [
    {
      "compileWarning": {
        "warningMessage": "string"
      },
      "timeRangeWarning": {
        "warningMessage": "string",
        "startDate": "2019-08-24T14:15:22Z",
        "endDate": "2019-08-24T14:15:22Z"
      },
      "numberOfResultsLimitWarning": {
        "numberOfResultsLimit": 0
      },
      "bytesScannedLimitWarning": {},
      "deprecationWarning": {
        "warningMessage": "string"
      },
      "blocksLimitWarning": {},
      "aggregationBucketsLimitWarning": {
        "aggregationBucketsLimit": 0
      },
      "archiveWarning": {
        "noMetastoreData": {},
        "bucketAccessDenied": {},
        "bucketReadFailed": {},
        "missingData": {}
      },
      "scrollTimeoutWarning": {},
      "fieldCountLimitWarning": {},
      "shuffleFileSizeLimitReachedWarning": {},
      "filesReadLimitWarning": {},
      "sidebarFilterCardinalityLimitWarning": {
        "fields": [
          "string"
        ],
        "cardinalityLimit": "string"
      }
    }
  ],
  "waitingForExecution": {},
  "submittedAt": "2019-08-24T14:15:22Z"
}

at most one of the following fields can be defined in this object: running,terminated,waiting_for_execution

Properties

Name	Type	Required	Restrictions	Description
running	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Running	false	none	none
terminated	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated	false	none	at most one of the following fields can be defined in this object: success,error,cancelled
metadata	[com.coralogixapis.dataprime.v1.ResponseMetadata]	true	none	does not contain duplicate messages
warnings	[com.coralogixapis.dataprime.v1.DataprimeWarning]	true	none	includes all warnings (e.g.: warnings that can apprear during query submission + runtime warnings)
waitingForExecution	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.WaitingForExecution	false	none	none
submittedAt	string(date-time)	true	none	example 2023-11-05T13:15:30Z

com.coralogixapis.dataprime.v1.GetBackgroundQueryDataResponse

{
  "response": {
    "results": {
      "results": [
        {
          "metadata": [
            {
              "key": "string",
              "value": "string"
            }
          ],
          "labels": [
            {
              "key": "string",
              "value": "string"
            }
          ],
          "userData": "string"
        }
      ]
    }
  }
}

using extensible structure for future-proofing

Properties

Name	Type	Required	Restrictions	Description
response	com.coralogixapis.dataprime.v1.GetDataResponse	false	none	none

com.coralogixapis.dataprime.v1.GetDataResponse

{
  "results": {
    "results": [
      {
        "metadata": [
          {
            "key": "string",
            "value": "string"
          }
        ],
        "labels": [
          {
            "key": "string",
            "value": "string"
          }
        ],
        "userData": "string"
      }
    ]
  }
}

Properties

Name	Type	Required	Restrictions	Description
results	com.coralogixapis.dataprime.v1.DataprimeResult	true	none	batch of results

com.coralogixapis.dataprime.v1.CancelBackgroundQueryResponse

{}

Properties

None

com.coralogixapis.dataprime.v1.Metadata.Tier

"TIER_UNSPECIFIED"

tier on which query runs, default: TIER_FREQUENT_SEARCH

Properties

Name	Type	Required	Restrictions	Description
anonymous	string	false	none	tier on which query runs, default: TIER_FREQUENT_SEARCH

Enumerated Values

Property	Value
anonymous	TIER_UNSPECIFIED
anonymous	TIER_ARCHIVE
anonymous	TIER_FREQUENT_SEARCH

com.coralogixapis.dataprime.v1.ExecutionProfile.ExecutionProfilePreset

"EXECUTION_PROFILE_PRESET_UNSPECIFIED"

null

Properties

Name	Type	Required	Restrictions	Description
anonymous	string	false	none	null

Enumerated Values

Property	Value
anonymous	EXECUTION_PROFILE_PRESET_UNSPECIFIED
anonymous	EXECUTION_PROFILE_PRESET_PERFORMANCE
anonymous	EXECUTION_PROFILE_PRESET_ACCURACY

com.coralogixapis.dataprime.v1.DataprimeResults.KeyValue

{
  "key": "string",
  "value": "string"
}

Properties

Name	Type	Required	Restrictions	Description
key	string	true	none	none
value	string	true	none	none

com.coralogixapis.dataprime.v1.DataprimeError.RateLimitReached

{}

Properties

None

com.coralogixapis.dataprime.v1.DataprimeError.ResourceExhausted

{}

Properties

None

com.coralogixapis.dataprime.v1.DataprimeError.Code

{
  "rateLimitReached": {},
  "resourceExhausted": {}
}

at most one of the fields can be defined in this object

Properties

Name	Type	Required	Restrictions	Description
rateLimitReached	com.coralogixapis.dataprime.v1.DataprimeError.RateLimitReached	false	none	none
resourceExhausted	com.coralogixapis.dataprime.v1.DataprimeError.ResourceExhausted	false	none	none

com.coralogixapis.dataprime.v1.ArchiveWarning.NoMetastoreData

{}

Properties

None

com.coralogixapis.dataprime.v1.ArchiveWarning.BucketAccessDenied

{}

Properties

None

com.coralogixapis.dataprime.v1.ArchiveWarning.BucketReadFailed

{}

Properties

None

com.coralogixapis.dataprime.v1.ArchiveWarning.MissingData

{}

Properties

None

com.coralogixapis.dataprime.v1.ResponseMetadata.Statistics

{
  "bytesScanned": "string"
}

Properties

Name	Type	Required	Restrictions	Description
bytesScanned	string(int64)	true	none	none

com.coralogixapis.dataprime.v1.ResultsDestination.TemporaryDestination

{}

Results stored in engine-owned system/engine.resultsets.bg_queries.{unique_id} dataset for 30 days.

Properties

None

com.coralogixapis.dataprime.v1.ResultsDestination.PersistentDestination

{
  "datasets": [
    {
      "dataspace": "string",
      "dataset": "string",
      "writeMode": "DATASET_WRITE_MODE_UNSPECIFIED"
    }
  ],
  "partitioning": {
    "unpartitioned": {},
    "dateHourPartitioned": {
      "dataprimePath": "string"
    }
  }
}

Results written to an user-defined dataset in caller's team

Properties

Name	Type	Required	Restrictions	Description
datasets	[com.coralogixapis.dataprime.v1.DatasetDestination]	true	none	none
partitioning	com.coralogixapis.dataprime.v1.Partitioning	false	none	at most one of the fields can be defined in this object

com.coralogixapis.dataprime.v1.DatasetDestination.DatasetWriteMode

"DATASET_WRITE_MODE_UNSPECIFIED"

null

Properties

Name	Type	Required	Restrictions	Description
anonymous	string	false	none	null

Enumerated Values

Property	Value
anonymous	DATASET_WRITE_MODE_UNSPECIFIED
anonymous	DATASET_WRITE_MODE_APPEND
anonymous	DATASET_WRITE_MODE_OVERWRITE

com.coralogixapis.dataprime.v1.Partitioning.DateHourPartitioned

{
  "dataprimePath": "string"
}

Partition results on values of timestamp type with date/hour granularity.

Properties

Name	Type	Required	Restrictions	Description
dataprimePath	string	true	none	DataPrime keypath of a timestamp to partition on, e.g. `$m.timestamp`. The keypath must exist in the query's results, and be of timestamp type.

com.coralogixapis.dataprime.v1.Partitioning.Unpartitioned

{}

Properties

None

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.WaitingForExecution

{}

Properties

None

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Running

{
  "runningSince": "2019-08-24T14:15:22Z"
}

Properties

Name	Type	Required	Restrictions	Description
runningSince	string(date-time)	true	none	example 2023-11-05T13:15:30Z

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated

{
  "success": {},
  "error": {
    "failed": {
      "reason": "string"
    },
    "timedOut": {}
  },
  "cancelled": {},
  "statistics": {
    "status": "string",
    "storage": {
      "objectStore": {
        "limits": {
          "scan": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "column": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "shuffleSize": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "filesRead": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          }
        },
        "bytesRead": "string",
        "crossRegionBytesRead": "string"
      },
      "openSearch": {
        "limits": {
          "scan": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "scrollTimeout": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          },
          "aggBuckets": {
            "reached": true,
            "limit": "string",
            "measuredValue": "string"
          }
        }
      }
    },
    "e2eDurationMs": "string",
    "outputRowCount": "string"
  },
  "runningSince": "2019-08-24T14:15:22Z",
  "terminatedAt": "2019-08-24T14:15:22Z"
}

at most one of the following fields can be defined in this object: success,error,cancelled

Properties

Name	Type	Required	Restrictions	Description
success	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Success	false	none	none
error	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Error	false	none	at most one of the fields can be defined in this object
cancelled	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Cancelled	false	none	none
statistics	com.coralogixapis.dataprime.v1.QueryStatistics	true	none	none
runningSince	string(date-time)	true	none	example 2023-11-05T13:15:30Z
terminatedAt	string(date-time)	true	none	example 2023-11-05T13:15:30Z

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Success

{}

Properties

None

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Cancelled

{}

Properties

None

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Error

{
  "failed": {
    "reason": "string"
  },
  "timedOut": {}
}

at most one of the fields can be defined in this object

Properties

Name	Type	Required	Restrictions	Description
failed	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Error.Failed	false	none	none
timedOut	com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Error.TimedOut	false	none	none

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Error.TimedOut

{}

Properties

None

com.coralogixapis.dataprime.v1.GetBackgroundQueryStatusResponse.Terminated.Error.Failed

{
  "reason": "string"
}

Properties

Name	Type	Required	Restrictions	Description
reason	string	true	none	none

Need help? Contact Support.

What's new? Find out here.

LLM? Read llms.txt.