# Background Queries

v1 is being sunsetted

Background Queries v1 is being sunsetted in favor of [Background Queries v2](https://coralogix.com/docs/user-guides/dataengine/background_queries_v2/index.md). v2 is currently in limited availability. To request access, contact [Support](mailto:support@coralogix.com).

Background Queries let you run long-running queries asynchronously using DataPrime or Lucene syntax. They are ideal for recurring, extensive analytical tasks (such as monthly or quarterly reports) and run in the background while you continue active monitoring within the Coralogix platform.

Use Background Queries to:

- Query logs and spans regardless of priority or time frame.
- Take advantage of larger scan and byte-size limits for in-depth data analysis.
- Lower costs by storing data in [Monitoring](https://coralogix.com/docs/user-guides/account-management/tco-optimizer/#features-by-priority-level) and [Compliance](https://coralogix.com/docs/user-guides/account-management/tco-optimizer/#features-by-priority-level) priority tiers, while retaining query capabilities for exploration.

## Build a query

**Step 1**. In your Coralogix toolbar, navigate to **Data Flow**, then **Background Queries v1**.

**Step 2**. Select **Background Query**.

**Step 3**. Define a new query:

- **Name**. The query name must consist only of lowercase letters, with optional numbers and underscores. It cannot begin with a number or an underscore.
- **Description**. [optional] Enter a description of your query for internal purposes.
- **Query**. Enter a [DataPrime](https://coralogix.com/docs/dataprime/introduction/welcome-to-the-dataprime-reference/index.md) query (**recommended**) or a [Lucene](https://coralogix.com/docs/user-guides/data-query/log-query/log-query-simply-retrieve-your-data/#lucene-query-syntax-reference) query. DataPrime allows you to query both logs and spans simultaneously, in addition to grouping, counting, aggregating, and making other powerful transformations to your data.
- **Filters.**
  - Select the time range for your query.
  - [Optional] Filter by application, subsystem, and/or severity. By default, all applications, subsystems, and severities are queried unless otherwise specified.

**Step 4**. Select **Run Background Query**. Once you have set up and run your query, a test runs to validate your setup.

## Save query results as a dataset

Turn any Background Query into a durable, queryable dataset you can reuse in [Explore](https://coralogix.com/docs/user-guides/monitoring-and-insights/logs-screen/logs-screen/index.md), [Custom Dashboards](https://coralogix.com/docs/user-guides/custom-dashboards/introduction/index.md), and API requests, without rerunning heavy scans or exporting data.

When you enable **Save results to Dataset** on a Background Query, Coralogix writes the query output to a persistent system dataset named `system/engine.resultsets.<QUERY_ID>`. The dataset is then available everywhere you work ([Explore](https://coralogix.com/docs/user-guides/monitoring-and-insights/logs-screen/logs-screen/index.md), [Custom Dashboards](https://coralogix.com/docs/user-guides/custom-dashboards/introduction/index.md), and APIs), so you can reuse results, join with other sources, visualize, and automate workflows over time.

### Create and save a Background Query as a dataset

1. Go to **Data Flow**, then **Background Queries v1**, and select **New Background Query**. Author your query in [DataPrime](https://coralogix.com/docs/dataprime/introduction/welcome-to-the-dataprime-reference/index.md) (recommended) or Lucene.
1. In **Query Details**, turn on **Save results to a Dataset**.
1. Select **Run Background Query**.
1. On completion:
   - The card shows **Dataset:** `system/engine.resultsets.<QUERY_ID>`.
   - Select **Go to Explore** to open [Explore](https://coralogix.com/docs/user-guides/monitoring-and-insights/logs-screen/logs-screen/index.md) preloaded with `source system/engine.resultsets.<QUERY_ID>`.

### Query the saved dataset

From [Explore](https://coralogix.com/docs/user-guides/monitoring-and-insights/logs-screen/logs-screen/index.md), [Custom Dashboards](https://coralogix.com/docs/user-guides/custom-dashboards/introduction/index.md), or elsewhere, query any dataset with:

```js
source <dataspace>/<dataset>
```

Saved Background Query outputs are stored as system datasets under `system/engine.resultsets.*` and queried like any other dataset. For example:

```js
source system/engine.resultsets.FCfvKq0bMsZrZgs2hLtQS
```

Use `source system/engine.resultsets.<QUERY_ID>` anywhere you can run DataPrime.

Note

Datasets currently operate on archived data. Use [`join`](https://coralogix.com/docs/dataprime/language-reference/commands-reference/join/index.md) to combine datasets with recent windows. See the [DataPrime cookbook](https://coralogix.com/docs/dataprime/index.md) for concise, copy-pasteable recipes designed to help you build queries faster.

### Where result datasets appear and how to manage them

Result datasets appear under **Data Flow**, then **Dataset Management**, then **System Datasets**, and are treated as standard system datasets.

Toggle **Write Access** (ingest) and **Read Access** (visibility) per dataset. Changes take effect immediately and persist; disabling read makes the dataset invisible without deleting stored data.

### Usage and billing

System datasets consume units from your daily quota, but `system/engine.resultsets.*` does not. This may be subject to change.

## Manage existing queries

To view and manage your existing queries, navigate to **Data Flow**, then **Background Queries v1**. In the Background Query Management screen, you can view query status and results or take other actions.

### View query status

Your query appears in the management screen with the status **Querying** as it is being executed.

### View query results

Once the query is executed, you can view your query results in one of two formats: **Logs Preview** or **Download TSV**.

**Logs Preview** lets you view your logs without ever indexing your data.

**Download TSV** lets you download a TSV file of your query results.

### Clone your query

Duplicate your current query by selecting **Clone**. In the new duplicated query, select **Run Background Query**.

## Limitations

See the [limitations](https://coralogix.com/docs/dataprime/API/limitations/#background-query-limitations) page for a comprehensive list of limitations on Background Queries usage.

## API

Coralogix offers a [gRPC-style](https://coralogix.com/docs/dataprime/API/direct-archive-query-grpc/index.md) and [HTTP-style](https://coralogix.com/docs/dataprime/API/direct-archive-query-http/index.md) Background Queries API.

## Permissions

To use Background Queries, you need the following permission:

| Preset       | Action                           | Description                 |
| ------------ | -------------------------------- | --------------------------- |
| DataQuerying | `LEGACY-ARCHIVE-QUERIES:EXECUTE` | Query data from the archive |

To manage the resulting system datasets (toggling **Write Access** for ingest and **Read Access** for visibility under **Data Flow**, then **Dataset Management**), you need `Manage Datasets` or an equivalent dataset-management permission.

Learn more about [Coralogix roles and permissions](https://coralogix.com/docs/user-guides/aaa/access-control/permissions/index.md).