Using DataPrime to troubleshoot common query issues
Common issues and fixes
1. “My query is running too slowly”
If your query feels sluggish, consider the following optimizations:
1.1 Prefer exact key comparisons over fuzzy matching
Use direct comparisons like:
filter my.key == 'hello'
instead of:
filter my.key.contains('hello')
Exact matches are significantly faster because they can take advantage of indexing. Use fuzzy logic only when necessary.
1.2 Narrow your timeframe
Querying large time ranges (weeks or months) increases scan volume and slows results. Reduce your time window when troubleshooting:
- Use the time picker to select only what’s necessary.
- Use relative time queries where applicable.
1.3 Avoid expensive transformations during querying
DataPrime supports dynamic transformations, but these come at a performance cost. Move complex formatting or parsing into Parsing Rules during ingestion to reduce query-time processing.
1.4 Filter early
Always reduce your working set before transforming or aggregating. For example:
source logs
| filter name == 'Chris'
| groupby path aggregate count()
Filtering first minimizes the data the engine needs to process downstream.
2. “Scan limit exceeded”
You’ve hit the scan cap for your query. This is a guardrail to prevent excessive resource usage.
Fixes:
- Switch to “All Logs Mode” or “All Traces Mode”: These modes query your full archive (e.g., S3) where scan limits are much higher.
- Simplify your query: Remove unnecessary transformations, filters, or nested expressions that force full scans.
For full context, see: Fair Usage Limits.
3. “The AI Query got my query wrong”
Coralogix AI Query is powered by a large language model and AI can be wrong sometimes. Try rewording your prompt.
Fix:
Refine your prompt by:
- Specifying the exact keypaths
- Being concise
- Using domain-specific language when helpful
4. “I get a deprecated warning when I use a command”
DataPrime evolves constantly. Deprecated functions are still supported, but should be updated.
Fix:
- Visit the Functions Reference or Commands Reference.
- Search for the deprecated command and use the suggested replacement.
This ensures your queries stay future-proof.
Expected output
You should see:
- Noticeable speed improvements in query results after filtering or narrowing time.
- Errors related to “Scan limit exceeded” disappear in archive modes.
- AI-generated queries that are more accurate with better prompts.
- Warnings resolved after replacing deprecated syntax.
Common pitfalls
- Using
contains()or regex matching as a default. These are slower than direct comparisons. - Running queries over large time ranges unnecessarily.
- Assuming AI queries will always “just work”, be specific.
- Ignoring deprecation warnings, which can eventually cause query failures.