Threat hunting with Olly

Effective threat hunting requires both comprehensive visibility and quick, data-driven insights. Olly, the AI-powered observability teammate within Coralogix, provides just that. Whether you’re tracking lateral movement, uncovering stealthy persistence, or correlating spikes in anomalous activity, Olly rapidly pinpoints the evidence and presents it in context so you can confirm or dismiss threats before they escalate. 

In this post, we’ll show how Olly streamlines every stage of the hunt, from hypothesis to hard proof, turning raw data into actionable security findings that keep your organization a step ahead of attackers.

What is threat hunting?

Threat hunting is the practice of proactively searching for malicious or suspicious activity that may not yet have triggered an alert. It shifts security teams away from purely reacting to detections and toward actively looking for signs of compromise within their environment.

Traditional security operations are largely reactive. Teams respond to alerts after they fire, often relying on known attack patterns or signature-based detections. This approach can miss novel or low-signal attacker activity and frequently produces high alert volumes with limited context. In many cases, investigation begins only after some level of impact has already occurred. For example, responding to a GuardDuty alert for unusual API activity without immediately understanding how that activity fits into the broader environment.

Threat hunting is the practice of proactively searching for malicious or suspicious activity that may not yet have triggered an alert. It shifts security teams away from purely reacting to detections and toward actively looking for signs of compromise within their environment.

Instead of waiting for alerts, analysts actively search for anomalies, indicators of compromise, or behavioral patterns that deviate from normal activity. By correlating data across sources such as CloudTrail, network flow logs, and application logs, hunters can uncover suspicious behavior earlier and with better context.

For example, analyzing CloudTrail activity over time may reveal unauthorized access attempts or abnormal usage patterns before they escalate into a confirmed incident.

Challenges in threat detection

Threat detection in cloud environments is difficult because many risky actions are also legitimate and intent depends heavily on context.

Context-dependent activity
Actions such as data downloads, privilege changes, or API calls may be normal or malicious depending on the user, resource, and timing. For example, an administrator exporting S3 data for reporting may be expected, while the same action by an unexpected principal could indicate exfiltration. Likewise, a privileged user attaching new policies to their own role may signal either routine administration or privilege escalation.

Rapidly changing cloud environments
Autoscaling instances, temporary credentials, and serverless workloads generate large volumes of logs and short-lived entities. Dynamic infrastructures make it harder to establish baselines and identify meaningful signals within noisy data.

Sophisticated multi-stage attacks
Attackers often abuse valid credentials, assume roles, and move laterally in ways that resemble normal operations. These techniques blend into expected behavior, making them difficult to detect without proactive investigation.

Threat hunting framework

Modern threat hunting programs require more than ad-hoc log exploration. A structured framework ensures hunts are repeatable, measurable, and grounded in real adversary behavior. By standardizing how hunts are planned, executed, and evaluated, security teams can consistently generate meaningful outcomes and continuously improve detection coverage.

At a high level, the framework is built around three core principles that guide every hunt—from initial idea to actionable detection.

1. Hypothesis-driven approach

Threat hunts begin with explicit hypotheses about adversary behavior or identity misuse. Rather than searching logs without direction, analysts start with an assumption based on known attacker tradecraft and attempt to validate or disprove it using available data. This keeps investigations intentional, focused, and threat-informed.

2. Detection engineering mapping

Each hypothesis is translated into concrete detection requirements. This includes identifying relevant log sources, event types, query logic, alerting conditions, and known sources of false positives. Over time, this process creates a reusable, evidence-based detection library that strengthens both proactive hunting and reactive alerting.

3. Outcome and coverage validation

Every hunt is evaluated based on its outcome—whether it results in a new detection, improved alert logic, refined baselines, or identified visibility gaps. Mapping hypotheses and results to MITRE ATT&CK tactics and techniques helps assess coverage, prioritize future hunts, and ensure alignment with real adversary behavior.

Threat Hunting Hypothesis Example

The following example demonstrates how this framework is applied in practice, showing how a high-level threat assumption is converted into a structured hypothesis and mapped to actionable detection logic. Below is an example of a threat hunting hypothesis devised for detection of IAM Policy Escalation in AWS using Cloudtrail logs.

Hypothesis ID: H5

Hypothesis Name: IAM Policy Escalation

Hypothesis Summary:
A low-privileged IAM user or role escalates privileges by attaching or creating a highly permissive policy (e.g., AdministratorAccess or wildcard “Action”:”*”).

Detection Strategy:
Detect policy-change API calls that add admin-level or wildcard permissions by principals not already in an approved admin list, then watch for immediate privileged activity.

Detection Instructions:

1. CloudTrail

filter  eventName in (AttachUserPolicy, PutUserPolicy, AttachRolePolicy, PutRolePolicy, CreatePolicyVersion, SetDefaultPolicyVersion) AND (requestParameters.policyArn:*AdministratorAccess* OR policy JSON contains “Action":"*").

2. Exclude

userIdentity.arn 

in allow-list of legitimate admin/automation roles.

3. Within 10 min of the policy change, look for high-risk APIs such as:

DeleteTrail, CreateAccessKey, PutBucketAcl, etc.

Detection Query Example:

(eventName:(AttachUserPolicy OR PutUserPolicy OR AttachRolePolicy OR PutRolePolicy OR CreatePolicyVersion OR SetDefaultPolicyVersion) AND (requestParameters.policyArn.keyword:*AdministratorAccess* OR requestParameters.policyDocument:("Action":"*"))) AND NOT userIdentity.arn.keyword:IN(approved_admin_roles)

Alerting Criteria:
Any admin-level or wildcard policy attached by a non-admin principal; raise severity if followed by privileged resource actions.False Positive Notes:
Legitimate change-window elevation, break-glass access, or CloudFormation stacks that intentionally attach AdministratorAccess.

Threat hunting with Olly: What it enables

Olly supports a more intuitive and efficient approach to threat hunting by allowing analysts to explore security and observability data using natural language. Instead of starting with complex queries or navigating multiple dashboards, analysts can ask plain-English questions across cloud logs, identity data, application logs, and metrics—making it easier to quickly form and test hypotheses.

Natural-language driven hunting

Olly allows analysts to interact with data using questions rather than queries. This reduces the friction of exploratory investigations and enables faster iteration when testing or refining a hunting hypothesis.

Cross-dataset investigation

A key strength of Olly is its ability to correlate signals across data sources. Analysts can pivot from a suspicious AWS API call to related network flows, application traces, and error patterns within a single interface, helping build a complete attack narrative without manual correlation.

Anomaly and pattern recognition

Olly helps surface meaningful signals by summarizing spikes, rare events, and deviations from normal behavior. Whether it’s an unexpected increase in API calls, anomalous IP activity, or unusual error trends, this allows analysts to focus on likely root causes rather than isolated events.

Rapid context gathering

During investigations, Olly accelerates access to relevant context. Questions such as “What changed before this incident?” or “Show all unusual activity for this user” return timelines, configuration changes, and related activity quickly, enabling analysts to move from suspicion to understanding faster.

Supporting iterative threat hunting

While these capabilities are powerful individually, their real value emerges when used together. Threat hunting with Olly is an iterative process—forming a hypothesis, exploring related signals across data sources, and refining questions as new context emerges. The following examples demonstrate how Olly can be applied in real-world scenarios to guide investigation through reasoning rather than predefined alerts.

Practical cases of threat hunting using Olly

The following scenarios demonstrate how Olly can be used in real-world threat hunting workflows. Each example highlights how natural-language exploration and cross-dataset correlation help uncover suspicious behavior beyond traditional alerting.

Let’s take a look at what Olly can detect using the proposed hypothesis we showcased earlier in the previous section. You can use the template of the hypothesis structure as a prompt and let Olly investigate per the provided instructions.

1. AWS: IAM Policy Escalation (H5)

This hunt focuses on identifying potential privilege escalation through IAM policy changes. Using Olly, analysts can search for events related to policy attachments, role modifications, or permission updates and quickly assess whether these actions align with expected administrative behavior. By examining who performed the change, what permissions were added, and when the activity occurred, Olly helps determine whether the behavior represents legitimate administration or a possible escalation attempt.

Below is the first of response generated by Olly after running through the hypothesis investigation guidelines:

Right away Olly also produces a short and concise Key Findings summary:

This is followed by a section offering Recommended next steps for further investigation:

Next we will asking Olly to perform further action in free text as shown below:

New recommendations are generated dynamically per new findings:

We then can choose to Expand the correlation window:

As more findings are gathered, Olly then generated an updated and expanded Key takeaways summary with additional Risk Assessment perspective:

New recommendations are then generated with additional suggestions on how to proceed the investigation:

This hunt demonstrates how Olly helps identify potential privilege escalation by adding context to IAM policy changes. By examining who made the change, what permissions were introduced, and how the activity fits within normal behavior, analysts can distinguish legitimate administration from actions that warrant further investigation.

Let’s examine another example where Olly was able to find an actual anomaly in Okta Audit Logs.

2. Okta: MFA failure followed by success from a different IP (H-OKTA-016)

This scenario targets identity-based attacks where an attacker may successfully authenticate after multiple failed MFA attempts. Olly can be used to analyze Okta authentication logs to identify sequences where MFA failures are followed by successful logins from a different IP address or location. By reviewing timing, IP changes, and user context, analysts can assess whether the activity suggests MFA fatigue, credential compromise, or legitimate user behavior.

Below is a proposal for a hypothesis template you can use as a prompt in Olly:

Hypothesis ID: H-OKTA-016

Hypothesis Name: MFA Fail then Success From Different IP

Hypothesis Summary:
An attacker fails multi-factor authentication from one IP address and then successfully completes MFA from a different IP shortly afterward, suggesting credential compromise or MFA abuse.

Detection Strategy:
Compare IP addresses associated with MFA authentication events for each user to identify changes between failed and successful attempts.

Detection Instructions:
Monitor user.authentication.auth_via_mfa events and inspect client IP addresses to detect situations where MFA failures and subsequent successes originate from different IPs.

Detection Query Example:

okta.eventType:user.authentication.auth_via_mfa

Alerting Criteria:
A different IP address or geolocation is observed between failed and successful MFA attempts within a 10-minute window.

False Positive Notes:
Mobile users switching between cellular, Wi-Fi, or VPN connections may produce similar patterns.

Below is the first result after asking Olly to perform the hunt based on the hypothesis:

Note that you can expand each findings box to reveal the Debug Logs for each DataPrime query that is being dynamically generated and executed in the background to retrieve the results from the integrated data sources.

As shown above Olly detected 65 time windows where a user failed MFA, then succeeded from a different IP within 10 minutes. The next proposed actions are then generated, and we pick an option to Focus on users with multiple failures (≥ 3) or ≥ 3 IPs in a single window so we can examine increased ratios in hopes of flagging an anomaly.

In the Noteworthy cases flagged by Olly, case 1 immediately sticks out with additional comments indicating a possible brute-force MFA attempt. We make note of the offending actor ID to be used later.

We then ask Olly to map all users based on the Okta Actor ID so we can easily identify the human identities involved – this shows Olly can understand context and reference relevant data already contained in the logs (ie mapping actor IDs to Email handles):

The offending actor ID and Email address are outlined in red, and moving forward we will only focus on Ashley’s activity in Okta. In the example below I ask Olly to focus on Ashley’s activity before and after the observed MFA Failures:

Findings Summary:

We then ask Olly to create a summary of Ashley’s actions before and after the failed MFA attempts with focus on event types and then produce a story and timeline of events:

Below are the Interpretations and Recommendations suggested by Olly:

Here is where Olly can really shine in the context of performing accurate threat hunting and threat assessments – We ask Olly to validate his own verdict:

The Olly proposes additional actions that can aid in reaching higher certainty and gives out its final verdict on Ashley’s behavior analysis:

Olly can then be used to produce a CISO level summary of our findings, which is again super useful when investigating an incident and produce summaries on the go:

This scenario highlights how Olly enables effective investigation of identity-based attacks by analyzing authentication patterns over time. Correlating MFA failures with subsequent successful logins from different IP addresses allows analysts to assess potential account compromise and prioritize follow-up actions with greater confidence.

3. Cross-source correlation: Identity to cloud privilege escalation

This hunt correlates suspicious identity activity with cloud-level impact. Olly enables analysts to connect a potential identity compromise—such as anomalous Okta authentication behavior—to subsequent privileged actions in AWS. Following the same user or role across data sources helps build a complete attack narrative from initial access to privilege escalation.

We asked Olly to try and combine several threat hunting hypotheses we created for both Okta and AWS and try to hunt based on its proposed correlation of hypothesis, and then elaborate on his reasoning behind picking these in particular, as shown below

After asking Olly to execute correlation above, we get the following results back after a couple of minutes:

Olly was able to flag one Admin (actor marked in green) adding another member to an admin group in Okta. Olly then gives its summary of observed findings:

By correlating identity signals with cloud activity, this hunt shows how seemingly independent events can form a coherent attack narrative. Olly makes it possible to follow an adversary’s path across platforms, helping analysts validate hypotheses and identify privilege escalation that might otherwise go unnoticed.

Closing thoughts: Evolving threat hunting with Olly

Threat hunting with Olly enables analysts to move beyond isolated alerts and investigate behavior across identity, cloud, and application layers. By combining natural-language exploration with cross-source correlation, security teams can uncover attacker intent, validate hypotheses, and build meaningful attack narratives. As environments become more dynamic and adversary behavior increasingly blends into normal operations, this ability to reason across data sources is essential for effective, modern threat hunting.