Prebuilt Policies

Coralogix provides prebuilt policies that you can apply to any AI application. Each evaluation includes configuration options, such as where to apply it (user prompt, LLM response, or both) and which categories to include or restrict.

This guide presents:

A full list of prebuilt policies available for evaluation in the Policy Catalog
How to create evals for prebuilt policies
How to manage evals

Select a policy

Select one or more of the following policies.

Security

Topic	Description
Prompt injection	Detects any user attempt of prompt injection or jailbreak.
SQL (read-only access)	Detects any attempt to use SQL operations which requires more than read-only access.
SQL (load limit)	Detects SQL statements that are likely to cause significant system load and affect performance.
SQL (restricted tables)	Detects the generation of SQL statements that access specific tables considered sensitive.
SQL (allowed tables)	Detects SQL operations on tables that are not configured in the eval.
PII	Detects the existence of PII (Personally Identifiable Information) in the user message or the LLM response based on the configured sensitive data types.

Hallucinations

Topic	Description
Context adherence	Measures whether the model's response strictly follows the provided context without introducing new information.
Context relevance	Assesses how relevant and similar the provided context is to the user query, ensuring it contains the necessary information for an accurate response.
Completeness	Evaluates how well the model’s response includes all relevant information from the context.
Correctness	Determines if a model's response is factually accurate.
SQL hallucination	Detects hallucinations in LLM-generated SQL queries.
Tool parameter correctness	Ensures the correct tools are selected and invoked with accurately derived parameters based on chat history.
Task adherence	Detects whether the LLM's answer aligns with the given system prompt.

Toxicity

Topic	Description
Sexism	Detects whether an LLM response or user prompt contains sexist content.
Toxicity	Detects any user message or LLM response containing toxicity.

Topics

Topic	Description
Restricted topics	Detects any user or LLM attempt to initiate a discussion on the topics mentioned in the eval.
Allowed topics	Ensures the conversation adheres to specific and well-defined topics.
Competition discussion	Detects whether any prompt or response includes references to competitors mentioned in the eval.

User experience

Topic	Description
Language mismatch	Detects when an LLM is answering a user question in a different language.

Compliance

Topic	Description
Restricted phrases	Ensures the LLM does not use specified prohibited terms and phrases by blocking or replacing them based on regex patterns.

Create evals for prebuilt policies

Use the Policy Catalog to add an evaluation to a policy.

Browse the Policy Catalog tiles to find the policy you want.
If the policy is not visible, use the Search field to locate it.
To narrow the policy list, do one or more of the following:
- Select a category.
- Filter by Evaluations.
On the policy card, select Add eval.
Configure the evaluation:
- If no configuration is required, Coralogix adds the evaluation to the application immediately.
- If configuration is required, set the relevant options:
  - Select whether to run the evaluation on the user prompt, the LLM response, or both.
  - Select categories (for example, self-harm or violence for the Toxicity evaluation), values (for example, topics for the Restricted topics evaluation), or other attributes available for the policy type.
Select Next.
In the Add eval to application dialog box, select the application or applications you want to apply the evaluation to.
- Applications that already use this evaluation are unavailable.
Select Done.