Log Analytics 2019 - Coralogix partners with IDC Research to uncover the latest requirements by leading companies

FREE REPORT

INTEGRATIONS

CircleCI

Coralogix and CircleCI

Integrate your CircleCI workflows and jobs pipeline with Coralogix to automatically receive reports and analyze version upgrades for their impact on the overall quality of your production system.

Coralogix Orb provides four tools:
coralogix/stats (job) – send the final report of a workflow job to Coralogix.
coralogix/logs (job) – send logs of workflow jobs to Coralogix for debugging.
coralogix/send (command) – send additional 3rd party logs generated during your workflow job to Coralogix.
coralogix/tag (command) – create a tag (and a report) for the workflow in Coralogix.

GitHub Repository

Getting Started

1. Use CircleCI version 2.1 at the top of your .circleci/config.yml file. If you do not already have Pipelines enabled, you’ll need to go to Project Settings -> Advanced Settings and turn it on.

version: 2.1

2. Add the orbs stanza below your version, invoking the orb:

orbs:
coralogix: coralogix/[email protected]

3. Use coralogix elements in your existing workflows and jobs.

4. Opt-in to use of third-party orbs on your organization’s Security settings page.

 

Usage Examples

Logs
This example shows how you can debug a workflow job in Coralogix. By adding our Orb and the coralogix/logs job at the end of the flow, you’ll receive all logs CircleCI generates during the workflow in your Coralogix account, helping you debug the different jobs.

Note that this may generate a large amount of data, and is recommended for debugging purposes only.

jobs:
  build:
    docker:
      - image: 'cimg/base:stable'
    steps:
      - run: echo "Typical build job"
orbs:
  coralogix: coralogix/[email protected]
version: 2.1
workflows:
  your-workflow:
    jobs:
      - build
      - coralogix/logs

Send
This example shows how you can send additional 3rd party logs generated during your workflow job to Coralogix.
By adding our Orb and the coralogix/send step, you can send additional logs from an external file to your Coralogix account.
This is especially useful if one of the stages of your job generates an external file with logs which you’d like to report to Coralogix and analyze along with the rest of your logs.

jobs:
  build:
    docker:
      - image: 'cimg/base:stable'
    steps:
      - run: echo "Build the application" > build.log
      - coralogix/send:
          file: build.log
orbs:
  coralogix: coralogix/[email protected]
version: 2.1
workflows:
  your-workflow:
    jobs:
      - build

Stats
This example shows how you can send the final report of a workflow job to Coralogix.
By adding our Orb and the coralogix/stats job at the end of the flow, you’ll receive the report automatically in your Coralogix account and can analyze and aggregate the results.

jobs:
  build:
    docker:
      - image: 'cimg/base:stable'
    steps:
      - run: echo "Typical build job"
orbs:
  coralogix: coralogix/[email protected]
version: 2.1
workflows:
  your-workflow:
    jobs:
      - build
      - coralogix/stats

Tag
This example shows how you can create a tag (and a report) for the workflow in Coralogix.
By adding our Orb and the coralogix/tag step, you can create tag (and a report) for this workflow in your Coralogix account to understand what effect it has on your production.
Learn more about tags

jobs:
  build:
    docker:
      - image: 'cimg/base:stable'
    steps:
      - run: echo "Build the application"
  deploy:
    docker:
      - image: 'cimg/base:stable'
    steps:
      - run: echo "Deploy the application"
      - coralogix/tag:
          application: MyApp
          subsystems: 'frontend,backend'
          tag: v1.0.0
  test:
    docker:
      - image: 'cimg/base:stable'
    steps:
      - run: echo "Test the application"
orbs:
  coralogix: coralogix/[email protected]
version: 2.1
workflows:
  your-workflow:
    jobs:
      - build
      - test:
          requires:
            - build
      - deploy:
          requires:
            - test

Jobs

Logs
This job is used to debug CircleCI workflow jobs in Coralogix by sending all the debug logs.
Note that this may generate a large amount of data, and is recommended for debugging purposes only.

Show Job Source

PARAMETER DESCRIPTION REQUIRED DEFAULT TYPE
application Your application name CircleCI string
category The log’s category logs string
circle-token Your CircleCI personal access token for interacting with the API. You can generate one here: https://circleci.com/account/api CIRCLE_TOKEN env_var_name
hostname Machine hostname. $(hostname) string
keep_artifacts Keep temporary log files generated during the job after being sent to coralogix. false boolean
private_key Private key provided by Coralogix (under Settings –> Send Your Logs). CORALOGIX_PRIVATE_KEY env_var_name
subsystem Your subsystem name. workflow-stats string

Commands

send
This command is used to send additional 3rd party logs generated during your workflow job to Coralogix.

PARAMETER DESCRIPTION REQUIRED DEFAULT TYPE
application Your application name CircleCI string
category The log’s category. artifact string
class The log’s class name. string
file Absolute or relative path to the file for sending. Yes string
hostname Machine hostname. $(hostname) string
method The log’s method name. string
newline_pattern Regular expression to split records in the file. \n|\r|\r\n string
private_key Private key provided by Coralogix (under Settings –> Send Your Logs). CORALOGIX_PRIVATE_KEY env_var_name
subsystem Your subsystem name. workflow string
thread Thread ID reported with the log. ‘${CIRCLE_WORKFLOW_ID}’ string

Tag
This command is used to create a tag (and a report) for the workflow in Coralogix, helping you understand what effect it has on your production.
Learn more about tags

PARAMETER DESCRIPTION REQUIRED DEFAULT TYPE
application Your application name. Yes string
icon A custom image to use for your tag. URL should point to a valid image file that is publicly available (png, jpeg, SVG). Max size 50KB. string
private_key Private key provided by Coralogix (under Settings –> Send Your Logs). CORALOGIX_PRIVATE_KEY env_var_name
subsystems Your subsystem names. You can set more than one subsystem name by using ‘,’ comma as a delimiter between subsystems names. Yes string
tag Name presented for the tag. Yes string

Workflow
Defines the command to help ship workflow logs to Coralogix.

PARAMETER DESCRIPTION REQUIRED DEFAULT TYPE
circle-token Your CircleCI personal access token for interacting with the API. You can generate one here: https://circleci.com/account/api CIRCLE_TOKEN env_var_name
keep_artifacts Keep generated workflow data. false boolean

Executors

Default
Default environment for Coralogix.

PARAMETER DESCRIPTION REQUIRED DEFAULT TYPE
image Docker image name cimg/base string
tag Docker image tag stable string

Orb Source


# This code is licensed from CircleCI to the user under the MIT license. See
# https://circleci.com/orbs/registry/licensing for details.
commands:
  send:
    description: |
      This command is used to send additional 3rd party logs generated during your workflow job to Coralogix.
    parameters:
      application:
        default: CircleCI
        description: Your application name.
        type: string
      category:
        default: artifact
        description: The log's category.
        type: string
      class:
        default: ""
        description: The log's class name.
        type: string
      file:
        description: Absolute or relative path to the file for sending.
        type: string
      hostname:
        default: $(hostname)
        description: Machine hostname.
        type: string
      method:
        default: ""
        description: The log's method name.
        type: string
      newline_pattern:
        default: \n|\r|\r\n
        description: Regular expression to split records in the file.
        type: string
      private_key:
        default: CORALOGIX_PRIVATE_KEY
        description: Private key provided by Coralogix (under Settings --> Send Your
          Logs).
        type: env_var_name
      subsystem:
        default: workflow
        description: Your subsystem name.
        type: string
      thread:
        default: ${CIRCLE_WORKFLOW_ID}
        description: Thread ID reported with the log.
        type: string
    steps:
    - jq/install
    - run:
        command: |
          if [ ! -f "<>" ]; then
              echo "File not found!"
              exit 0
          fi
          LOG_FILE=$(realpath "<>")
          LOG_BULKS=$(mktemp -d)
          pushd $LOG_BULKS
            split -C 2M -d "$LOG_FILE" ''
          popd
          for bulk in $LOG_BULKS/*; do
            LOG_BULK=$(jq -nMc \
              --rawfile lines "$bulk" \
              --arg private_key "${<>}" \
              --arg application "<>" \
              --arg subsystem "<>" \
              --arg hostname "<>" \
              --arg category "<>" \
              --arg class "<>" \
              --arg method "<>" \
              --arg thread "<>" '{
              "privateKey": $private_key,
              "applicationName": $application,
              "subsystemName": $subsystem,
              "computerName": $hostname,
              "logEntries": [
                $lines | split("<>"; "") | to_entries | .[] | {
                  "timestamp": (now * 1000 + .key),
                  "severity": 3,
                  "text": .value | select(.!=""),
                  "category": $category,
                  "className": $class,
                  "methodName": $method,
                  "threadId": $thread
                }
              ]
            }')
            curl -X POST -H 'Content-Type: application/json' -d "$LOG_BULK" -s 'https://api.coralogix.com/api/v1/logs' > /dev/null
          done
          rm -rf "$LOG_BULKS"
        name: Send to Coralogix
        when: always
  tag:
    description: |
      This command is used to create a tag (and a report) for the workflow in Coralogix, helping you understand what effect it has on your production.
      Learn more at https://coralogix.com/tutorials/software-builds-display/
    parameters:
      application:
        description: Your application name.
        type: string
      icon:
        default: ""
        description: |
          A custom image to use for your tag.
          URL should point to a valid image file that is publicly available (png, jpeg, SVG).
          Max size 50KB.
        type: string
      private_key:
        default: CORALOGIX_PRIVATE_KEY
        description: Private key provided by Coralogix (under Settings --> Send Your
          Logs).
        type: env_var_name
      subsystems:
        description: |
          Your subsystem names.
          You can set more than one subsystem name by using ',' comma as a delimiter between subsystems names.
        type: string
      tag:
        description: Name presented for the tag.
        type: string
    steps:
    - run:
        command: |
          curl "https://api.coralogix.com/api/v1/addTag?key=${<>}&application=<>&subsystem=<>&name=<>&iconUrl=<>"
        name: Push Coralogix Tag
  workflow:
    description: |
      Defines the command to help ship workflow logs to Coralogix.
    parameters:
      circle-token:
        default: CIRCLE_TOKEN
        description: |
          Your CircleCI personal access token for interacting with the API.
          You can generate one here: https://circleci.com/account/api
        type: env_var_name
      keep_artifacts:
        default: false
        description: Keep generated workflow data.
        type: boolean
    steps:
    - run:
        command: |
          WORKFLOW=$(curl -s -H "Circle-Token: ${<>}" "https://circleci.com/api/v2/workflow/$CIRCLE_WORKFLOW_ID")
          WORKFLOW_JOBS=$(curl -s -H "Circle-Token: ${<>}" "https://circleci.com/api/v2/workflow/$CIRCLE_WORKFLOW_ID/jobs" | jq '.items')
          WORKFLOW_LENGTH=$(echo "$WORKFLOW_JOBS" | jq 'length')
          PROJECT_SLUG=$(echo "$WORKFLOW" | jq -r '.project_slug')

          mkdir -p /tmp/coralogix
          rm -f /tmp/coralogix/workflow-*.log
          for (( i=0; i<$WORKFLOW_LENGTH; )); do
              WORKFLOW_JOBS=$(curl -s -H "Circle-Token: ${<>}" "https://circleci.com/api/v2/workflow/$CIRCLE_WORKFLOW_ID/jobs" | jq '.items')
              JOB_NUMBER=$(echo "$WORKFLOW_JOBS" | jq -r "sort_by(.job_number) | .[$i].job_number")
              JOB_STATUS=$(echo "$WORKFLOW_JOBS" | jq -r "sort_by(.job_number) | .[$i].status")
              JOB_NAME=$(echo "$WORKFLOW_JOBS" | jq -r "sort_by(.job_number) | .[$i].name")
              if [[ $JOB_NAME != coralogix/* ]]; then
                  if [ "$JOB_STATUS" != "running" ] && [ "$JOB_STATUS" != "blocked" ]; then
                      echo "Collecting data for job $JOB_NAME..."
                      JOB=$(curl -s -H "Circle-Token: ${<>}" "https://circleci.com/api/v1.1/project/$PROJECT_SLUG/$JOB_NUMBER")
                      JOB_LOGS=$(echo "$JOB" | jq '.steps[].actions[].output_url | select(.!=null)')
                      if [ -z "$JOB_LOGS" ]; then
                          sleep 10
                          continue
                      fi
                      echo "$JOB" | jq -c 'del(.steps, .circle_yml)' >> /tmp/coralogix/workflow-stats.log
                      echo "$JOB_LOGS" | xargs -n1 curl -s --output - --compressed | jq -r '.[].message' >> /tmp/coralogix/workflow-logs.log
                      i="$((i+1))"
                  else
                      echo "Job $JOB_NAME is $JOB_STATUS..."
                      sleep 10
                  fi
              else
                  i="$((i+1))"
              fi
          done
        name: Build workflow data
        when: always
    - when:
        condition: <>
        steps:
        - store_artifacts:
            path: /tmp/coralogix
description: |
  Integrate your CircleCI workflows and jobs pipeline with Coralogix to automatically receive reports and analyze version upgrades for their impact on the overall quality of your production system.
  Coralogix Orb provides four tools at your disposal:
    coralogix/stats (job) - send the final report of a workflow job to Coralogix.
    coralogix/logs (job) - send logs of workflow jobs to Coralogix for debugging.
    coralogix/send (command) - send additional 3rd party logs generated during your workflow job to Coralogix.
    coralogix/tag (command) - create a tag (and a report) for the workflow in Coralogix.
  Website: http:/www.coralogix.com
  Learn more: https://coralogix.com/category/tutorials/
  Repository: https://github.com/coralogix-circleci/coralogix-orb
examples:
  logs:
    description: |
      This example shows how you can debug a workflow job in Coralogix. By adding our Orb and the coralogix/logs job at the end of the flow, you'll receive all logs CircleCI generates during the workflow in your Coralogix account, helping you debug the different jobs.
      Note that this may generate a large amount of data, and is recommended for debugging purposes only.
    usage:
      jobs:
        build:
          docker:
          - image: cimg/base:stable
          steps:
          - run: echo "Typical build job"
      orbs:
        coralogix: coralogix/[email protected]
      version: 2.1
      workflows:
        your-workflow:
          jobs:
          - build
          - coralogix/logs
  send:
    description: |
      This example shows how you can send additional 3rd party logs generated during your workflow job to Coralogix.
      By adding our Orb and the coralogix/send step, you can send additional logs from an external file to your Coralogix account.
      This is especially useful if one of the stages of your job generates an external file with logs which you'd like to report to Coralogix and analyze along with the rest of your logs.
    usage:
      jobs:
        build:
          docker:
          - image: cimg/base:stable
          steps:
          - run: echo "Build the application" > build.log
          - coralogix/send:
              file: build.log
      orbs:
        coralogix: coralogix/[email protected]
      version: 2.1
      workflows:
        your-workflow:
          jobs:
          - build
  stats:
    description: |
      This example shows how you can send the final report of a workflow job to Coralogix.
      By adding our Orb and the coralogix/stats job at the end of the flow, you'll receive the report automatically in your Coralogix account and can analyze and aggregate the results.
    usage:
      jobs:
        build:
          docker:
          - image: cimg/base:stable
          steps:
          - run: echo "Typical build job"
      orbs:
        coralogix: coralogix/[email protected]
      version: 2.1
      workflows:
        your-workflow:
          jobs:
          - build
          - coralogix/stats
  tag:
    description: |
      This example shows how you can create a tag (and a report) for the workflow in Coralogix.
      By adding our Orb and the coralogix/tag step, you can create tag (and a report) for this workflow in your Coralogix account to understand what effect it has on your production.
      Learn more at https://coralogix.com/tutorials/software-builds-display/
    usage:
      jobs:
        build:
          docker:
          - image: cimg/base:stable
          steps:
          - run: echo "Build the application"
        deploy:
          docker:
          - image: cimg/base:stable
          steps:
          - run: echo "Deploy the application"
          - coralogix/tag:
              application: MyApp
              subsystems: frontend,backend
              tag: v1.0.0
        test:
          docker:
          - image: cimg/base:stable
          steps:
          - run: echo "Test the application"
      orbs:
        coralogix: coralogix/[email protected]
      version: 2.1
      workflows:
        your-workflow:
          jobs:
          - build
          - test:
              requires:
              - build
          - deploy:
              requires:
              - test
executors:
  default:
    description: |
      Default environment for Coralogix.
    docker:
    - image: <>:<>
    parameters:
      image:
        default: cimg/base
        description: Docker image name
        type: string
      tag:
        default: stable
        description: Docker image tag
        type: string
    resource_class: small
jobs:
  logs:
    description: |
      This job is used to debug CircleCI workflow jobs in Coralogix by sending all the debug logs.
      Note that this may generate a large amount of data, and is recommended for debugging purposes only.
    executor: default
    parameters:
      application:
        default: CircleCI
        description: Your application name.
        type: string
      category:
        default: logs
        description: The log's category.
        type: string
      circle-token:
        default: CIRCLE_TOKEN
        description: |
          Your CircleCI personal access token for interacting with the API.
          You can generate one here: https://circleci.com/account/api
        type: env_var_name
      hostname:
        default: $(hostname)
        description: Machine hostname.
        type: string
      keep_artifacts:
        default: false
        description: Keep temporary log files generated during the job after being
          sent to coralogix.
        type: boolean
      private_key:
        default: CORALOGIX_PRIVATE_KEY
        description: Private key provided by Coralogix (under Settings --> Send Your
          Logs).
        type: env_var_name
      subsystem:
        default: workflow-logs
        description: Your subsystem name.
        type: string
    steps:
    - workflow:
        circle-token: <>
        keep_artifacts: <>
    - send:
        application: <>
        category: <>
        file: /tmp/coralogix/workflow-logs.log
        hostname: <>
        private_key: <>
        subsystem: <>
  stats:
    description: |
      This job is used to get the final report of a workflow job in Coralogix where you can analyzea and aggregate the results.
    executor: default
    parameters:
      application:
        default: CircleCI
        description: Your application name.
        type: string
      category:
        default: logs
        description: The log's category.
        type: string
      circle-token:
        default: CIRCLE_TOKEN
        description: |
          Your CircleCI personal access token for interacting with the API.
          You can generate one here: https://circleci.com/account/api
        type: env_var_name
      hostname:
        default: $(hostname)
        description: Machine hostname.
        type: string
      keep_artifacts:
        default: false
        description: Keep temporary log files generated during the job after being
          sent to coralogix.
        type: boolean
      private_key:
        default: CORALOGIX_PRIVATE_KEY
        description: Private key provided by Coralogix (under Settings --> Send Your
          Logs).
        type: env_var_name
      subsystem:
        default: workflow-stats
        description: Your subsystem name.
        type: string
    steps:
    - workflow:
        circle-token: <>
        keep_artifacts: <>
    - send:
        application: <>
        category: <>
        file: /tmp/coralogix/workflow-stats.log
        hostname: <>
        private_key: <>
        subsystem: <>
orbs:
  jq: circleci/[email protected]
version: 2.1

Start solving your production issues faster

Let's talk about how Coralogix can help you better understand your logs

No credit card required

Get a personalized demo

Jump on a call with one of our experts and get a live personalized demonstration