Coralogix Blog

Kafka Streams Window By & RocksDB Tuning

Kafka Streams offers a feature called a window. In this post, I will explain how to implement tumbling time windows in Scala, and how to tune RocksDB accordingly. Kafka Streams Terminology Clock Time:

Yonatan Kiron

BetterHelp Customer Case Study

Untangling a chaotic mess can be daunting. And that’s exactly the problem Alain Adler, the Head of Engineering at BetterHelp, faced. BetterHelp is the world’s largest e-counseling platform, on a

Coralogix

Analyzing Log Data: Why It’s Important

From production monitoring to security concerns, businesses need to analyze and review their logs on a daily basis to make sure their system is up to par. Here are the

Ariel Assaraf

Heroku Continuous Integration & Deployment with Docker [Hands-On Tutorial]

In this tutorial, we will be using Heroku to deploy our Node.js application through CircleCI using Docker. We will set up Heroku Continuous Integration and Deployment (CI/CD) pipelines using Git

Ankit Jain

Onelogin Log Insights with Coralogix

OneLogin is one of the top leading Unified Access Management platforms, enabling organizations to manage and Access their cloud applications in a secure way. OneLogin makes it simpler and safer for

Shuki Shahrabany

Unleash your Auth0 Log Insights With Coralogix

Auth0 is one of the top leading identity management platforms in the world. It’s focused on providing solutions for application builders, specifically solutions needed for custom-built applications. Auth0 provides expertise

Shuki Shahrabany

Tutorial: Elasticsearch Snapshot Lifecycle Management (SLM)

Let’s face it, nothing is perfect. The better we architect our systems, though, the more near-perfect they become. But even so, someday, something is likely to go wrong, despite our

Stanislav Prihoda

A Practical Guide to Logstash: Shipping Logs to Coralogix with Logstash

Logstash is a tool to collect, process, and forward events and log messages and this Logstash tutorial will get you started quickly. It was created by Jordan Sissel who, with

Amir Raz

Kibana Settings: Spaces, Export Dashboard, and more

Kibana is considered the “window” to Elasticsearch and indeed it’s a powerful UI for searching, filtering, analyzing, and visualizing Elasticsearch data, but Kibana settings are also used to configure, administer

Stanislav Prihoda

Managing Docker Logs with ELK and Fluentd

This article provides an overview of managing and analyzing Docker logs and explores some of the complexities that may arise when looking through the log data. Challenges to overcome: Collecting

Samuel Slade

Filebeat Configuration Best Practices Tutorial

In this post, we will cover some of the main use cases Filebeat supports and we will examine various Filebeat configuration use cases. Filebeat, an Elastic Beat that’s based on

Shuki Shahrabany

Elasticsearch Performance Tuning – Index Design

You’ve created the perfect design for your indices and they are happily churning along. However, in the future, you may need to reconsider your initial design. Maybe you want to

Stanislav Prihoda

Using Log Data to Prevent Lambda Cold Starts

AWS Lambda enables you to run serverless functions in the AWS cloud, by manually triggering functions or by creating trigger events. To ensure your Lambda functions are running smoothly, you

Nic Sevic

Protect Your AWS Infrastructure with GuardDuty and Coralogix

What is GuardDuty Cloud environments like AWS can be a challenge for security monitoring services to operate in since assets tend to dynamically appear and disappear. Making matters more challenging,

Amir Raz

Integrating Coralogix Alerts with PagerDuty

Coralogix offers the most extensive alerting mechanism in the logging market, including built-in webhooks for PagerDuty, custom webhooks, and automatic resolve notifications. PagerDuty is a leader in digital operations management. In

Lior Redlus

Tutorial: Logstash Grok Patterns with Examples

Logstash can parse CSV and JSON files easily because data in those formats are perfectly organized and ready for Elasticsearch analysis. Sometimes, though, we need to work with unstructured data,

Arun Mohan

Okta Log Insights

This post will show you how Coralogix can provide analytics and insights for your Okta logs, both performance, and security. Okta is one of the leading Identity provider platforms in

Shuki Shahrabany

Advanced Guide to Kibana Timelion

Kibana Timelion is a time-series based visualization language that enables you to analyze time-series data in a more flexible way. compared to other visualization types that Kibana offers. Instead of

Shuki Shahrabany

Kubernetes Logging with Elasticsearch, Fluentd and Kibana

Kubernetes, a Greek word meaning pilot, has found its way into the center stage of modern software engineering. Its in-built observability, monitoring, metrics, and self-healing make it an outstanding toolset

Chris Cooney

How DevOps Monitoring Impacts Your Organization

DevOps monitoring didn’t simply become part of the collective engineering consciousness. It was built, brick by brick, by practices that have continued to grow and flourish with each new technological

Chris Cooney

AWS Lambda Logging & Monitoring Tutorial

Going serverless relieves you of setting up servers, updating operating systems, or maintaining physical infrastructure. But what happens when a function doesn’t work and things go wrong? Given the added

Felix Christl

Morgan NPM Logger – The Beginner’s Guide

In this guide, we’ll cover how you can use Morgan npm to log requests and other aspects of your web application built on Express (or any of the similarly architected frameworks around).

Fernando Doglio

Heroku Logs – The Complete Guide

Platforms like Heroku give you the freedom to focus on building great applications rather than getting lost setting up and maintaining infrastructure. One of the many great features of working

Jason Simon, Mark Ronald Moore

Fastly Logs Insights

This tutorial will show you how can Coralogix provide analytics and insights for the Fastly logs you ship to Coralogix, both performance, and security. To get all the Coralogix dashboards

Shuki Shahrabany

Elasticsearch Update Index Settings

You’ve created the perfect design for your indices and they are happily churning along. However, in the future, you may need to reconsider your initial design and update the Elasticsearch

Stanislav Příhoda

The Definitive Guide to Configuration Management Tools

Many of the available configuration management tools, such as Ansible, Terraform, Puppet, Chef, and Saltstack provide automation for infrastructure, cloud, compliance and security management, and integration for deployment and continuous deployment (CI / CD). But what is the best tool to start automating your particular environment?

Dário Estevão, Rafael Miranda

The Pivotal Role of Log Analytics in Modern IT Infrastructures

In this survey of over 200 CIOs in the US, the IDC analyses the critical role played by Log Analytics in any modern infrastructure. IDC Opinion To effectively manage modern

Coralogix

Complete Winston Logger Guide With Hands-on Examples

Logging is critical for monitoring and troubleshooting your Node.js project. The open-source Winston logger helps take a load off our shoulders by making it easier to centralize, format, enrich, and

Fernando Doglio

Getting Started with Grafana Dashboards using Coralogix

One of the most common dashboards for metric visualization and alerting is, of course, Grafana. In addition to logs, we use metrics to ensure the stability and operational observability of our

Mary Mats

Flattened Datatype Mappings – Elasticsearch Tutorial

1. Introduction In this article, we’ll learn about the Elasticsearch flattened datatype which was introduced in order to better handle documents that contain a large or unknown number of fields.

Arun Mohan

Continuously manage your CircleCI implementation with Coralogix

For many companies, success depends on efficient build, test and delivery processes resulting in higher quality. However, development and deployment environments can become complex very quickly, even for small and

Amir Raz

Python Logging Guide – Best Practices and Hands-on Examples

In the vast computing world, there are different programming languages that include facilities for logging. From our previous posts, you can learn best practices about Node logging, Java logging, and

Venkatesh-Prasad Ranganath, Priscill Orue

A practical guide to FluentD

In this post we will cover some of the main use cases FluentD supports and provides example FluentD configurations for the different cases. What is Fluentd Fluentd is an open

Amir Raz

Overcoming DNS barriers for Kubernetes Scaling

It was a cloudy winter morning when I had arrived at the office and found, to our horror, that a Kubernetes cluster was suffering from extremely high CPU and network usage and had become almost completely non-functional. To make things worse, restarting

Yuval Khalifa

Guide: Parsing Multiline Logs

In the context of logging, multiline logs happen when a single log is written as multiple lines in the log file. This can either be caused by not using a

Shuki Shahrabany

Easily Build Jenkins Pipelines – Tutorial

Are you building and deploying software manually and would like to change that? Are you interested in learning about building a Jenkins pipeline and better understand CI/CD and DevOps at

Valentin Despa

Elasticsearch Disk and Data Storage Optimizations with Benchmarks

Out of the four basic computing resources (storage, memory, compute, network), storage tends to be positioned as the foremost one to focus on for any architect optimizing an Elasticsearch cluster.

Stanislav Příhoda

42 Elasticsearch Query Examples – Hands-on Tutorial

Elasticsearch provides a powerful set of options for querying documents for various use cases so it’s useful to know which query to apply to a specific case. The following is

Arun Mohan

Optimizing a Golang service to reduce over 40% CPU

10 years ago, Google was facing a critical bottleneck caused by extremely prolonged C++ compilation times and needed a totally new way to solve it. Google engineers tackled the challenge

Eliezer Yaacov

How to get the most out of your ELB logs

What is ELB Amazon ELB (Elastic Load Balancing) allows you to make your applications highly available by using health checks and intelligently distributing traffic across a number of instances. It

Amir Raz

Guide: Smarter AWS Traffic Mirroring for Stronger Cloud Security

So, you’ve installed Coralogix’s STA and you would like to start analyzing your traffic and getting valuable insights but you’re not sure that you’re mirroring enough traffic or wondering if

Yuval Khalifa

Using Auditbeat to protect your critical infrastructure

What are Beats Beats are lightweight, purpose-built agents that acquire data and then feed it to Elasticsearch. Beats use the libbeat framework that makes it easy to create customized beats for any

Amir Raz

Terraform Quick Start Tutorial

This Terraform tutorial is an excellent introduction to configuration management tools. Terraform is growing due to it simplifying the management of environments in the cloud. Several cloud services appeared over

Dário Estevão, Rafael Augusto

Ansbile Quick Start Tutorial

This quick start Ansible tutorial will explain how to get started quickly using this open-source configuration management tool. Due to the great (and increasing) number of supported resources and its

Dário Estevão, Rafael Augusto

Puppet Quick Start Tutorial

Puppet is an open-core software configuration management tool recommended for orchestrating environments that have rigid compliance requirements, maintaining an immutable configuration of nodes, with reports and role-based access control. This

Dário Estevão, Rafael Augusto

How to Install and Configure Jenkins on the Mac OS

Are you looking for information on how to install Jenkins for Mac OS? Before we get started, there are (at least) two ways to install Jenkins on your macOS system

Valentin Despa

How to Install and Configure Jenkins on Windows 10

If you’re looking for information on how to install Jenkins on Windows 10 this is the tutorial for you. We’ll also discuss how to configure it to get your going

Valentin Despa

10 Alerts and Visualizations for S3 Server Access Logs to take control of AWS infrastructure

AWS S3 access logs provide detailed records for requests made to S3 buckets. They’re useful for many applications. For example, access log information can be useful in security and access

Amir Raz

13 Security Alerts and Visualizations for VPC Flow Logs

AWS VPC Flow Logs record details about the traffic passing through your application, including requests that were allowed or denied according to your ACL (access control list) rules. It also

Amir Raz

Guide: RegEx 101 for Managing Log Data

Regular expressions can be crucial for wrangling log data efficiently. You may want to extract specific data from your logs to make it easier to analyze and visualize. Sometimes you

Andrei Chernikov

Does Complexity Equal Security?

“Teacher somewhere in India: The world you see is supported by a giant turtle. Student: And what holds this giant turtle down? Teacher: Another giant turtle, of course. Student: And

Yuval Khalifa

The who, the what, the where & when, the why & some AI

When you hear the word “permission” what’s the first thing that comes to mind? For me, it’s the picture in which I’m telling my kids not to do some mischief.

Yuval Khalifa

Does Open Source Equal Free?

Open source tools have been gaining momentum in recent years, largely due to the contribution of several factors, including improved user interfaces, the inability of commercial companies to compete on

Yuval Khalifa

Achieve better AWS security with just 10 Cloudtrail logs alerts

CloudTrail logs track actions taken by a user, role, or an AWS service, whether taken through the AWS console or API operations. In contrast to on-premise-infrastructure where something as important

Amir Raz

Avoiding death by external side effects — a tale of Kafka Streams

At Coralogix, we strive to ensure that our customers get a stable, real-time service at scale. As part of this commitment, we are constantly improving our data ingestion pipeline resiliency

Yoni Farin

Instantly Parse The Top 12 Log Types

Throughout the past few months, I had the opportunity to work with and serve hundreds of Coralogix’s customers, the challenges in performing efficient Log Analytics are numerous, from collecting, searching,

Mary Mats

AWS Centralized Logging Guide

The key challenge with modern visibility on clouds like AWS is that data originates from various sources across every layer of the application stack, is varied in format, frequency, and

David Bitton

Introduction to Kafka Tutorial

Kafka is an open source real-time streaming messaging system and protocol built around the publish-subscribe system. In this system, producers publish data to feeds for which consumers are subscribed to.

David Bitton

Create Kafka Topics in 3 Easy Steps

Creating a topic in production is an operative task that requires awareness and preparation. In this tutorial, we’ll explain all the parameters to consider when creating a new topic in

Yoni Farin

Coralogix teams up with AWS Marketplace for Log Management

AWS offers a plethora of log and metric data but in order to extract meaningful insights and react to production issues on a dime, a centralized logging solution is critical.

Coralogix

Elasticsearch Mapping Exceptions – The complete guide

As Elasticsearch is gradually becoming the standard for textual data indexing (specifically log data) more companies struggle to scale their ELK stack. We decided to pick up the glove and

Shuki Shahrabany

CDNs and Why You Need Them

This post will explain about CDNs, one of the most important components of the modern web applications. It’s the technology that allows the world’s most popular events to be brought

Lior Tal

New Relic Vs Appdynamics Vs Dynatrace

We’ll look at 3 popular APM tools: New Relic, Appdynamics and Dynatrace to see where their strengths and weaknesses lie, how they can help you improve, and how they compare

Ariel Assaraf

Test Automation Tools to Accelerate CI/CD

So much of our world has moved away from the slow and methodical, towards the agile and iterative. In transport, for example, everything is “on demand”, constantly changing and adaptable.

Lior Tal

EKS vs. KOPS

In the past, applications would be deployed by installation on a host, using the operating system package manager. This was a heavy solution with tremendous reliance on the operating system

Ariel Assaraf

Splunk vs SumoLogic vs ELK

When it comes to the best platforms for log analysis and security information and event management (SIEM) solutions, 3 trivial alternatives come up: Splunk, Sumo Logic and ELK.

Ariel Assaraf

Terraform vs Helm Charts

Since Docker first came onto the scene in 2013 and really popularized containerization, many organizations have chosen to deploy cloud workloads using Docker containers. Containers come with numerous benefits over

Lior Tal

Travis CI vs CircleCI

Travis CI vs CircleCI The way we think about development started with Waterfall – sequential, solid, conservative – moved to Agile, whose origins can be traced back to a somewhat

Ariel Assaraf

AWS Lambda vs Azure Functions vs Google Cloud Functions

Serverless computing is on the rise, having already earned the mantle of “The Next Big Thing”. For developers, serverless computing means less concern regarding infrastructure when deploying code, as all

Lior Tal

Grafana Vs Graphite

The amount of data being generated today is unprecedented. In fact, more data has been created in the last 2 years, than in the entire history of the human race.

Ariel Assaraf

Log Management Comparison: ELK vs Graylog

Production logs can help ensure application security, reveal business insights and find and understand errors, crashes, and exceptions. But as useful as logs are, they’re difficult to manage and hard

Lior Tal

How To Explain DevOps – 10 Ways To Get It Perfectly Right

DevOps: 10 Explanations That Get It Perfectly Right Describing your profession to other people is never easy, especially if you work in the development field. Non-technical people often lack the

Ariel Assaraf

Ruby logging best practices and tips

Ruby is an opinionated language with inbuilt Ruby logging options that will serve the needs of small and basic applications. Whilst there are fewer alternatives to these than say, the

Coralogix

Let’s talk about your logs – a personal invitation

Hi there! I’m Ariel, CPO @Coralogix. If you are reading these lines then I’m guessing you are a developer, sysadmin, a DevOps engineer or an IT specialist, and that you

Coralogix

Why I love November in San- Francisco

I love San Francisco, I love being in a city so diverse and so passionate about tech, I love that the first billboards you see at the airport are of

Ariel Assaraf

Node logging best practices and tips

As is traditional with the JavaScript world, there are a dizzying amount of options for node logging. In this article, I will dig into some of the better and lesser

Coralogix

Game of Logs: delivery assurance is coming

Long gone are the days were software companies released a version every six months, after rigorous planning, testing and stress runs (AKA the waterfall model). Today, agile companies release versions

Guy Kroupp

Quick Tips for Heroku Logging with Coralogix

TL;DR: Coralogix released a new addon for Heroku: https://elements.heroku.com/addons/coralogix. In this post we cover best Logging practices, how different cloud providers enable them, what makes Heroku logging special, and how Coralogix

Coralogix

Tricks to being A better developer

The daily routine in the world of any Developer rarely ever reaches a point of tranquility. It takes real passion and drive to be able to seize each day, ready

Guy Kroupp

8 Awesome YouTube Channels for Developers To Subscribe To

You know how sometimes you go on YouTube with the intention to extract a bit of info, then end up being pulled into the rabbit hole of irrelevant videos? Don’t

Ariel Assaraf

Top 3 Time-Consuming Annoyances for Developers

Developers, Data Scientists, and Software Engineers alike have a lot in common when it comes to their jobs. They are in high demand, get paid well, and are amongst the

Guy Kroupp

DevOps: Overcoming the Barriers to Adoption

Considering the countless benefits of incorporating DevOps into practices, the barriers that are often faced towards adoption can be considered comical. If you’re facing difficulties in convincing the higher-ups that

Ariel Assaraf

How much does the free ELK stack cost you?

The free ELK stack (Elasticsearch, Logstash, Kibana) is not as free as it is cracked up to be. This post will focus on the costs of maintaining your own ELK

Ariel Assaraf

What DevOps Look for When Choosing a Logging and Monitoring Tool

By now DevOps are hopefully aware that among the multitude of modern analytics methods available to them, an advanced logging and monitoring tool provides the most complete view into how

Ariel Assaraf

How Machine Learning Is Revolutionizing Log analytics

One of Machine Learning’s main advantages is the ability to independently recognize and analyze patterns which is taking repetitive tasks in a range of industries and offloading them to be

Guy Kroupp

E-commerce logging – Why Understanding Site Data Is Critical to Success

Before we dive into e-commerce logging, a short history recap: In 1992, avowed book nerd and prescient businessman Charles M. Stack launched an e-commerce site called books.com. Online shopping was a

Ariel Assaraf

Testing Vs Debugging, which do you prefer?

“My job, in theory, is to build, test, deploy and maintain software. My job, in reality, is more like the Sisyphus vortex.” This is the response I received from a

Ariel Assaraf

DevOps: Release the Hidden Power of Your Log Files

Though the widespread embrace of the DevOps approach is showing no signs of winding down, its efficacy now appears to be a bone of contention among tech leaders. A recent

Guy Kroupp

Coralogix CSO on machine learning powered log analytics

Cloud environments and Open Source software lowered the bar for anyone to implement software solutions at scale. However, as software becomes more complex and at scale, the task of monitoring

Lior Redlus

Papertrail alternatives – why you should check them

What Is Papertrail? Papertrail describes itself as “frustration-free log management.” This characterization is relatively accurate, as its platform focuses on helping users manage their own logs, rather than intelligently analyzing

Ariel Assaraf

5 time-saving tools for C# development

John Lennon once famously said ‘Imagine there’s no debugging. It’s easy if you try.’ Ok…he didn’t say exactly that, but he would have if he’d tried to do some C#

Guy Kroupp

Terms Every Devops Pro Needs to Know

Fully implementing a DevOps culture is a demanding endeavor for even the best-prepared organization. In addition to developing new workflows and processes to integrate the development and operations functions of

Ariel Assaraf

Key Steps to Implementing Enterprise DevOps

Whether your team is currently using an Agile approach to manage its software development or has yet to institute a formal project management system, implementing enterprise DevOps solutions is a

Guy Kroupp

Five Tools Every Java Developer Needs

There are a lot of tools out there but these are the ones you really need Search the internet for “Java Developer Tools” and millions of articles come back. You’ll

Guy Kroupp

The Most Important Things to Log in Your Application Software

Hint – you should log to be proactive not just to have your data stored When it comes to deciding what events to log in a software application, you should

Ariel Assaraf

Website Performance Monitoring – You Are Doing It Wrong

With all the data available today, it’s easy to think that tracking website performance is as simple as installing a few tools and waiting for alerts to appear. Clearly the

Ariel Assaraf

Loggly Alternatives

If you’re looking to gain more than just logging management from your log management host, Loggly isn’t the only company you should be considering. Here are a few pointers that

Guy Kroupp

13 DevOps Influencers, Groups and Blogs You Need to Follow

With the DevOps movement making its mark on every major industry, everyone should be paying attention. Whether you are an Operations/DevOps engineer yourself or are head of business, you should

Ariel Assaraf

The dark side of Micro services

If you build it right and not just use it as a buzz word, I believe we can all agree that microservices are great and that they make your code & production

Guy Kroupp

Why are we analyzing only 1% of our logs?

Here at Coralogix, one of the main goals we have is what we call “Make Big Data small” the idea is to allow our users to view a narrow set

Ariel Assaraf

JAVA logging – how to do it right

Proper logging is something you want to plan and standardize before you start writing your code, especially if it involves different teams or separate locations. During the last couple of

Ariel Assaraf

How to leverage your Log Analytics

How to Leverage your Log Analytics from Coralogix Here at Coralogix, we’ve mentioned before how Log Analytics has increasingly been gaining attention and is creating a way to cut back

Ariel Assaraf

The 5 quality pillars any R&D must own

The 5 Quality Pillars Any R&D Must Own from Coralogix In today’s competitive world of software development, one of the key factors for a company’s success is the R&D

Ariel Assaraf

How we process our Big Data using 1.5Kb

Process your Big Data using 1.5 Kb from Coralogix During one of our data-munging sessions here at Coralogix, we found ourselves needing to assess the cardinality of large data sets.

Lior Redlus

These 4 parameters will make or break your E-commerce

During our research here at Coralogix, we discovered the growing need and awareness of Log Analytics as companies strive to control their log data and monitor their application and server

Ariel Assaraf

Our journey from Homeland security to Actionable Log Analytics

5 years ago, when I finished my Army service at the IDF 8200 intelligence unit, it was clear to me that my future is in the world of intelligence and

Ariel Assaraf

Log Analytics will turn that junk on your servers into gold

Log Analytics will turn that junk on your servers into gold from Coralogix Although many tend to think Log Analytics is a new concept, logs were actually born together with

Ariel Assaraf

Delivery vs. Quality – Why not both?

Delivery vs. Quality – Why not both? from Coralogix On our previous post we talked about the huge amount of time that software developers spend on maintenance. In this post

Guy Kroupp

This is what your developers are doing 75% of the time, and this is the cost you pay

This is what your developers are doing 75% of the time, and this is the cost you pay from Coralogix Following our last post on how log analytics and code

Ariel Assaraf

$100M raised | 12 Months | 4 Companies | 1 Surging market

In the world of DevOps and continuous delivery, Log Analytics tools are quickly emerging as a crucial component for an effective strategy towards improved maintenance procedures and support of complex

Ariel Assaraf

The day google went down for 5 minutes

So what are the costs of downtime? [ssf id=298592458] We gathered some researches showing exactly how expensive are downtimes for software and internet companies, these amazing numbers show how dependent

Ariel Assaraf

Get a personalized demo