How much does free ELK stack cost you?
- Ariel Assaraf
- March , 2017,03
Free ELK stack (Elasticsearch, Logstash, Kibana) is not as free as it is cracked up to be.
Allow me to explain: Have you ever heard of The Weber-Fechner law?
Strangely enough, the Weber-Fechner theory is responsible for one of the most common mistakes companies make when choosing their log analytics solution.
Generally speaking, this law describes how people perceive change as a percentage of its baseline. By applying this theory to economic decision making, cognitive psychologists Amos Tversky and Daniel Kahneman discovered that people evaluate prices relative to a reference point, thereby making them more sensitive to a new expense rather than adding the same amount to an existing expense (see chart below).
Well, remember those “free” ELK instances you have on your cloud? Their existence may prove to be the best example of the Weber-Fechner theory. These instances end up costing more than they initially appear at face value, however most people tend to consider them free or cheaper than they are, as the price is added to the total amount that is paid to AWS.
That is why just like the chart below, you perceive their price lower than it actually is.
Of course the answer to this question varies, and depends on several aspects like:
We went for the classic case of a mid-size company:
1) 1 Master instance (c4.large, West US, no HA):
$0.124/hour * 720H/month = 89$/month
2) 2 data instances (r4.xlarge) according to ES recommendation + with must have redundancy:
$0.296/hour * 2 * 720 = 426$/month
3) Disk, general purpose SSD (gp2)
$0.12/hour * 50GB/day * 14/days retention * 2 (data redundancy) * 1.2 (recommended extra disk for ES) = 201$/month
Total HW expenses per month: 89$ + 426$ + 201$ = 716$
And now for the cost most companies tend to ignore, despite it being what keeps the company running.
Total estimated price for building your own ELK stack on AWS: 716$ + 110$ + 1590$ = 2416$/month
1) 1 Master instance (c4.large, west US, no HA):
$0.183/hour * 720H/month = 131$/month
2) 2 ES machines (r4.xlarge.elasticsearch)
2 * 0.437$/hour * 720H/month = 629$/month
3) Hard Disk, EBS Standard volumes:
$0.162/hour * 50GB/day * 14/days retention * 2 (data redundancy) * 1.2 (recommended extra disk for ES) = 272$/month
Calculated monthly on a 2 years basis: 44$/month.
Total estimated price for a simple managed ES on AWS with Kibana and Logstash: 1032$ + 574$ = 1606$/month
When you compare these numbers to services which cost about 2500$/month for 50GB/day 14 days retention and offer a fully managed cluster, alerting capabilities, higher availability, better redundancy, auto scaling, and not to mention machine learning capabilities and anomaly detection, it is almost impossible to understand why would anyone choose to set-up his own cluster, isn’t it?
Well, Weber and Fechner may have the answers for this one…Signup to Coralogix