ELK Stack

Elastic Stack is now our primary platform for collecting metrics and logs.

Elastic Stack is a set of open-source tools for centralized collection, storage, analysis, and visualization of logs and data.

ELK stands for

• Elasticsearch (a distributed search and analytics engine built on Apache Lucene)
• Logstash (a pipeline for collecting, transforming, and shipping data),
• Kibana (a web UI for visualizing and analyzing data)

There's also Beats as a separate component, which is what actually pumps the data in — lightweight agents for collecting data at the sources (servers, PCs, IoT devices).

How we use it

We started using it for its "classic" purpose - collecting, aggregating, and browsing logs, with retention afterwards.