Rotating Indexes in ElasticSearch with Curator

Published on Author gryzli

At certain point in time, you will want to rotate (delete) your old indexes in ElasticSearch. Most of the time people are using time based naming convention for their index names like: index_name-Year-Month-Day or index_name-Year.Month.Day. Tools like Filebeat/Logstash can also use such naming conventions. Before you go and write your own script that will do… Continue reading Rotating Indexes in ElasticSearch with Curator

Working With Ingest Pipelines In ElasticSearch And Filebeat

Published on Author gryzli

What are ingest pipelines and why you need to know about them ? Ingest Pipelines are powerful tool that ElasticSearch gives you in order to pre-process your documents, during the Indexing process. In fact they are integrating pretty much of the Logstash functionality, by giving you the ability to configure grok filters or using different… Continue reading Working With Ingest Pipelines In ElasticSearch And Filebeat

Advanced Filebeat Configuration

Published on Author gryzli

If you have made it through the initial filebeat installation , you may want to do some more interesting stuff with Filebeat. Here I will share some of my experience with it.   Configuring Filebeat To Tail Files This was one of the first things I wanted to make Filebeat do. The idea of ‘tail‘… Continue reading Advanced Filebeat Configuration

Installing And Configuring Filebeat On Centos/RHEL

Published on Author gryzli

Filebeat is a perfect tool for scraping your server logs and shipping them to Logstash or directly to ElasticSeearch. You will find some of my struggles with Filebeat and it’s proper configuration.   Installing Filebeat under Centos/RHEL As with all ELK products the installation process is really easy and straight forward. Filebeat could be easily… Continue reading Installing And Configuring Filebeat On Centos/RHEL

ElasticSearch CheatSheet

Published on Author gryzli

ElasticSearch Example Queries  Searching with wildcard query.   Changing the size of the search result to 100 items. 

Filter aggregations by minimum document count in the result by using min_doc_count setting.

Check cluster health information

Check cluster settings

Check index mappings and settings

Check indexes recovery stats

Check shard… Continue reading ElasticSearch CheatSheet

ElasticSearch Security – Installing And Configuring Search-Guard How-To

Published on Author gryzli

Security is one of the major “missing” things from the free ELK Stack compilation, so let’s talk about achieving it! Soon or later there will come time, when you may want some more “Secure” ElasticSearch cluster, and by “Secure” I mean some of the following : Encrypted communication between cluster nodes Encrypted communication between “Indexing… Continue reading ElasticSearch Security – Installing And Configuring Search-Guard How-To

ElasticSearch Choosing Field Mappings

Published on Author gryzli

If you want to spend good time with ElasticSearch you must choose very carefully your elasticsearch index field mappings. Proper field mappings are extremely important in order to be able to search properly inside your data. Keep in mind that ElasticSearch differs a lot between major versions. The current article is written for the current… Continue reading ElasticSearch Choosing Field Mappings

How ElasticSearch Works (Basic Concepts)

Published on Author gryzli

I’m going to skip the intro about Elasticsearch and it’s primary application (which is for search) and will go straight to the point. If you need some basic understanding of what is Elasticsearch and how to use it, I suggest you to start with the official documentation which is one of the best software documentations… Continue reading How ElasticSearch Works (Basic Concepts)

Kibana Installation Under Centos / RHEL

Published on Author gryzli

Kibana is part of the famous ELK stack (ElasticSearch , Logstash , Kibana) and is best used for Vizualization and Interaction with your ES cluster.   When installing Kibana, it is good to make sure that your Kibana version is coresponding to your ES version. For example if you have installed ES 6.X , it… Continue reading Kibana Installation Under Centos / RHEL

Elasticsearch Installation How-To (Centos 7)

Published on Author gryzli

Elasticsearch install is pretty straight forward (also making cluster of elastic nodes).The following howto is about installing elasticsearch 6.x, which is the current latest version. It is good to know some basic concepts of ElasticSearch before using it.  I’m not going to talk about the hardware requirements, because they strongly depend on the setup and… Continue reading Elasticsearch Installation How-To (Centos 7)