... | @@ -132,6 +132,9 @@ Elastic stack (ELK stack) is a group of products from [Elastic](https://www.elas |
... | @@ -132,6 +132,9 @@ Elastic stack (ELK stack) is a group of products from [Elastic](https://www.elas |
|
- **Kibana** is a data visualization tool that works on top of Elasticsearch
|
|
- **Kibana** is a data visualization tool that works on top of Elasticsearch
|
|
- **Beats** are agents that collects various metrics and send them to other destinations such as Logstash. Some of the most commonly used Beats are Filebeat, Metricbeat or Packetbeat.
|
|
- **Beats** are agents that collects various metrics and send them to other destinations such as Logstash. Some of the most commonly used Beats are Filebeat, Metricbeat or Packetbeat.
|
|
|
|
|
|
## Configuration
|
|
#### Configuration
|
|
|
|
|
|
[Elastic Cloud on Kubernetes(ECK)](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-overview.html) is the recommended way of deploying Elastic stack on k8s. There are individual Helm charts for each component of the stack but these will be deprecated soon. We first create all the necessary CRDs(Custom Resource Definition) such as Beat, Kibana etc. and then install the elastic operator. By default this will be installed in the `elastic-system` namespace. |
|
[Elastic Cloud on Kubernetes(ECK)](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-overview.html) is the recommended way of deploying Elastic stack on k8s. There are individual Helm charts for each component of the stack but these will be deprecated soon. We first create all the necessary CRDs(Custom Resource Definition) such as Beat, Kibana etc. and then install the elastic operator. By default this will be installed in the `elastic-system` namespace.
|
|
\ No newline at end of file |
|
We should start by installing `Kibana` and `Elasticsearch` resources as these require little to none configuration, we only need to set the `elasticsearchRef` attribute correctly in the Kibana deployment.
|
|
|
|
Next we will deploy Filebeat, which collects log from logfiles. We configure it to look at files at `/var/lib/containers/` directory which is where k8s stores logs from its containers and then we send the output to Logstash service on port 5044. Since Filebeat needs to read files on all pods across all nodes on our cluster, we need it give it elevated privileges using `ClusterRole` and `ClusterRoleBinding`.
|
|
|
|
Lastly, we need to configure Logstash. Unfortunately, Logstash is not part of the ECK platform, which complicates its deployment slightly, but there is an official [recipe](https://github.com/elastic/cloud-on-k8s/tree/main/config/recipes/logstash) to include it in ECK. Logstash pipeline is used to transform logs using [grok](https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html) filter. |