... | ... | @@ -9,36 +9,9 @@ The purpose of this project is to test various solutions for monitoring of Kuber |
|
|
## Monitoring solutions
|
|
|
- [Prometheus](Prometheus)
|
|
|
- [Kubernetes Dashboard](Kubernetes-Dashboard)
|
|
|
- [Jaeger](Jaeger)
|
|
|
|
|
|
## Jaeger
|
|
|
|
|
|
[Jaeger](https://www.jaegertracing.io) is an open source distributed tracing system used for monitoring micro-service based environments. In k8s cluster we usually have lots of services communicating with each other. When we start experiencing delays or other issues in our applications, it can be hard to find out which service is malfunctioning. That what tracing is for, it allows us to track each request from its origin to finish, how long each service too to respond and even if the service returned an error. Jaeger will collect these traces and allow us to perform operations on them.
|
|
|
|
|
|
#### Instrumentation
|
|
|
|
|
|
Instrumentation means that out application is emitting traces, metrics etc. Jaeger itself doesn't produce the traces, so we need and instrumenting framework. It is [recommended](https://www.jaegertracing.io/docs/1.40/getting-started/#instrumentation) to use [OpenTelemetry](https://opentelemetry.io)(OTel). OTel allows producing and sending of traces to a tracing backend like Jaeger. OTel has support for all of the most popular programming languages. Traces in OTel are build from `spans`, which is a unit of work.
|
|
|
|
|
|
#### Setup
|
|
|
To setup Jeager tracing, we should first install libraries specific to language of our code. So for our python backend we install the packages using pip
|
|
|
```bash
|
|
|
pip install opentelemetry-distro
|
|
|
pip install opentelemetry-exporter-jaeger
|
|
|
```
|
|
|
After installing the necessary libraries, we need to make changes to our code. I decided to implement the tracing on the backend service and one route - /api/host. In reality, tracing should be implemented on all relevant services like frontend, nginx and backend but that would be very time consuming.
|
|
|
The adjusted code can be seen [here](https://gitlab.fi.muni.cz/xbrazda/kubernetes-monitoring/-/blob/main/web-backend/app/host.py)
|
|
|
Following [this](https://opentelemetry-python.readthedocs.io/en/latest/exporter/jaeger/jaeger.html) guide, there are two main changes we need to make.
|
|
|
First, we need our code to produce traces. The way to do this in python is using `with` statements, each `with` block will crate a span, whose lifetime will be ended when the code within the block is done executing. We can also start tracing at the begging of a function using the `@` decorator.
|
|
|
After that, we need to setup Jeager exporter to export the traces emitted by our code to the Jaeger backend. Only changes we need to make to the template code are the host name of our k8s Jaeger service and the name of service.
|
|
|
|
|
|
#### Deployment
|
|
|
|
|
|
[This](https://www.jaegertracing.io/docs/1.40/operator/) guide shows us how to deploy the Jaeger operator to our k8s cluster. Jaeger consist of different parts, such as Collector, Agent, Query. There are 3 strategies to deploying Jaeger: AllInOne, production, and streaming. The AllInOne is the default strategy which will package all the components of Jeager into one image. It is the easiest strategy to sue and recommended for testing purposes. After applying the commands, we need to port-forward the Jaeger service and exposing port `16686`, which will allow us to access the Jager UI.
|
|
|
|
|
|
#### Preview
|
|
|
![image](uploads/98af747326f56e2b65a9edc14631dcbf/image.png)
|
|
|
*Jaeger UI*
|
|
|
![image](uploads/91d0dd381d8a895d918ec689051a0eaf/image.png)
|
|
|
*Details of a trace*
|
|
|
|
|
|
## Elastic Stack
|
|
|
|
... | ... | |