... | ... | @@ -3,46 +3,7 @@ |
|
|
The purpose of this project is to test various solutions for monitoring of Kubernetes(k8s) environment. Inside a k8s cluster we can observe various metrics, for example: interactions between pods inside a cluster, whether pods are up or down, utilization of system resources inside of pods and many more. It is useful to track these metrics to get a sense of overall health of the cluster and prevent failures in advance. We will use `minikube` tool, which creates a local k8s cluster with single node. It is recommended to run this project in VM(virtual machine).
|
|
|
|
|
|
## Web app
|
|
|
|
|
|
If we wish to compare monitoring solutions, we should generate some traffic inside the cluster. I decided to create a simple web application, consisting of `frontend` and `backend`. Frontend is a container based on `node` image using the `Vue.js` framework. Backend is using `flask`, a micro framework for `python`. Both of these building blocks have their k8s pod and service. There is a `nginx` proxy inside of the `frontend` pod, which will redirect URLs starting with `/api` to the `backend` pod, other requests will be served from the `frontend` pod. Api of the `backend` pod is used to collect information and manipulate system resources. There is an issue however, since we are running everything locally using `minikube`, it is impossible to separate resources used by the pod from resources used outside of the k8s cluster.
|
|
|
|
|
|
#### Structure of Web app
|
|
|
|
|
|
- **/** **/host**
|
|
|
- shows information about the container the `backend` app runs on
|
|
|
- **/cpu**
|
|
|
- shows current cpu load, allows creation of cpu stressing porcesses and displays those processes
|
|
|
- **/mem**
|
|
|
- shows total, used, free and available memory and allows creation and diplay of memory stressing processes
|
|
|
|
|
|
#### Api Reference
|
|
|
|
|
|
- host
|
|
|
- GET /api/host - return information about the container the `backend` app runs on
|
|
|
- cpu
|
|
|
- GET /api/cpu - return current cpu load collected by [mpstat](https://linux.die.net/man/1/mpstat) tool. The collection takes 1 second to complete
|
|
|
- POST /api/cpu/create - creates a [stress-ng](https://wiki.ubuntu.com/Kernel/Reference/stress-ng) process that will cause cause the load of cpu to increase by specified amount.
|
|
|
- params:
|
|
|
- load: the amount in % to stress cpu for
|
|
|
- GET /api/cpu/running - returns json with all the currently running `stress-ng` processes that are stressing the cpu
|
|
|
- POST /api/cpu/kill - kill one or all `strss-ng` cpu processes
|
|
|
- parmas:
|
|
|
- pid: pid of process to kill
|
|
|
- all: if all=true then kill all processes
|
|
|
- memory
|
|
|
- GET /api/memory - return total, used, free and avaliable memory collected by [free](https://man7.org/linux/man-pages/man1/free.1.html) tool
|
|
|
- params:
|
|
|
- format - one of ("kb", "mb") - the format in whci hthe statistics should be displayed. Default is "mb"
|
|
|
- POST /api/memory/create - spawns a `stress-ng` process that occupy the specified amount of RAM.
|
|
|
- params:
|
|
|
- size - size of process
|
|
|
- format - one of ("kb, mb") - format to append to size
|
|
|
- GET /api/memory/running - returns json with all the currently running `stress-ng` processes that are stressing memory
|
|
|
- POST /api/memory/kill - kill one or all `strss-ng` memory processes
|
|
|
- params:
|
|
|
- pid: pid of process to kill
|
|
|
- all: if all=true then kill all processes
|
|
|
|
|
|
[Web app](Web-app)
|
|
|
## Prometheus
|
|
|
|
|
|
#### Overview
|
... | ... | |