Skip to main content

Kubernetes

What you’ll learn#

  • What is Kubernetes
  • Why we use Kubernetes
  • What are the recommendation on resources requirements

Canarytrace is designed for use in Kubernetes and this has many advantages:

  • Canarytrace isn't testing framework, but complete test stack with additional components.
  • Designed for pattern 1:1:1 (= 1 monitor script, 1 Canarytrace runner, 1 instance of a browser) - due to strict isolation and predictable resource allocation for each run, the results are credible and comparable.
  • Many tasks are delegated to a lower level, on Kubernetes. It's a better approach than to solve all tasks on the testing framework level. Each Canarytrace component has its own responsibility. E.g.
    • Canarytrace runner (Webdriver.io + services) only loads, runs of test scripts and live reporting.
    • Elasticsearch stack for storing all data from Canarytrace runner and such as engine for agregate data for test report, backups, trends etc.
    • Beats are small datashippers and we use Beats for collect all logs from our docker containers.
    • Kibana for data analysis and visualizations from Canarytrace runner, for the preparation of a test report in many forms e.g. report for testers, architect, devops or test manager.
    • Canarytrace Listener for automatically analyzing many types of data scanned from a browser, alerting by thresholds to many services such as slack or email, simplification of the test result for integration with other tools such as Zabbix.
    • Kubernetes parallelizes, guarantees, manages and isolates each instance of the Canarytrace runner. Every instance of Canarytrace starts on the same conditions. This is an advantage because the tests don't affect each other.
    • We no longer need tools like Jenkins or Selenium cluster.
    • Thanks to Kubernetes you are not vendor locked and at the same time Canarytrace can be run on AWS, DigitalOcean, Azure Cloud, GCE or on your VPS.

Required resources for one instance#

Requirements on resource will be higher if you will be perform a performance audit. Without performance audit will be perform availability check and download information about network trafic in a browser.

Recommended requirements for performance audit#

  • Loading web pages into a modern browser is not an easy task. The measurement results may be skewed by insufficient resources and therefore the following settings are recommended for performance audit.
  • Minimum 2 dedicated cores (4 recommended)
  • Minimum 2GB RAM (4-8GB recommended)
ResourcesCPU requestsCPU limitsMemory requestsMemory limits
Canarytrace Runner200m800m300M1600M
Browser2000m4000m2000M4000M
Total2200m4800m2300M5600M

How to get a deployment scripts#

All deployment scripts are distributed with Canarytrace docker images.

# Download deployments scripts from docker image
docker run --rm -it --entrypoint /bin/mv -v $(pwd):/deployments quay.io/canarytrace/canarytrace-pub:4.21.6-pro-20220222144833-79 /opt/canary/deployments/ /deployments/
# deployments folder is transferred from the docker image to localhost
ᐰ ls -lah deployments/
drwxr-xr-x@ 7 rdpanek staff 224B 7 kvě 23:55 .
drwxr-xr-x 4 rdpanek staff 128B 13 kvě 17:50 ..
-rw-r--r-- 1 rdpanek staff 185B 7 kvě 23:55 secret.yaml
-rw-r--r-- 1 rdpanek staff 2,8K 7 kvě 23:55 smoke-desktop-shipper.yaml
-rw-r--r-- 1 rdpanek staff 2,6K 7 kvě 23:55 smoke-desktop.yaml
-rw-r--r-- 1 rdpanek staff 2,3K 7 kvě 23:55 smoke-mobile.yaml
-rw-r--r-- 1 rdpanek staff 2,7K 7 kvě 23:55 user-journey.yaml

  • secret.yaml is example and contains credentials and endpoint for Elasticsearch and license (key:value). You can use together with smoke-desktop.yaml

  • smoke-desktop-shipper.yaml is full example. Shipper send static files into AWS S3. Read more

  • smoke-desktop.yaml Canarytrace CronJob with desktop settings in smoke mode.

  • smoke-mobile.yaml Canarytrace CronJob with mobile settings in smoke mode.

  • user-journey.yaml Canarytrace CronJob in user-journey mode. More info

How to deploy Canarytrace#

Smoke mode#

Canarytrace in mode smoke is maintenance free approach. You just enter list of landing pages and deploy into your Kubernetes. Each landing page must be separated by a semicolon.

  1. Open and edit smoke-desktop.yaml
  2. Enter path for your docker image e.g. image: quay.io/canarytrace/canarytrace-pub:4.21.6-pro-20220222144833-79

Docker image#

Docker images in this documentation will not work - you must have a docker image with Canarytrace from us with licence. Contact us to obtain a docker image and license.

  1. Edit landing pages ( max. 5 URLs) BASE_URL
  2. Enter your license LICENSE
  3. Setup Elasticsearch cluster path ELASTIC_CLUSTER and credentials ELASTIC_HTTP_AUTH
  4. Deploy into your Kubernetes cluster kubectl -n canarytrace create -f smoke-desktop.yaml
List of running Canarytrace
kubectl --kubeconfig=/Users/rdpanek/Downloads/kubeconfig.yaml -n canarytrace get pods
# output
NAME READY STATUS RESTARTS AGE
canarytrace-27426165-6kbt2 2/2 Running 0 89s

How to edit list of landing pages

  1. Get list of CronJobs
kubectl --kubeconfig=~/canary/xxx.kubeconfig.yaml get cronjobs
# output
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
canarytrace-xxx-1-desktop */5 * * * * False 1 66s 17d
canarytrace-xxx-1-mobile */5 * * * * False 1 66s 17d
  1. Edit CronJob and scroll down to BASE_URL
kubectl --kubeconfig=~/canary/xxx.kubeconfig.yaml edit cronjob canarytrace-xxx-1-desktop
# output
27 containers:
28 - env:
29 - name: BASE_URL
30 value: https://canarytrace.com/;https://www.teststack.cz/
  1. Edit collection of landing pages in BASE_URL and save changes

You can display your Canarytrace instances

  • kubectl --kubeconfig=~/canary/xxx.kubeconfig.yaml get pods

Filebeat#

Filebeat logging all stdout and stderr streams from all Canarytrace docker containers in your cluster.

  • All data from Filebeat are stored to filebeat-* index

Manually debuging Canarytrace#

For manually debuging Canarytrace runner and other components in a docker containers - you can use command line tool kubectl for tail stdout and stderr streams from all containers in your Kubernetes cluster or Lens

Get logs from Canarytrace pod from your localhost
# tesla-1597447320-2gg4r is name of your pod
kubectl -n canary logs -f tesla-1597447320-2gg4r -c canary

This is easy for learning and maintanance Canarytrace containers, but not but not effective.

Filebeat provides#

  • Show all logs in a Kibana in real time - access to logos without knowledge with kubectl tool
  • Searching events, errors and other strings in a streams
  • Filtering by name of docker container
  • Create visualizations from events
  • Create alerting from events in a log by conditions

How to run Filebeat#

Download Filebeat Kubernetes deployment
curl -L -O https://raw.githubusercontent.com/elastic/beats/8.2/deploy/kubernetes/filebeat-kubernetes.yaml
  • Edit ELASTICSEARCH_HOST and next environemnt variables.
  • Deploy kubectl create -f filebeat-kubernetes.yaml

You can check filebeat logs, that it doesn't contain any error messages

kubectl -n kube-system get pods
// outpu
...
filebeat-cznq9 1/1 Running 0 9h
kube-proxy-gjp6p 1/1 Running 0 12d

Get logs from container with filebeat

kubectl -n kube-system logs -f filebeat-cznq9

Filebeat data browsing#

  • Open your Kibana > Discover and index filebeat-*

Filebeat logs in a Kibana

Search input

  • kubernetes.container.name : canary display stdout and stderr streams from Canarytrace runner docker container.
  • message: "testStepStart: HomePage open" for display all record which contains this string.

Filebeat log viewer#

  • Kibana contains a built-in log viewer
  • Open your Kibana > Observability > Logs

Filebeat log viewer

Edit this page