Turning data into understandable insights with K6 load testing

Rody Bothe

·

Follow

6 min read

·

Jan 10, 2022

--

1

Listen

Share

K6 makes performance testing easy with Prometheus and Grafana in Docker.

K6 as performance testing tool has improved the lives of many QA’s for the better. Initially Jmeter was the tool to go with, however in recent years the meteoric rise of K6 has taken performance testing to the next level and we are loving it!

We, as QA, are continuously executing tests on our product application. Anyone in the company should be able to read, and quickly understand the found test results/data in a simple overview. The problem should not be, trying to understand all sorts of numbers or having to manually visualise results in an Excel file and share it with stakeholders. A dashboard provides the perfect visualisation to give insights.

We want: continuous monitoring available at all times and to everyone!

In this blog, we are going to do just that:

• Write a basic K6 performance test for demo purposes
• Use Prometheus as an output in K6
• Use Grafana to visualise that output coming from K6
• Using Docker to setup and run K6, Prometheus & Grafana

Initially K6 did not support Prometheus as a data source, they recently introduced the possibility to use it, but we would need to build a new binary that supports that, called XK6 remote write. We want to utilise Docker for the entire setup, which means we need to do some things different. So let’s get into it right away!

Setup & Preparation

We need to clone the repository from Github

Once cloned, we navigate to the directory in the terminal and we create a new folder called : “tests”. In that folder we will create a new file “basic_k6_test.js” as can be seen in the example, the test will just send a GET request to an endpoint, and check if there is a 200 status code in the response. (the test is just for demoing purpose)

Basic K6 test

import http from 'k6/http';
import { describe } from 'https://jslib.k6.io/expect/0.0.4/index.js';
import { check, sleep } from 'k6';export default function () {
    describe('Basic k6 test', () => {        let res = http.get('https://test.k6.io');
        check(res, {
            'is status 200': (r) => r.status === 200
        });
        sleep(1);
    });
}

Now in the root folder of the cloned repository we are going create a new Dockerfile. The Dockerfile will build the new XK6 binary that allows us to communicate with Prometheus.

Dockerfile

FROM golang:1.17-alpine as builder
WORKDIR "$GOPATH"/src/go.k6.io/k6
COPY . .RUN apk --no-cache add git &&\
    go install -trimpath go.k6.io/xk6/cmd/xk6@latest &&\
    xk6 build --with github.com/grafana/xk6-output-prometheus-remote@latest &&\
    cp k6 "$GOPATH"/bin/k6FROM alpine:3.13
RUN apk add --no-cache ca-certificates && \
    adduser -D -u 12345 -g 12345 k6
COPY --from=builder /go/bin/k6 /usr/bin/k6USER 12345
ENTRYPOINT ["k6"]
CMD [""]

Because we want to run multiple services together, we should write a docker-compose file, that will just do that. We will define three services: K6, Prometheus & Grafana.

docker-compose.yml

version: '3.7'volumes:
  prometheus-data:
  grafana-data-storage:services:
  k6-prom:
    build: .
    depends_on:
      - grafana
    volumes:
      - .:/app:delegated
    environment:
      - K6_PROMETHEUS_RW_SERVER_URL=http://prometheus:9090/api/v1/write
    command: run --vus 150 --duration 30s -o xk6-prometheus-rw /app/tests/basic_k6_test.js  prometheus:
    image: prom/prometheus:latest
    depends_on:
      - grafana
    ports:
      - 9000:9090
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus-data:/prometheus
    command: --enable-feature=remote-write-receiver --config.file=/etc/prometheus/prometheus.yml  grafana:
    image: grafana/grafana:latest
    volumes:
      - grafana-data-storage:/var/lib/grafana
    ports:
      - 4000:4000
    environment:
      - GF_SERVER_HTTP_PORT=4000

K6 will have as build context the Dockerfile in the root, hence the .

The environment variable K6_PROMETHEUS_RW_SERVER_URL will allow K6 to use Prometheus as an output and send the retrieved data from the test to the Prometheus endpoint. In this case, the command will actually run the K6 test with 150 users for a duration of 30 seconds.

Prometheus server is the second service, it will use its own image. The anonymous volume will look in the root folder for a file called “prometheus.yml” and bind it to the “etc/prometheus/” folder in the docker container.

The third service we are running is Grafana. In the example, the port has been changed to 4000:4000, but this is just specifying the port for Docker. GF_SERVER_HTTP_PORT=4000 is telling Grafana that it needs to use that port specified by Docker. We also use a named volume here to be able to restart the container and have all data/dashboards still remaining here.

So far we have only defined the services in a docker-compose.yml and a Dockerfile to build the binary, but in order for this to work we also need a prometheus.yml

prometheus.yml

global:
  scrape_interval: 30s
  scrape_timeout: 10s
scrape_configs:
  - job_name: services
    metrics_path: /metrics
    static_configs:
      - targets:
          - 'prometheus:9090'

Initialise the fun!

The docker-compose file is not yet up and running. So let’s see if the preparation work was successful and test the connection between K6 & Prometheus. Open the terminal again and navigate to where the original Github repository was cloned and type:

docker-compose up

This can take a bit of time initially, as it needs to build the image, start the services and run the tests. If everything is working correctly, a screen like this should be seen.

If you go to localhost:9000 (which refers to Prometheus) and type k6_ you should see a menu with the requests that are coming in from K6 into Prometheus. That should look like what you can see below.

So now that we see that the connection works between K6 and Prometheus, we want to port that data into a visual representation in Grafana.

Grafana

You can access Grafana by going to localhost:4000. Initially, it will ask you to login, the username & password are both admin. Because we mapped the named volume, you will only need to do this once. When you are in the Grafana main menu, go to Configuration > Data source > add data source and find Prometheus.

Once Prometheus is selected, you will need to add more data to the data source. Use http://prometheus:9090 as URL, this is defined in the prometheus.yml

Then at the bottom you can see if the data source works by clicking “Save and test”

Dashboards

So everything works, great! Now we want to actually start and visualise the data from K6 & Prometheus in Grafana. K6 has a great community and there are some template dashboards that you could use if you would want. In our case we create a new dashboard and then use the one seen as a panel.

Now the only thing remaining is sharing this data with your team and other stakeholders for a birds-eye view.

Summary

By introducing these simple workflow steps in your organisation, you can visualise data, which can help to show the actual live status of your application to any stakeholder at all times. Data needs to be easily accessible, understandable, easy to maintain and give insights to any stakeholder within your organisation. This, will just do that!

The current approach has been done locally, however this can be easily transferred to an ecosystem within any organisation and any domain.

“You can’t improve what you don’t measure”

Hope you enjoyed the blog for any questions, feel free to reach out and thanks a lot Oleg Bespalov

Kind regards,
Rody Bothe, Staff QA Automation Engineer at Bynder