Report for functional test

Grafana k6 OSS Support
1

@mstoykov or other experts,
I am using k6 scripts to do functional api test, not performance test. By reading reporting documents, it seems to me that all the plugs-ins k6 can use are for performance statistics only. Is it the correct assumption? I’d like to have a report that shows which test case pass or fail and where it fails.
Thanks.

2

Hey @minwu

thanks for posting the question.

It’s true that we have built k6 primarily for load/performance testing, but you can easily use it for functional testing or monitoring, as many users do. You can use the check API with a small modification to execute functional “unit tests”. See the example script below.

import http from 'k6/http'
import { Counter } from 'k6/metrics'
import { sleep, check as loadTestingCheck } from "k6";

let failedTestCases = new Counter('failedTestCases');

export let options = {
  thresholds: {
    failedTestCases: [{ threshold: 'count==0' }], // add "abortOnFail: true" to exit immediately
  },
  iterations: 1, // for local testing
  // vus: 1,  // for `k6 cloud` testing
  // duration: '30s',
}

// Functional check (like assert). 
let check = function (obj, conditionArray, tags) {
  let result = loadTestingCheck(obj, conditionArray, tags || {});
  failedTestCases.add(!result);
  return result;
}

export default function () {
  let res = http.get('https://test-api.loadimpact.com/public/crocodiles/')

  check(res.status, {
    "API is working": (status) => status === 200
  });

  check(res.json().length, {
    "got 8 crocodiles": (number_of_objects) => number_of_objects === 8
  });

  check(res.json("0.name"), {
    "First crocodile is Bert": (name) => name === "Bert"
  });

  check(res.json("1.name"), {
    "Second crocodile is Pepe": (name) => name === "Pepe"
  });
}

Here’s a sample execution with k6 cloud
https://app.k6.io/runs/public/776c9487ee3b4acbafa8693ffc7a612d?tab=checks

And here’s a sample execution with k6 run

image
image1044×758 73.8 KB

3 Likes
3

@pawel,
Thank you very much for providing the detailed code. The graphic report on the checks is just what I need. However, I wonder if any of the plug-ins can also visualize the checks on local test? I read the documents about InfluxDB and grafana. Is grafana able to have a customer dashboard that shows the pass and fails of checks?

4

I remember that we have had some issues with getting check results in the default influxdb connector, but I think you can try our experimental timescaledb connector posted here: GitHub - grafana/k6-timescaledb-stack: Load testing stack - k6, TimescaleDB and Grafana
it has nice visualization for checks and other metrics in grafana:

image
image1563×330 45.5 KB

5

@pawel,
Thanks a lot. To take it to the next level, if running different test cases sequentially, What does the report look like?

6

@minwu give it a shot, it should take only ~5min to set it up :slight_smile:

7

@pawel, thanks for your suggestion

8

Hello @minwu, also you can use k6-expect npm package for simplifying assert definitions.

Sample output:

     █ User check
       ✓ https://jsonplaceholder.typicode.com/users/10 is 200
       ✓ https://jsonplaceholder.typicode.com/users/10 responded with valid json
       ✓ Id is 10
       ✓ Name is not empty
       ✓ Phone number matches '\d{3}-\d{3}-\d{4}' pattern
       ✓ Geolocation is less than 0
       ✓ Company contains 'LLC'