Custom Counter and Rate for k6chaijs expect assertions

Grafana k6 OSS Support
1

Hi!

I have read from the docs that k6chaijs has better error handling than checks so I’m using describe - expect in my k6 frontend tests.

What would be a proper way to create a custom metric for failing expects, count of total tests executed and use this as threshold (to make sure that expects were actually executed), just like the check would? Thresholds

In this sample snippet, I would need to place this inside a try-catch block because otherwise, I would get

ERRO[0015] Uncaught (in promise) Error
running at ue (https://jslib.k6.io/k6chaijs/4.3.4.3/index.js:2:1165(57))
frontendat https://jslib.k6.io/k6chaijs/4.3.4.3/index.js:4:55024(125)

That also means that the catch block becomes the catch all for any error, like navigation timeout, in theory.

  describe('Sample describe block', async () => {
    try {
      expect(1, `1`).to.equal(2);
    } catch (error) {
      console.error(error);
    } finally {
// this is run on k6 browser 
      await page.close();
      await browser.close();
    }
  });

I would like to use these in my thresholds and define whether or not to abort the run if the expects have been failing.

Thanks in advance for the guidance!

5

Hi there!

Within your catch block you can determine whether the thrown object is an assertion error, and if so, change the value that is added to the metric.

Something like this:

import { chai, describe, expect } from 'https://jslib.k6.io/k6chaijs/4.3.4.0/index.js';
import { Rate } from 'k6/metrics';

const someAssertionMetric = new Rate('someAssertion');

export let options = {
  thresholds: {
    someAssertion: [
      { threshold: 'rate > 0.9', abortOnFail: true, delayAbortEval: '10s' },
    ],
  },
};

export default function testSuite() {
  describe('Dummy example', () => {
    let assertionPassed = true;
    try {
      expect(Math.floor(Math.random () * 5) + 1).to.equal(1);
    } catch (e) {
      if (e instanceof chai.AssertionError) {
        console.log(e.message); // e.g. "expected 4 to equal 1"
        assertionPassed = false;
      }
    }
    someAssertionMetric.add(assertionPassed);
  });
}

You can use the metric in a threshold just like any other metric.

Hope this helps!

2 Likes
7

thanks for this @imiric ! apologies for the delayed response as i have been working on a different project recently. this helps!