K6 unstable when running via Azure Pipelines

Grafana k6 OSS Support
1

Hello :slight_smile:

I am busy implementing k6 on our CI/CD pipeline.
We have set up a VM to run all k6 tests.

Currently, I have 2 tests active: Smoke and a Load test. Both utilize the code below:

import { check, group } from 'k6';
import http from 'k6/http';

const user_data = JSON.parse(open(`${__ENV.USERS_FOR_VU}`));

// Test configuration
export let options = JSON.parse(open(`${__ENV.TEST_CONFIG}`));

// User scenario
export default function () {
    group('Init Auth Call to Sign In page', function () {
        let spa_response;

        spa_response = http.get(`${__ENV.TEST_HOSTNAME}`);
        check(spa_response, {
            'is app status 200': (r) => r.status === 200
        });

        group('Sign In to ids4 and inspect user for sign in access', function () {
            let userNumber = Math.floor((Math.random() * (user_data.users.length)));
            let vu_user = user_data.users[userNumber];
            spa_response = spa_response.submitForm({
                formSelector: 'form',
                fields: {
                    Username: vu_user.username,
                    Password: vu_user.userpassword,
                    RememberLogin: false
                }
            });
            check(spa_response, {
                'is login request status 200': (r) => r.status === 200
            });
        });
    });
}

However when running the test via PS on the VM via the pipeline I get the following error for every iteration:
time="2021-03-26T15:39:24Z" level=warning msg="Request Failed" error="Get \"\": unsupported protocol scheme \"\"" time="2021-03-26T15:39:24Z" level=error msg="GoError: invalid response type\n\tat reflect.methodValueCall (native)\n\tat file:///C:/azagent/A2/_work/3/s/src/test-scripts/ci-load-smoke-test-v1.js:27:36(36)\n\tat github.com/loadimpact/k6/js/common.Bind.func1 (native)\n\tat file:///C:/azagent/A2/_work/3/s/src/test-scripts/ci-load-smoke-test-v1.js:19:23(29)\n\tat github.com/loadimpact/k6/js/common.Bind.func1 (native)\n\tat file:///C:/azagent/A2/_work/3/s/src/test-scripts/ci-load-smoke-test-v1.js:11:19(5)\n" executor=ramping-vus scenario=default source=stacktrace

However when running the above directly on the VM again using PS the test runs and completes as expected.

Has anybody seen something similar?

I Should mention the VM is on Azure as is our CI/CD pipeline (Azure DevOps).

2

Hello!

"Get \"\" would suggest that __ENV.TEST_HOSTNAME is empty - are you sure it is being set somewhere?

3

Hi Tom!
Thanks very much for your reply.

You pointed the problem out right there. Way up the line in helm templates task where the environment gets set dynamically was an error in a PS function that setup the environment.