K6 stopped working on Azure DevOps Release Pipeline

Grafana k6
1

We have an application which we need to performance test before releasing updates into production.
Til “recently”, the performance tests worked as expected.
I’m not exactly sure when it stopped working because we didn’t update the solution for awhile, so there was no need to run the tests.

Locally, I was able to replicate the situation and, as discussed on stackoverflow, I was able to get the tests working either with NODE_OPTIONS=--openssl-legacy-provider or output.hashFunction = 'xxhash64'.

Unfortunately, Azure Pipelines does not support --openssl-legacy-provider (only a small number of NODE_OPTIONS are whitelisted and I’m not sure if output.hashFunction = 'xxhash64' is doing anything remotely as the tests continued to fail.

After modifying my tests to log more information, I now see errors like the following:

2023-04-04T09:10:43.0310873Z time=“2023-04-04T09:10:42Z” level=warning msg=“Request Failed” error=“Post "https://example-test.azurewebsites.net/api/v2.0/Example\”: dial tcp 20.50.2.0:443: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond."

I’m at a loss what else to try as this is working locally and nothing changed from when it was working before.

Here is my test:

import http from 'k6/http';
import {authenticateUsingAzure} from './oauth/azure';
import {check, fail} from 'k6';
import {Options} from 'k6/options';

const queryUrl = `https://${__ENV.SFCACHE_SUBDOMAIN}.azurewebsites.net/api/v2.0/Salesforce`;

const oneWeekAgo = getWeeksAgo(1);
const twoWeeksAgo = getWeeksAgo(2);

function getWeeksAgo(weeksAgo: number) {
	const daysAgo = weeksAgo * 7;
	const date = new Date();
	date.setDate(date.getDate() - daysAgo);
	return date;
}

function toSfdcDate(date: Date) {
	return date.toISOString()
		.split('T')[0];
}

const queryData = {
	sObjectName: 'ServiceAppointment',
	filters: [
		'WorkTypeName__c IN { "Installation IA","Meter Exchange ME", "Installation Preparation IPA"}',
		`TargetStart__c > "${toSfdcDate(twoWeeksAgo)}"`,
		`TargetStart__c < "${toSfdcDate(oneWeekAgo)}"`,
		'Status != "Canceled"'
	],
	joinCriteria: [
		'Case:Case.Id==Case__c',
		'Contact:Contact.Id==ContactId',
		'ServiceResource:ServiceResource.Id==Technician__c',
		'WorkOrder:WorkOrder.Id==WorkOrder__c'
	],
	pageNumber: 1,
	pageSize: 10,
	sortField: 'TargetStart__c',
	sortTransition: 'Ascending',
	includeDeleted: false,
	synchronizeFirst: false,
	pendingPreference: 'BestGuessCurrent'
};
const queryDataString = JSON.stringify(queryData);

export function setup() {
	return authenticateUsingAzure(
		__ENV.AZURE_TENANT_ID,
		__ENV.AZURE_CLIENT_ID,
		__ENV.AZURE_CLIENT_SECRET,
		__ENV.AZURE_SCOPES
	);
}

export const options: Options = {
	vus: 1000,
	duration: '30s',
	thresholds: {
		checks: [{
			threshold: 'rate == 1.00',
			abortOnFail: true
		}],
	}
};

export default (oAuthData: any) => {
	// Arrange
	const params = {
		headers: {
			'Content-Type': 'application/json',
			'Authorization': `Bearer ${oAuthData.access_token}`,
		},
	};

	// Act 
        //(THIS is where the failure is!)
	const response: any = http.post(queryUrl, queryDataString, params);

	// Assert
	if (check(response, {'status is 200': r => r.status === 200})) {

		const responseBody = response?.body;
		try {
			const jsonBody = JSON.parse(responseBody);
			check(jsonBody, {'body is array': body => Array.isArray(body)});
			check(jsonBody, {'body contains 10 results': body => body.length === 10});
		}
		catch {
			const error: string = responseBody.substring(0, responseBody.indexOf('\r'));
			console.error("Error response body:", error);
		}
	}
	else
	{
		const responseBody = response?.body;
		const error: string = responseBody.substring(0, responseBody.indexOf('\r'));
		console.error("Error response body:", error);
		fail('unexpected response');
	}
};

Here is my webconfig.js:

const path = require('path');
const {CleanWebpackPlugin} = require('clean-webpack-plugin');
const CopyPlugin = require('copy-webpack-plugin');
const GlobEntries = require('webpack-glob-entries');
const crypto = require("crypto");

const crypto_orig_createHash = crypto.createHash;
crypto.createHash = algorithm =>
    crypto_orig_createHash(
        algorithm === 'md4'
            ? 'xxhash64'
            : algorithm
    );

module.exports = {
    mode: 'production',
    entry: GlobEntries('./src/*test*.ts'), // Generates multiple entry for each test
    output: {
        path: path.join(__dirname, 'dist'),
        libraryTarget: 'commonjs',
        filename: '[name].js',
        hashFunction: 'xxhash64'
    },
    resolve: {
        extensions: ['.ts', '.js'],
    },
    module: {
        rules: [
            {
                test: /\.ts$/,
                use: 'babel-loader',
                exclude: /node_modules/,
            },
        ],
    },
    target: 'web',
    externals: /^(k6|https?:\/\/)(\/.*)?/,
    // Generate map files for compiled scripts
    devtool: "source-map",
    stats: {
        colors: true,
    },
    plugins: [
        new CleanWebpackPlugin(),
        // Copy assets to the destination folder
        // see `src/post-file-test.ts` for an test example using an asset
        new CopyPlugin({
            patterns: [{
                from: path.resolve(__dirname, 'assets'),
                noErrorOnMissing: true
            }],
        }),
    ],
    optimization: {
        // Don't minimize, as it's not used in the browser
        minimize: false,
    },
};

Helps, here is my azure-pipeline.yml build pipeline.

# ASP.NET Core (.NET Framework)
# Build and test ASP.NET Core projects targeting the full .NET Framework.
# Add steps that publish symbols, save build artifacts, and more:
# https://docs.microsoft.com/azure/devops/pipelines/languages/dotnet-core

trigger:
- master

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

jobs:
# Removed other jobs
- job: buildAndPushK6
  pool:
    vmImage: 'ubuntu-latest'
  steps:    
    - task: NodeTool@0
      displayName: 'Install Node.js'
      inputs:
        versionSpec: 18.x

    - script: yarn install
      workingDirectory: 'SalesforceCache.PerformanceTest'
      displayName: 'yarn install'

    - script: yarn webpack
      workingDirectory: 'SalesforceCache.PerformanceTest'

    - task: PublishBuildArtifacts@1
      displayName: 'publish artifact'
      condition: ne(variables['Build.Reason'], 'PullRequest')
      inputs:
        pathToPublish: SalesforceCache.PerformanceTest/dist
        artifactName: k6_artifact

I’m just using Azure DevOp’s web interface to create the release pipeline, but here are the relevant details:

Agent job:
Display name: Run load test
Agent pool: Azure Pipelines
Agent specification: windows-2022
Parallelism: None
Job cancel timeout: 1

Run k6 test script:
Display name: Run with k6
Test script filename: $(System.DefaultWorkingDirectory)_Examplek6_artifact\example.js
Additional arguments: -e SFCACHE_SUBDOMAIN=$(SFCACHE_SUBDOMAIN) -e AZURE_TENANT_ID=$(AZURE_TENANT_ID) -e AZURE_CLIENT_ID=$(AZURE_CLIENT_ID) -e AZURE_CLIENT_SECRET=$(AZURE_CLIENT_SECRET) -e AZURE_SCOPES=$(AZURE_SCOPES)

2

In case this helps anyone else, I managed to get around the Azure pipeline limitation on NODE_OPTIONS by adding

-e NODE_OPTIONS="--openssl-legacy-provider"

to the Additional arguments

1 Like