Want unique data per vus,and each loop sequentially in loop

Grafana k6 OSS Support
2

Hi there,

for your use case you might want to look at this example instead, which explains how to manually slice the data for each VU.

An example of this that uses Papaparse:

import Papa from './papaparse.js';

const csvFile = open('sample.csv');

export function setup() {
  let results = Papa.parse(csvFile, { header: true });
  return results.data;
}

function getSlice(data, n) {
    let partSize = Math.floor(data.length / n);
    return data.slice(partSize*__VU, partSize*__VU+partSize);
}

export default function (data) {
  let slice = getSlice(data, __ENV.K6_VUS);
  for (let row of slice) {
    // handle each row
  }
}

If you run this with e.g. K6_VUS=50 k6 run -i 1000 script.js then each VU should process only its slice of the data, and will repeat this roughly 20 times (since iterations are shared across VUs). Depending on your dataset you might need to play around with the number of iterations to achieve the correct split you need and to process all data completely, but this should get you on the right track.

Note that the file is only parsed once in setup(), but since the fully parsed results are passed to each VU (i.e. it’s not possible to do this split in setup()), this will require K6_VUS*data amount of memory. There are plans to make this more efficient and easier to do (see #532, #997), but for now this is the recommended approach.

HTH,

Ivan

1 Like