K6 constant-arrival-rate 1h Test Error

Grafana k6
1

I am running a 1 hour constant-arrival-rate test. My test is supposed to generate 1TB of traffic at the end of the hour. The SUT is an encoder that responses with the encoded data in the response body, which which needs to sum up to 1TB by the end of the hour. 1TB / Hour == 2.2222 Gigabits per second. My scenario is:

let VersionScenarios = {
    scenario: {
        executor: vu_type,
        exec: 'my_request',
        preAllocatedVUs: 1500,
        rate: Math.floor(((1024) / (2.2 * 8)) * 2.2222222 ), // 1 TB per hour ~= 2.22222 Gbps
        duration: '1h',
    },
}

My executor is:

export function my_request () {
    const url = `${__ENV.HOST}/${__ENV.SEGMENT}`
    let response = http.get(url);
    const CheckOutput = check(
        response, {
            "response.status is 200": (response) => response.status === 200,
        }
    )
    if (!CheckOutput) {
        console.log(response.request.url)
        console.log(response.status_text)
    }
    // console.log(url.searchParams); // debug
    // console.log(JSON.stringify(response.headers)); // debug
}

The logs at the beginning of the test say:

running (0h00m58.1s), 1500/1500 VUs, 7496 complete and 0 interrupted iterations
_144p [ 2% ] 0004/1500 VUs 0h00m58.1s/1h0m0s 129 iters/s

Why is the first line saying 1500/1500 VUs, then the 2nd line says 0004/1500? How do I interoperate the logs?

My main problem is: Half way through the test I get the following error:

level=warning msg=“Insufficient VUs, reached 1500 active VUs and cannot initialize more”

And the my test stops doing 2.222 Gigabits per second, and does about 1 Gigabit per second for the rest of the test. I have tried with 8, 16, 100, 200, 500, and 1500 VUs, and every time at about 30 minutes I get this error and the test stops doing 2.2 Gbps and does about 1Gbps.

FWIW I am running k6 in an ECS cluster and have noticed that when doubling the memory of the ECS cluster, I do not have this problem. Why is a constant-arrival-rate test consuming more and more memory as the test runs? Isn’t it constant?

5

Hi @swalker1000, welcome to the community forum,

Why is the first line saying 1500/1500 VUs, then the 2nd line says 0004/1500? How do I interoperate the logs?

The first one is how many VUs are initialized and are max and the second one is how many of the VUs for that scenario are currently in use.

The rate here is how many iterations k6 will start each timeUnit (1s by default). If the rate is 5 for example k6 will start 5 iterations. Each iteration needs a VU to execute though, so you need 1 VU per each currently running iteration. If k6 doesn’t have a VU to start an iteration it can’t and will print:

level=warning msg=“Insufficient VUs, reached 1500 active VUs and cannot initialize more”

With your calculation of rate you have 129 as a rate (which is also what 129 iters/s means ;).

I guess your calculations only work if each request is getting you 2.257 mb if my math is correct. I am not certain that is how you understood what rate is though?

You probably get only 4 VUs as everything works well at the beginning, but maybe the SUT slows down over time and k6 has more and more of the bodies that it downloads in memory. You can probalby find this out by outputing the metrics somewhere and looking at the data such iteration_duration as well as looking at how much memory is used by the ECS.

Given what your test does I would highly recommend setting discardResponseBodies this means that while k6 will still download 2.2Gbps it won’t put them in memory and then have to GC while you never use that data.

This alone will likely solve your problem.

Hope this helps you!