Test a list of URL's - WEB API delete method

Godinho · January 30, 2023, 4:14pm

Hi,

I’ve created dummy dataon a data base, and I’ve generated a file with a list of URLs (arounf 10 milliion).
To be honest all I needed from the file is the ID, and then I could use something like:
http.delete('http://something/pi/employee/'+ ID, null, params);

I’ve check this post: How to load test list of URLs but I don’t need to ramdominze, just share the list to all VUs and maybe use the iterration to get the index file. I’ve tryed this:

import exec from 'k6/execution';
import http from 'k6/http';
import { SharedArray } from 'k6/data';
import { sleep } from 'k6';
import papaparse from 'https://jslib.k6.io/papaparse/5.1.1/index.js';

export let options = {
  insecureSkipTLSVerify: true,
  noConnectionReuse: false,
  stages: [
    { duration: "5m", target: 100 }, // simulate ramp-up of traffic from 1 to 100 users over 5 minutes
    { duration: "10m", target: 100 }, // stay at 100 users for 10 minutes
    { duration: "5m", target: 0 }, // ramp-down to 0 users
  ],
};

const sharedData = new SharedArray("URLs", function () {
    let data = papaparse.parse(open('net.list'), { header: true }).data;
    return data;
});


export default function () {
  
    const params = {
        headers: { 'Content-Type': 'application/json', },
    };
    const url = 'http://10.47.2.220/java/api/employee/' + data[exec.scenario.iterationInTest];

    http.delete(url, null, params);
    sleep(1);
}

But the K6 exists with Killed… Maybe the URLs file it too big, it’s 389M. The list:

root@openvpn:~/K6# head -n10 net.list
url
http://10.47.2.220/net/api/employee/44
http://10.47.2.220/net/api/employee/50
http://10.47.2.220/net/api/employee/56
http://10.47.2.220/net/api/employee/62
http://10.47.2.220/net/api/employee/68
http://10.47.2.220/net/api/employee/71
http://10.47.2.220/net/api/employee/74
http://10.47.2.220/net/api/employee/80
http://10.47.2.220/net/api/employee/83

Can anyone help me?

Thank you.

eyeveebee · February 1, 2023, 9:08am

Hi @Godinho

Welcome to the community forums and thanks for raising the issue.

I see in the docs k6 shared array is tested up to 100k lines, not 10 million. More details on this gist. I will try to test locally with a bigger array, to see if I can reproduce this.

What do you have in the file, the full URL or just the id? Based on const url = 'http://10.47.2.220/java/api/employee/' + data[exec.scenario.iterationInTest]; it seems just the id, but the head -n10 net.list shows the full URL. I don’t expect this to be an issue, just would make the deletion fail, but the test should still run.

From your side, maybe you can try a few things to see where the limit is:

Do you see the OS hit any resource limits (memory, CPU,…) when k6 crashes?
- We have some guides around tuning the load generator if this was the case: Running large tests and Fine tuning OS.
Is this related to the scenario, or when running the VU code?
- Does this work with a basic test, e.g. 10 VUs, and just few iterations? It can help determine where we are hitting the limit. If just loading the shared array fails.
What do you see in the test output? Is the test running past the init phase and already running iterations when it fails? Any errors/warnings in the logs before it crashes?

I’d appreciate if you can share your findings on the above. And I’ll let you know how it goes on my laptop with 10 million data lines using the benchmark.

Cheers!

eyeveebee · February 1, 2023, 5:31pm

Hi @Godinho

I tested with the following script, creating a 10M data set:

import { check } from "k6";
import http from "k6/http";
import { SharedArray } from "k6/data"

export const options = {
    insecureSkipTLSVerify: true,
    noConnectionReuse: false,
    vus: 1,
    duration: "10s",
}

const n = 10000000;
function generateArray() {
    var arr = new Array(n);
    for (var i = 0; i < n; i++) {
        arr[i] = { "something": "something else" + i, "password": "12314561" };
    }
    return arr;
}

const data = new SharedArray("my data", generateArray);

export default function () {
    var iterationData = data[Math.floor(Math.random() * data.length)];
    //console.log("iterationData: ", iterationData);
    var res = http.post("https://httpbin.test.k6.io/anything", JSON.stringify(iterationData), { headers: { "Content-type": "application/json" } })
    check(res, { "status 200": (r) => r.status === 200 })

}

And I did not see any issues related to the SharedArray. Not reading this from a file, though I would not expect this to be too different.

          /\      |‾‾| /‾‾/   /‾‾/   
     /\  /  \     |  |/  /   /  /    
    /  \/    \    |     (   /   ‾‾\  
   /          \   |  |\  \ |  (‾)  | 
  / __________ \  |__| \__\ \_____/ .io

  execution: local
     script: shared-array.js
     output: -

  scenarios: (100.00%) 1 scenario, 1 max VUs, 40s max duration (incl. graceful stop):
           * default: 1 looping VUs for 10s (gracefulStop: 30s)


running (10.1s), 0/1 VUs, 92 complete and 0 interrupted iterations
default ✓ [======================================] 1 VUs  10s

     ✓ status 200

     checks.........................: 100.00% ✓ 92       ✗ 0  
     data_received..................: 78 kB   7.7 kB/s
     data_sent......................: 21 kB   2.1 kB/s
     http_req_blocked...............: avg=2.76ms   min=2µs      med=7µs      max=253.65ms p(90)=12µs     p(95)=14µs    
     http_req_connecting............: avg=1.12ms   min=0s       med=0s       max=103.65ms p(90)=0s       p(95)=0s      
     http_req_duration..............: avg=106.55ms min=103.31ms med=106.36ms max=114.74ms p(90)=107.73ms p(95)=107.97ms
       { expected_response:true }...: avg=106.55ms min=103.31ms med=106.36ms max=114.74ms p(90)=107.73ms p(95)=107.97ms
     http_req_failed................: 0.00%   ✓ 0        ✗ 92 
     http_req_receiving.............: avg=139.39µs min=38µs     med=137.5µs  max=267µs    p(90)=222.7µs  p(95)=245.89µs
     http_req_sending...............: avg=43.19µs  min=12µs     med=42.49µs  max=124µs    p(90)=70.7µs   p(95)=75.44µs 
     http_req_tls_handshaking.......: avg=1.13ms   min=0s       med=0s       max=104.27ms p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=106.37ms min=103ms    med=106.15ms max=114.67ms p(90)=107.55ms p(95)=107.81ms
     http_reqs......................: 92      9.106895/s
     iteration_duration.............: avg=109.77ms min=103.78ms med=106.88ms max=362ms    p(90)=108.11ms p(95)=109.37ms
     iterations.....................: 92      9.106895/s
     vus............................: 1       min=1      max=1
     vus_max........................: 1       min=1      max=1

And the memory usage was stable during the test on my laptop:

I also tested with similar configuration as you:

export let options = {
  insecureSkipTLSVerify: true,
  noConnectionReuse: false,
  stages: [
    { duration: "5m", target: 100 }, // simulate ramp-up of traffic from 1 to 100 users over 5 minutes
    { duration: "10m", target: 100 }, // stay at 100 users for 10 minutes
    { duration: "5m", target: 0 }, // ramp-down to 0 users
  ],
};

In my test the tests mostly fail as the test API cannot stand this load, though the k6 process does

Memory:

CPU:

Are you able to pinpoint what might be different in your case? Can you provide a bit more context (see my previous message), to try and figure out where the test fails for you?

Cheers!