How to use data from an json endpoint with SharedArray?

Grafana k6 OSS Support
1

I’m trying to use k6 for the first time, and run some initial tests.

I have a very simple test which sets some session cookies and then hits an endpoint.

The session tokens that are used in the test come from a json endpoint provided by our test system, and these change every time the system is deployed, so the test needs to get them from this endpoint at the point the test runs.

Currently I am making the request to the sessions endpoint in the ‘setup’ function in k6, making them available to my test from there. BUT when I’m running a test with just 200 VUs, k6 is consuming all of the 16GBs of memory on the box before it’s even finished ramping up, and then crashing.

I’ve realised from some googling that this is because every VU gets it’s own copy of the data from setup, rather than this being shared, which I did not expect. And because we are returning 50,000 sessions (where the tests will need to get to eventually), this is using a lot of memory.

I’ve tried to change tact and use the SharedArray. However this only seems to have loading data from files in mind, not endpoints.
When I try to make a http request in the SharedArray callback, I get the error ‘GoError: Making http requests in the init context is not supported’.
When I try to create the SharedArray inside setup, I get ’ GoError: new SharedArray must be called in the init context’

Does anyone know of a way to use these two things together, or an alternative approach that will work?

2

Hi @bee-anchor , welcome to the forum :tada:

I would recommend that you just wget https://yourendpoint and load it from the script. There currently really isn’t another workaround as :

You can’t make requests in the init context and this is unlikely to change as it needs to stay as close to “no-sideffects” as possible, especially as it gets called for every VU and even just to get stuff and compute the options.

SharedArray in setup is likely going to be also not a thing that will be added (at least not a named one) as both now in the cloud as well as in any future OSS distributed model the setup will need to be called once per the whole test and then somehow we will have to move SharedArray around I guess, which seems like a bad idea, especially as setupData (what setup returns) already mostly exist for that.

We have discussed making SharedArray possible in the VU code so you can just on the first iteraton of a VU try to make the request, which should be fairly simple and if you want we are taking PRs :wink:

The other thing that has been discussed is setupData to become immutable and preferably backed by something like SharedArray, which will be a bit more complex as currently setup can return w/e and it will just be marshalled to JSON and w/e survives is what the VUs (and teardown) actually get. So this likely be a lot more involved at least because it will be more tricky :wink:

Hope this helps you