K6-operator not following relative path

Hello maybe this is not strictly a k6-operator issue, maybe Kubernetes, or my lack of experience. I am following the steps in this blog article - Running distributed k6 tests on Kubernetes but replacing my script and config map. from the output below do I have to use full paths?

kubectl get pods

NAME                            READY   STATUS   RESTARTS   AGE
subs-k8s-lg-initializer-hblqp   0/1     Error    0          19m

kubectl get k6

NAME          STAGE            AGE   TESTRUNID
subs-k8s-lg   initialization   19m   

kubectl describe configmap subs-peak-gaterush-test

Data
====
v1_subs_GET_scenario.js:
----
import { peak_gate_rush as eligibility } from "../v1/GET/subs_accounts_eligibility.js";
import { peak_gate_rush as subsDetails } from "../v1/GET/subs_accounts_subDetails.js";
import { peak_gate_rush as pricing } from "../v1/GET/subs_accounts_pricing.js";

kubectl logs subs-k8s-lg-initializer-hblqp

cwilliams2@us88665a3ea648 test_suites % kubectl logs subs-k8s-lg-initializer-hblqp 
time="2023-05-28T17:48:40Z" level=error msg="GoError: The moduleSpecifier \"../v1/GET/subs_accounts_eligibility.js\" couldn't be found on local disk. Make sure that you've specified the right path to the file. If you're running k6 using the Docker image make sure you have mounted the local directory (-v /local/path/:/inside/docker/path) containing your script and modules so that they're accessible by k6 from inside of the container, see https://k6.io/docs/using-k6/modules#using-local-modules-with-docker.\n\tat go.k6.io/k6/js.(*requireImpl).require-fm (native)\n\tat file:///test/v1_subs_GET_scenario.js:1:0(25)\n" hint="script exception"
cwilliams2@us88665a3ea648 test_suites % 

just for grins I changed the setup to bypass the scenario file where I import functions from but I am using relative paths to other files in every script. Does anyone know how properly use relative paths?

logs subs-k8s-lg-initializer-rc2vf 
time="2023-05-28T18:24:53Z" level=error msg="GoError: The moduleSpecifier \"../../../k6-utils-imported/1.1.0/index.js\" couldn't be found on local disk. Make sure that you've specified the right path to the file. If you're running k6 using the Docker image make sure you have mounted the local directory (-v /local/path/:/inside/docker/path) containing your script and modules so that they're accessible by k6 from inside of the container, see https://k6.io/docs/using-k6/modules#using-local-modules-with-docker.\n\tat go.k6.io/k6/js.(*requireImpl).require-fm (native)\n\tat file:///test/subs_accountId_only.js:10:0(27)\n" hint="script exception"

Hi @PlayStay

To make scripts available to the runner container, you can use a ConfigMap or a LocalFile.

In Running distributed k6 tests on Kubernetes, the example is of a single-file test with ConfigMap. I find Paulā€™s demo helpful for this as well: https://github.com/javaducky/demo-k6-operator/blob/main/README.md#resource-setup. It creates the ConfigMap with several locally available files.

You can also the scripts available via a VolumeClaim, especially for bigger scripts.

From the data you initially shared, are you running multi-file-tests? If that is the case, what mechanism are you using to include the rest of the files: subs_accounts_eligibility.js, subs_accounts_subDetails.js, subs_accounts_pricing.js? Is that mechanism failing for you?

If you can share more details of how you configure the script access we can further help :bowing_woman:

Cheers!

Hi there @eyeveebe thanks for responding! Sorry I didnā€™t describe the entire project itā€™s kinda insane, however the tips youā€™ve provided will help me tinker towards a solution to my path issue.

Hereā€™s the structure of the project is closely mimics the k6.git repo samples folder which was recently renamed to examples. within the old samples folder there was a REST folder with a login retrieve an auth token apply the token to an api requiring that token to authenticate a call etc etcā€¦ I modified that sample to my current managerie Iā€™m about to describe.

Parent folder Subs

  • Contains get_api_token.js file
  • Contains commons.js with shared global variable to export to api test scripts where needed. also options definitions
  • Seed-Data folder - contains data files needed to parameterize API URI and header variables.
  • Pipeline folder - Jenkins declarative pipeline scripts for various environments dev, test, stage, prod
  • readme_dir folder - readme stuff
  • test_suites - scenario test scripts to combine individual api test scripts into scenarioā€™s. as you mentioned multi-file-tests
  • api-test-script-folder containing structure apiver/protocol/script.js ā€“ v1/GET/subs_accounts_xyz.js

the above structure works fine for local use (k6 run) and k6 cloud, namely by using relative path import statements for functions and relative path statements for reading seed data files. for each level by folder where files are required to import or be read from Iā€™m using relative paths so I donā€™t have to know the workspace location when I run the scripts local to any laptop, from jenkins, from k6 cloud, from any ec2 or eks or gke container etcā€¦

However when I use the same approach with k6-operator, I get the path not found error. I figured I had to somehow tell the tool to pass on to the cluster where these files exist, but my lazy persona was hoping that k6-operator would just ā€˜knowā€™ my directory structure and pass along paths, environment variable and other such entry criteria.

Hope this helps clarify how bleak I feel right now assuming I may have to define the structure of my project in the configmap or CR.yml file. For dozens and dozens of projects :sob:

however Iā€™m grateful for the tipsā€¦

best,

oh here is the git commit for the samples name change in case you want to see the OG example that drove me to this madness :).

  • commit 64ede011e6895eeae8f1b2d8ca61075018c0140c - apr 2023

here is the commit for the removal of the REST API folder - mar 2023

  • commit c09919b4961f9d93a32f482089a068d36f667ee2

Hmmm Looks like volumeClaim is what I need but I donā€™t quit understand the documentationā€¦

what is PVC? how to I apply the volumeClaim to my config map? Does the spec belong in the customer resource file with configMap and volumeClaim defined within the spec? Is the test path from the root of the configMap folder or is the path to file where kubectl apply should be executed?

If you have a PVC with the name stress-test-volumeClaim containing your script and any other supporting file(s), you can pass it to the test like this:

spec:
  parallelism: 2
  script:
    volumeClaim:
      name: "stress-test-volumeClaim"
      file: "test.js"

:wave: Actually, the first option to try in such cases is the archive: if your project is not too large, you could just k6 archive it and put the archive.tar file into configmap. It is mentioned in the ā€œmulti-file testsā€ link above.

As for volumeClaim, thatā€™s a totally separate option from configmap. Volumes are Kubernetes feature so Kubernetes docs can be referenced on how to set it up:

Once volume is set up in Kubernetes cluster and project files are copied to it, then it can be specified in K6 yaml file as described in the k6-operator README.

1 Like

thanks @olha use k6 archive would be too easy :). apparently there is a 3M limitā€¦ My archive is 23M. :man_shrugging: oh well. time to dig through more documents.

odd that the archive size isnā€™t a problem for k6 cloud?? Is this a feature difference in OSS?

kubectl create configmap subs-stress-test --from-file archive.tar         
error: failed to create configmap: Request entity too large: limit is 3145728

is there a way to store the archive.tar file in an s3 bucket for instance rather than locally so the configMap wonā€™t barf on size?

Hi @PlayStay,
This is a rather common request :smile: It is not currently supported but thereā€™s an open issue here and corresponding PR. Hopefully, itā€™ll be merged soon.

1 Like

Thanks @olha via k6 community forum I have enough of a demo to show the concept to real propellor heads who maintain our cluster(s). When I figure out how to source a multiple MB configMap Iā€™ll post back here.

Tā€™anks again

2 Likes

how to copy files inside the volumes ?

Hi @patenvy not sure what youā€™re asking. I have not used the Volume Claim technique as yet. Is this what your question refers to?