When signing custom plugin I get the error self-signed certificate in chain

Hi,
I am trying to sign a custom plugin using the command npx @grafana/toolkit plugin:sign --rootUrls http://grafana.staged-by-discourse.com, I am behind a corporate proxy that does SSL inspection of traffic. I have tried everything listed below and still keep getting the message “Self-signed certificate in chain” when trying to sign the plugin.

I have done the following so far

  • npm config set constrict-ssl false
  • npm config set cafile /path/to/cert.pem
  • export NODE_EXTRA_CA_CERTS=\bundle.pem
  • set my proxy correctly i.e. proxy and https-proxy for npm
  • npm config set ca “cert string”
  • Tried setting NODE_TLS_REJECT_UNAUTHORIZED=0

I also ran NODE_DEBUG=‘tls,https’ npx @grafana/toolkit plugin:sign --rootUrls http://grafana.staged-by-discourse.com to see if my certificates are being picked up correctly. I am 100% my certificates are correct.

Nothing seems to solve the problem, any suggestions would be highly appreciated

Did you add your corporate certs to CA certs bundle.pem?

Yes I did and they are correct certificates verified that too. Also I cleared my npm cache to make sure there was nothing going on there

Could you check if curl -v.... https://grafana.com is able to create verified tls connection with that bundle.

That is what baffles me that the certificates work using CURL, following is the output

Command - curl -v --cacert bundle.pem https://grafana.com > curl.txt

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0*

Trying 34.120.177.193:443...

* Connected to [grafana.com](http://grafana.com/) (34.120.177.193) port 443 (#0)

* ALPN: offers h2

* ALPN: offers http/1.1

* CAfile: bundle.pem

* CApath: none

} [5 bytes data]

* TLSv1.3 (OUT), TLS handshake, Client hello (1):

} [512 bytes data]

* TLSv1.3 (IN), TLS handshake, Server hello (2):

{ [88 bytes data]

* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):

} [1 bytes data]

* TLSv1.3 (OUT), TLS handshake, Client hello (1):

} [512 bytes data]

* TLSv1.3 (IN), TLS handshake, Server hello (2):

{ [155 bytes data]

* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):

{ [6 bytes data]

* TLSv1.3 (IN), TLS handshake, Certificate (11):

{ [4287 bytes data]

* TLSv1.3 (IN), TLS handshake, CERT verify (15):

{ [264 bytes data]

* TLSv1.3 (IN), TLS handshake, Finished (20):

{ [52 bytes data]

* TLSv1.3 (OUT), TLS handshake, Finished (20):

} [52 bytes data]

* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384

* ALPN: server did not agree on a protocol. Uses default.

* Server certificate:

* subject: C=US; ST=New York; L=New York; O=Raintank Inc.; CN=[grafana.com](http://grafana.com/)

* start date: Jun 27 00:00:00 2023 GMT

* expire date: Jan 3 16:58:01 2024 GMT

* subjectAltName: host "[grafana.com](http://grafana.com/)" matched cert's "[grafana.com](http://grafana.com/)"

* issuer: C=XX; ST=XXX; L=XXXX; O=AAA; OU=BBB; CN=CCC

* SSL certificate verify ok.

} [5 bytes data]

> GET / HTTP/1.1

> Host: [grafana.com](http://grafana.com/)

> User-Agent: curl/7.84.0

> Accept: */*

>

{ [5 bytes data]

* Mark bundle as not supporting multiuse

< HTTP/1.1 200 OK

< Server: nginx/1.23.3

< Date: Sat, 01 Jul 2023 20:41:33 GMT

< Content-Type: text/html

< Transfer-Encoding: chunked

< Vary: Accept-Encoding

< Last-Modified: Sat, 01 Jul 2023 19:21:35 GMT

< Vary: Accept-Encoding

< ETag: W/"64a07cbf-38b63"

< Build: 7ddd35becf6083a0426bdf738d2c62068db9386f

< X-UA-Compatible: IE=Edge,chrome=1

< Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

< Referrer-Policy: origin-when-cross-origin

< Content-Security-Policy: object-src 'none'; script-src 'self' 'unsafe-eval' 'unsafe-inline' [https://go2.grafana.com](https://go2.grafana.com/) [https://cdn.jsdelivr.net](https://cdn.jsdelivr.net/) [https://cdnjs.cloudflare.com](https://cdnjs.cloudflare.com/) [https://unpkg.com](https://unpkg.com/) [https://x.clearbitjs.com](https://x.clearbitjs.com/) [https://app.clearbit.com](https://app.clearbit.com/) .......

< Cache-Control: public, max-age=0, must-revalidate

< X-Frame-Options: DENY

< Access-Control-Allow-Origin: *

< Access-Control-Allow-Methods: GET, OPTIONS

< Access-Control-Allow-Headers: DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range

< Access-Control-Expose-Headers: Content-Length,Content-Range

< x-cache-status: HIT

< X-Cached: HIT

< X-Frame-Options: DENY

< Via: 1.1 google

< Alt-Svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000

<

{ [13831 bytes data]

100 363k 0 363k 0 0 826k 0 --:--:-- --:--:-- --:--:-- 827k

* Connection #0 to host [grafana.com](http://grafana.com/) left intact

Time for source code plugin-tools/packages/sign-plugin/src/utils/manifest.ts at main · grafana/plugin-tools · GitHub
Find if axios supports that custom CA certs.

But I would say it is not worth it. I would sign it outside of corp network or I would execute the same API call as that CLI is doing manually.

Thanks, unfortunately I cannot bypass the proxy and move the code out of corp network. Going the manual route might be the best option, do you know which API calls it is making and the respective payloads? If not, I can dig through the source code to figure things out.

See linked source code.

Thanks for all your help

Did you find a solution? It will be nice if you will post it here for others with the same problem.

Right now, I am trying the manual method which is cumbersome. For a long term solution, the axios pieces will need to be modified. Once I am successful in the manual method I will post the solution here. On a side note, I am doing the following steps

  • npm run dev
  • sign the plugin manually and updating the file dist/MANIFEST.txt
  • npm run build

I see the file MANIFEST.txt being removed and when I restart Grafana it shows the plugin is unsigned, what am I doing wrong here

I don’t have a very elegant manual workaround, I did the following to make things work.

  • run the command npx @grafana/sign-plugin@latest --rootUrls=http://grafana.staged-by-discourse.com
  • Let the command fail and find out the location where the npm module was installed
  • opened the file packages/sign-plugin/src/commands/sign.command.ts in Visual Studio code
  • Set a breakpoint on line 20 i.e. console.log(‘Signing manifest…’);
  • Copy the value of the variable manifest
  • Convert the contents of manifest into proper json
  • Using Postman do an HTTP POST to https://grafana.com/api//plugins/ci/sign , set your Authorization to Bearer Token which is the GRAFANA API KEY, set your JSON payload to the JSON output and make sure Postman does not do SSL verification

Copy the response into dist/MANIFEST.txt file.

Not the best way to do it. I know the exact axios change required, if someone can explain to me how I can run a local copy of the packages then I can test the change

1 Like

Yeah, that’s corporate env. I have similar experience with mimirtool. But I’m lucky, because I’m able to disable corporate VPN client for a moment.

Ref: When signing custom plugin I get the error self-signed certificate in chain · Issue #290 · grafana/plugin-tools · GitHub

Just a FYI, this version does not use axios and works behind a corporate proxy, plugin-tools/ at academo/do-not-use-axios · grafana/plugin-tools · GitHub

1 Like