[ui5/cli] Failed to download ui5 packages when working behind proxy.

Question

[ui5/cli] Failed to download ui5 packages when working behind proxy.

zouyi100 opened this issue 2 years ago · 16 comments

Expected Behavior

Download packages successfully.

Current Behavior

Now i'm working behind a http proxy in company. i have a .npmrc file to set the https-proxy and proxy.

In project level, i can run npm i and download the package successfully. But when i run npm run start -> ui5 serve -o index-dev.html --verbose, it will be failed when auto download the ui5 dependencies defined in ui5.yaml.

framework:
  name: SAPUI5
  version: "1.112.3"
  libraries:
    - name: sap.f
    - name: sap.m
    - name: sap.tnt
    - name: sap.ui.core
    - name: themelib_sap_horizon

Steps to Reproduce the Issue

config https-proxy and proxy in .npmrc, and check npm config list
npm i (success)
run npm run start got

Error Message:
Resolution of framework libraries failed with errors:
Failed to resolve library sap.f: No matching version found for @openui5/sap.f@1.113.0.
Failed to resolve library sap.m: No matching version found for @openui5/sap.m@1.113.0.
Failed to resolve library sap.ui.core: No matching version found for @openui5/sap.ui.core@1.113.0.
Failed to resolve library themelib_sap_fiori_3: No matching version found for @openui5/themelib_sap_fiori_3@1.113.0.

run ui5 serve --verbose got

...
verb ui5Framework:npm:Registry Using npm configuration (extract):
verb ui5Framework:npm:Registry    registry: https://registry.npmjs.org/
verb ui5Framework:npm:Registry    proxy: http://xxxxxxxxx@xxxxxxxxx:xxxx/
verb ui5Framework:npm:Registry    globalconfig: C:\Program Files\nodejs\etc\npmrc
verb ui5Framework:npm:Registry    userconfig: C:\Users\zoi7sgh\.npmrc
verb ui5Framework:npm:Registry    cache: C:\Users\zoi7sgh\.ui5\framework\cacache
verb ui5Framework:npm:Registry    cwd: C:\Users\zoi7sgh\Documents\Project\test\openui5-sample-app
...
verb ui5Framework:npm:Installer Installing @openui5/sap.f in version 1.113.0 to C:\Users\zoi7sgh\.ui5\framework\staging\@openui5-sap.f-1.113.0...
...
verb ui5Framework:AbstractResolver Failed to process library sap.f
verb ui5Framework:AbstractResolver Error: No matching version found for @openui5/sap.f@1.113.0.
verb ui5Framework:AbstractResolver Call stack: @openui5/sap.f: No matching version found for @openui5/sap.f@1.113.0.
    at module.exports (C:\Users\zoi7sgh\Documents\Project\Test\openui5-sample-app\node_modules\@ui5\cli\node_modules\npm-pick-manifest\lib\index.js:209:23)
    at RegistryFetcher.manifest (C:\Users\zoi7sgh\Documents\Project\Test\openui5-sample-app\node_modules\@ui5\cli\node_modules\pacote\lib\registry.js:119:22)
    at async Installer.fetchPackageManifest (file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/node_modules/@ui5/project/lib/ui5Framework/npm/Installer.js:59:22)    at async file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/node_modules/@ui5/project/lib/ui5Framework/Openui5Resolver.js:44:29
    at async Promise.all (index 0)
    at async Openui5Resolver._processLibrary (file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/node_modules/@ui5/project/lib/ui5Framework/AbstractResolver.js:87:33)
    at async file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/node_modules/@ui5/project/lib/ui5Framework/AbstractResolver.js:112:5
    at async Promise.all (index 0)
    at async Openui5Resolver._processLibraries (file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/node_modules/@ui5/project/lib/ui5Framework/AbstractResolver.js:110:19)
    at async Openui5Resolver.install (file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/node_modules/@ui5/project/lib/ui5Framework/AbstractResolver.js:189:3)    
    at async Object.enrichProjectGraph (file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/node_modules/@ui5/project/lib/graph/helpers/ui5Framework.js:372:29)    
    at async graphFromPackageDependencies (file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/node_modules/@ui5/project/lib/graph/graph.js:73:3)
    at async serve.handler (file:///C:/Users/zoi7sgh/Documents/Project/Test/openui5-sample-app/node_modules/@ui5/cli/lib/cli/commands/serve.js:89:11)
...
⚠️  Process Failed With Error

Go to non-proxy network and remove proxy in .npmrc, everything working well.

I think npm in ui5 cli not read the config in .npmrc.

Context

UI5 Module Version (output of ui5 --version when using the CLI): 3.0.7
Node.js Version: v18.16.0
npm Version: v9.5.1
OS/Platform: Windows 10 (also test in this docker/dev-environments-javascript with same config)

Answer 1 · 2023-04-24T10:03:07.000Z

Thanks for reporting. Could you please also share the contents of C:\Users\zoi7sgh\.npmrc and C:\Program Files\nodejs\etc\npmrc? We only care about what other configuration you have, so please redact any sensitive values.

This will help us in trying to reproduce the issue. Thank you.

My current assumption is that maybe you have other relevant proxy configuration in those files. And while npm is using them, UI5 Tooling is currently ignoring those.

Answer 2 · 2023-04-24T10:09:29.000Z

Hi @RandomByte
Thanks for your reply. This is the c:\userfolder.npmrc

proxy=http://xxxxxxx@xxxxxxxxx:xxxx/
https-proxy=http://xxxxxxx@xxxxxxxxx:xxxx/
registry=https://registry.npmjs.org/

There is no npmrc in nodejs\etc\npmrc. I'm now using the nvm on windows to enable my nodejs env.
Also in my ubuntu wsl env. There is a npmrc in ~. also with the same issue.

Answer 3 · 2023-04-24T10:29:19.000Z

So if ui5 tooling ignore those proxy config, how could we set proxy for ui5-tooling.

Answer 4 · 2023-04-24T11:10:21.000Z

Could you please run npm info @openui5/sap.f in the same folder to verify whether npm can fetch the relevant package information?

Answer 5 · 2023-04-25T01:24:32.000Z

Result is

PS C:\Users\zoi7sgh\Documents\Project\test\openui5-sample-app> npm info @openui5/sap.f

@openui5/sap.f@1.113.0 | Apache-2.0 | deps: 3 | versions: 415
OpenUI5 UI Library sap.f

keywords: openui5, sapui5, ui5

dist
.tarball: https://registry.npmjs.org/@openui5/sap.f/-/sap.f-1.113.0.tgz
.shasum: f08098485ceacdab1fcd084b530f9429df8e2c7a
.integrity: sha512-X5YPuV3h6mhszzfQJ9aOYAljHvJAv1eGx/7rPCL9CsDH9o3zs/bcJ9V+Ejn27dIaqW2Wcf4KG4jX1oOu8perFQ==
.unpackedSize: 1.5 MB

dependencies:
@openui5/sap.m: 1.113.0         @openui5/sap.ui.core: 1.113.0   @openui5/sap.ui.layout: 1.113.0

maintainers:
- randombyte <r@ndombyte.net>
- openui5-bot <ui5-bot+npm-openui5-bot@sap.com>

dist-tags:
latest: 1.113.0

published 5 days ago by openui5-bot <ui5-bot+npm-openui5-bot@sap.com>

npm is working well behind the proxy after i setting the proxy config in npmrc.

Answer 6 · 2023-04-25T07:37:41.000Z

Thanks 👍🏻
Although the configuration is read by UI5 Tooling and passed to pacote (the library npm uses to download packages) it doesn't seem that the proxy configuration is considered.

I'm currently trying to figure out whats wrong there and will keep you posted.

Answer 7 · 2023-04-25T07:49:06.000Z

From the code, I noticed that we don't pass the https-proxy to pacote: https://github.com/SAP/ui5-project/blob/0733cdaf18658897058a24ce1278c4051693dd51/lib/ui5Framework/npm/Registry.js#L71

However, there's no public documentation on that option in pacote or npm-registry-fetch

Answer 8 · 2023-04-25T07:52:33.000Z

From the code, I noticed that we don't pass the https-proxy to pacote: https://github.com/SAP/ui5-project/blob/0733cdaf18658897058a24ce1278c4051693dd51/lib/ui5Framework/npm/Registry.js#L71

However, there's no public documentation on that option in pacote or npm-registry-fetch

We do pass the option but only log some options. I will debug to see why it is not picked up and compare it with how npm does it.

Answer 9 · 2023-04-26T13:31:28.000Z

@zouyi100 I've pushed a PR that should solve the issue.
To be sure that this was a regression, could you please verify whether the behaviour worked before with @ui5/cli v3.0.0?
You can use this command to install exactly that version in openui5-sample-app:

npm install -D -E @ui5/cli@3.0.0

Answer 10 · 2023-04-26T15:04:47.000Z

Reopening until the proposed fix has been released and to get additional feedback from @zouyi100

Answer 11 · 2023-04-27T02:41:27.000Z

Hi @matz3
I have tested by using 3.0.0 which working well behind the proxy.

Answer 12 · 2023-04-27T06:05:34.000Z

Thanks a lot @zouyi100 for providing your feedback and opening this issue.

Answer 13 · 2023-04-27T06:47:44.000Z

Hi @flovogt
I think we need keep this ticket open until the new release on npm. Now the latest version is still v3.1.0 which i guess not include this fix. (Still show error when use 3.1.0)

Answer 14 · 2023-04-27T06:57:43.000Z

True. Thanks @zouyi100 I thought we have already triggered a release.

Answer 15 · 2023-04-27T08:03:55.000Z

The fix is included in @ui5/cli v3.1.1

Answer 16 · 2023-04-27T08:55:10.000Z

Working well with v3.1.1. Thank you, team! 👍