Error downloading packages
Opened this issue · 1 comments
Description
Downloading some packages causes an error.
This error happens with the latest version of vuln-reach-cli and with the config file at https://gitlab.com/-/snippets/3623569.
The error is intermittent. When rerunning vuln-reach-cli, packages that failed to download before might be downloaded successfully.
I verified that packages that vuln-reach-cli failed to download are available and can be downloaded manually. For example, @aashutoshrathi/word-wrap can be downloaded with $ wget $(npm info @aashutoshrathi/word-wrap dist.tarball).
Expected Behavior
All packages are downloaded and reachability analysis is performed.
Actual Behavior
$ vuln-reach-cli config.toml
Downloading ...
Error: error sending request for url (https://registry.npmjs.org/@aashutoshrathi/word-wrap): error trying to connect: dns error: failed to lookup address information: nodename nor servname provided, or not known
Caused by:
0: error trying to connect: dns error: failed to lookup address information: nodename nor servname provided, or not known
1: dns error: failed to lookup address information: nodename nor servname provided, or not known
2: failed to lookup address information: nodename nor servname provided, or not known
Thank you for reporting this!
It looks like a few connections are failing. The reason is most likely that the CLI is starting all connections concurrently and thus runs into a rate limit of sorts when there is a high number of them.
The tarballs are cached and aren't redownloaded, so as a stopgap it should be enough to re-run the CLI to download the missing packages without incurring in the connection failures from the high concurrency.
On our end, buffering the connections instead of awaiting all of them together should be sufficient to solve this.