pfultz2/cget

"cget install" fails for items accessible to wget; proxy consideration

Opened this issue · 3 comments

jbsysadmin commented

Is there anything that can be edited, or set to allow cget to function with proxy defined?

Context: python-3.8.1, attempts to install SAIGE (https://github.com/weizhouUMICH/SAIGE) fail because cget can't find a dependency known to be at specified location.

Util wget, R, and the OS (Centos 6.x, plus newer custom compiled items) have proxy settings configured. Installs not using cget, but downloading remote resources, all succeed in retrieving what they need.

Any leads?

pfultz2 commented

cget uses the six.moves.urllib.request.FancyURLopener to download URLs. This is a wrapper around urllib.FancyURLopener.

I see now that there is a urllib.request.ProxyHandler class that can handle proxies. I should probably switch to that in cget. Let me try to figure out how this class works.

All the downloads happen in the download_to function.

jbsysadmin commented

Hello again! Any luck having a version of cget which can function with a proxy URL defined? The maintainers of SAIGE insist on using cget, but their installer won't work in a proxy setting. Building on non-proxy VMs, then rsyncing over is not an option; compilations should complete on the allocated work machines.

pfultz2 commented

So it looks like python should use the same proxy on the system by default. Maybe it doesn't detect them. Either way, you should be able to override them using environment variables like HTTP_PROXY or HTTPS_PROXY. This is what is shows here.