lightly-ai/lightly

Respect HTTP proxy environment variables

benz0li opened this issue · 2 comments

benz0li commented

@japrescott As already explained in my emails:

One can configure urllib3.PoolManager’s ca_certs via environment variable LIGHTLY_CA_CERTS.

  1. lightly/lightly/api/utils.py

    Lines 225 to 242 in 715844b

    def get_api_client_configuration(
    token: Optional[str] = None,
    raise_if_no_token_specified: bool = True,
    ) -> Configuration:
    host = get_lightly_server_location_from_env()
    ssl_ca_cert = getenv("LIGHTLY_CA_CERTS", None)
    if token is None:
    token = getenv("LIGHTLY_TOKEN", None)
    if token is None and raise_if_no_token_specified:
    raise ValueError(
    "Either provide a 'token' argument or export a LIGHTLY_TOKEN environment variable"
    )
    configuration = Configuration()
    configuration.api_key = {"ApiKeyAuth": token}
    configuration.ssl_ca_cert = ssl_ca_cert
    configuration.host = host

  2. lightly/lightly/openapi_generated/swagger_client/rest.py

    Lines 83 to 105 in 715844b

    # https pool manager
    if configuration.proxy:
    self.pool_manager = urllib3.ProxyManager(
    num_pools=pools_size,
    maxsize=maxsize,
    cert_reqs=cert_reqs,
    ca_certs=configuration.ssl_ca_cert,
    cert_file=configuration.cert_file,
    key_file=configuration.key_file,
    proxy_url=configuration.proxy,
    proxy_headers=configuration.proxy_headers,
    **addition_pool_args
    )
    else:
    self.pool_manager = urllib3.PoolManager(
    num_pools=pools_size,
    maxsize=maxsize,
    cert_reqs=cert_reqs,
    ca_certs=configuration.ssl_ca_cert,
    cert_file=configuration.cert_file,
    key_file=configuration.key_file,
    **addition_pool_args
    )

But as long as configuration.proxy remains unset, there is no way that urllib3.ProxyManager (and thus a HTTP Proxy) is used.

Not only configuration.ssl_ca_cert but also configuration.proxy should be settable via an environment variable to fully support running behind a corporate proxy/firewall.

Usually this is done by common environment variables like HTTP_PROXY, HTTPS_PROXY, ALL_PROXY (or their lowercase counterparts).

FYI @IgorSusmelj @nasserdr

benz0li commented

Current workaround for the Lightly worker: Mounting a patched rest.py to /home/boris/openapi/build/swagger_client/swagger_client/rest.py.

Patch: Inserting configuration.proxy = "http://<host>:<port>" just before

lightly/lightly/openapi_generated/swagger_client/rest.py

Lines 83 to 85 in 715844b

# https pool manager
if configuration.proxy:
self.pool_manager = urllib3.ProxyManager(

japrescott commented

Hey @benz0li
Thanks a lot for opening a issue.
The proxy variable will need to be set within the get_api_client_configuration fn.
Will open a PR later today

  • 👍1