Proxy use when running Galaxy
Closed this issue · 7 comments
I'm behind an HTTP proxy when trying to run Galaxy so I can't download packages or docker images without a proxy. I've verified that Docker is working fine behind the proxy, and any packages that it fails on I can pull from the local machine. Is there something I need to do to get the containers to be able to use the proxy?
I had to modify the docker template to build the conductor, but running the django container causes an error when trying to download postgresql. Here is the fuck traceback:
The full traceback is:
File "/tmp/ansible_F1CRZ_/ansible_modlib.zip/ansible/module_utils/urls.py", line 1039, in fetch_url
client_key=client_key)
File "/tmp/ansible_F1CRZ_/ansible_modlib.zip/ansible/module_utils/urls.py", line 948, in open_url
r = urllib_request.urlopen(*urlopen_args)
File "/_usr/lib64/python2.7/urllib2.py", line 154, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib64/python2.7/urllib2.py", line 429, in open
req = meth(req)
File "/tmp/ansible_F1CRZ/ansible_modlib.zip/ansible/module_utils/urls.py", line 766, in http_request
raise ConnectionError('Failed to connect to %s at port %s: %s' % (self.hostname, self.port, to_native(e)))
fatal: [django]: FAILED! => {
"changed": false,
"failed": true,
"invocation": {
"module_args": {
"attributes": null,
"backup": null,
"checksum": "",
"client_cert": null,
"client_key": null,
"content": null,
"delimiter": null,
"dest": "/tmp/pgdg-centos95-9.5-3.noarch.rpm",
"directory_mode": null,
"follow": false,
"force": false,
"force_basic_auth": false,
"group": null,
"headers": null,
"http_agent": "ansible-httpget",
"mode": null,
"owner": null,
"regexp": null,
"remote_src": null,
"selevel": null,
"serole": null,
"setype": null,
"seuser": null,
"sha256sum": "",
"src": null,
"timeout": 10,
"tmp_dest": null,
"unsafe_writes": null,
"url": "https://download.postgresql.org/pub/repos/yum/9.5/redhat/rhel-7-x86_64/pgdg-centos95-9.5-3.noarch.rpm",
"url_password": null,
"url_username": null,
"use_proxy": true,
"validate_certs": true
}
},
"msg": "Failed to connect to download.postgresql.org at port 443: [Errno 101] Network is unreachable"
}
to retry, use: --limit @/tmp/tmpcJ3UCs/playbook.retry
PLAY RECAP *********************************************************************
django : ok=1 changed=0 unreachable=0 failed=1
2017-07-20T12:19:48.102279 Error applying role! [container.core] caller_file=/_ansible/container/core.py caller_func=apply_role_to_container caller_line=627 engine=<container.docker.engine.Engine object at 0x3689950> exit_code=2 playbook=[{'hosts': u'django', 'roles': ['django-role'], 'vars': {u'galaxy_project_dir': u'/galaxy', u'galaxy_rabbitmq_pass': u'galaxy', u'galaxy_postgres_user': u'galaxy', u'galaxy_postgres_db': u'galaxy', u'galaxy_env': u'DEV', u'galaxy_site_name': u'localhost', u'galaxy_user': u'django', u'galaxy_admin_password': u'admin', u'galaxy_email_username': u'', u'galaxy_venv': u'/venv', u'galaxy_accounts_handler': u'console', u'galaxy_send_email': False, u'galaxy_site_aliases': [u'localhost', u'127.0.0.1', u'0.0.0.0'], u'galaxy_configure_supervisor': False, u'galaxy_email_port': 0, u'galaxy_create_superuser': True, u'galaxy_allauth_handler': u'console', u'galaxy_admin_username': u'admin', u'galaxy_rabbitmq_vhost': u'galaxy', u'galaxy_email_password': u'', u'galaxy_main_handler': u'console', u'galaxy_rabbitmq_user': u'galaxy', u'galaxy_django_handler': u'django_logfile', u'galaxy_postgres_password': u'galaxy', u'galaxy_site_env': u'DEV', u'galaxy_email_hostname': u''}}]
2017-07-20T12:19:48.105360 Playbook run finished. [container.core] caller_file=/_ansible/container/core.py caller_func=conductorcmd_build caller_line=753 exit_code=2
Traceback (most recent call last):
File "/usr/bin/conductor", line 11, in
load_entry_point('ansible-container', 'console_scripts', 'conductor')()
File "/_ansible/container/init.py", line 19, in wrapped
return fn(*args, **kwargs)
File "/_ansible/container/cli.py", line 379, in conductor_commandline
**params)
File "/_ansible/container/init.py", line 19, in wrapped
return fn(*args, **kwargs)
File "/_ansible/container/core.py", line 755, in conductorcmd_build
raise RuntimeError('Build failed.')
RuntimeError: Build failed.
2017-07-20T08:19:48.244033 Conductor terminated. Cleaning up. [container.docker.engine] caller_file=/home/USER/ansible-container-develop/container/docker/engine.py caller_func=await_conductor_command caller_line=356 command_rc=1 conductor_id=u'7f380338f64b7102240314833e44022675e7b8b0f0e24b02f561e2cd93e7a839' save_container=False
2017-07-20T08:19:48.259581 Conductor exited with status 1 [container.cli] caller_file=/home/USER/ansible-container-develop/container/cli.py caller_func=call caller_line=291
make: *** [build] Error 1
Not sure there is a programmatic way to do what you need. The first step is to add the http_proxy and/or https_proxy environment variables to the container.yml file, like so:
version: "2"
settings:
save_conductor_container: false
conductor:
environment:
http_proxy: http://my-proxy-server
https_proxy: https://my-proxy_server
vars_files:
- develop.yml
services:
django:
...
environment:
- C_FORCE_ROOT=1
- http_proxy=http://my-proxy-server
- https_proxy=https://my-proxy-server
....
gulp:
...
environment:
- http_proxy=http://my-proxy-server
- https_proxy=https://my-proxy-server
...
registries: {}
Is there any where else that I might need to add the environmental variables? I've added them to the main container.yml but the get_url module is still failing even when it says use_proxy=true. I know it's something with the containers because I can wget the file from the server.
As an experiment, you might try modifying the task directly, so that it looks like the following:
- name: Get the file
get_url:
url: ...
dest: ....
environment:
http_proxy: http://my-proxy-server
See http://docs.ansible.com/ansible/latest/playbooks_environment.html for more details, if needed.
I was hoping to avoid modifying the tasks by setting the environment variables at the service level in container.yml.
@chouseknecht
Doesn't look like get_url supports the environment parameter. It only supports the use_proxy which is set as true.
" "msg": "Unsupported parameters for (get_url) module: environment. Supported parameters include: attributes,backup,checksum,client_cert,client_key,content,delimiter,dest,directory_mode,follow,force,force_basic_auth,group,headers,http_agent,mode,owner,regexp,remote_src,selevel,serole,setype,seuser,sha256sum,src,timeout,tmp_dest,unsafe_writes,url,url_password,url_username,use_proxy,validate_certs""
It's not a module parameter. Take a look at the example below. The module name and environment should be at the same indentation level.
- apt: name=cobbler state=installed
environment:
http_proxy: http://proxy.example.com:8080
Yep that did it! Thanks so much!! If there's a place for it I'd love to document which steps needed to have the environment step added so no one else needs to try and fight with all of this.
This issue was moved to ansible/galaxy#43