elastic/logstash-docker

"Error /bin/bash: jinja2: command not found" when build with python 3.6 in Ubuntu 18.04

mittaus opened this issue · 8 comments

Error /bin/bash: jinja2: command not found when exec ELASTIC_VERSION=6.3.1 make with Python 3.6 in Ubuntu 18.04

I'm install pre requisites for build and test in Ubuntu 18.04:

  • Docker
  • GNU Make
  • Python 3.5 with Virtualenv

Command and result:

user@ubuntu:~/projects/logstash-docker$ ELASTIC_VERSION=6.3.1 make
jinja2 -D elastic_version='6.3.1' -D staging_build_num='' -D version_tag='6.3.1' -D image_flavor='oss' -D artifacts_dir='' templates/Dockerfile.j2 > build/logstash/Dockerfile-oss;   jinja2 -D elastic_version='6.3.1' -D staging_build_num='' -D version_tag='6.3.1' -D image_flavor='full' -D artifacts_dir='' templates/Dockerfile.j2 > build/logstash/Dockerfile-full; 
/bin/bash: jinja2: command not found
/bin/bash: jinja2: command not found
Makefile:124: recipe for target 'dockerfile' failed
make: *** [dockerfile] Error 127

python version and config:

user@ubuntu:~/projects/logstash-docker$ update-alternatives --config python
There are 2 choices for the alternative python (providing /usr/bin/python).

  Selection    Path                Priority   Status
------------------------------------------------------------
  0            /usr/bin/python3.6   2         auto mode
  1            /usr/bin/python2.7   1         manual mode
* 2            /usr/bin/python3.6   2         manual mode

pip version

user@ubuntu:~/projects/logstash-docker$ pip --version
pip 18.1 from /home/user/.local/lib/python3.6/site-packages/pip (python 3.6)

ninja version

user@ubuntu:~/projects/logstash-docker$ pip show jinja2
Name: Jinja2
Version: 2.10
Summary: A small but fast and easy to use stand-alone template engine written in pure python.
Home-page: http://jinja.pocoo.org/
Author: Armin Ronacher
Author-email: armin.ronacher@active-4.com
License: BSD
Location: /usr/local/lib/python3.6/dist-packages
Requires: MarkupSafe
Required-by: 
jarpy commented

It looks like you might have a broken virtualenv in your build directory. Do you have a venv directory? Is it empty?

Try rm -rf venv before invoking make again.

Hi @jarpy

I have venv directory in my home directory:

user@ubuntu:~$ ls -all  python-virtual-environments/env/
total 28
drwxr-xr-x 6 user user 4096 Oct 25 16:22 .
drwxr-xr-x 3 user user 4096 Oct 25 16:19 ..
drwxr-xr-x 2 user user 4096 Oct 25 16:22 bin
drwxr-xr-x 2 user user 4096 Oct 25 16:19 include
drwxr-xr-x 3 user user 4096 Oct 25 16:19 lib
lrwxrwxrwx 1 user user    3 Oct 25 16:19 lib64 -> lib
-rw-r--r-- 1 user user   69 Oct 25 16:22 pyvenv.cfg
drwxr-xr-x 3 user user 4096 Oct 25 16:22 share

build directory for logstash

user@ubuntu:~$ ls -all projects/logstash-docker/
total 80
drwxr-xr-x  9 user user  4096 Oct 25 17:37 .
drwxr-xr-x 14 user user  4096 Oct 25 14:28 ..
drwxr-xr-x  2 user user  4096 Oct 25 14:30 bin
drwxr-xr-x  4 user user  4096 Oct 25 14:28 build
-rw-r--r--  1 user user     0 Oct 25 17:37 docker-compose-full.yml
-rw-r--r--  1 user user     0 Oct 25 17:37 docker-compose-oss.yml
drwxr-xr-x  2 user user  4096 Oct 25 14:28 examples
drwxr-xr-x  8 user user  4096 Oct 25 16:54 .git
-rw-r--r--  1 user user   153 Oct 25 14:30 .gitignore
-rw-r--r--  1 user user 11357 Oct 25 14:28 LICENSE
-rw-r--r--  1 user user  5382 Oct 25 14:28 Makefile
-rw-r--r--  1 user user  1157 Oct 25 14:30 README.md
-rw-r--r--  1 user user   117 Oct 25 14:28 requirements.txt
drwxr-xr-x  3 user user  4096 Oct 25 17:25 snapshots
drwxr-xr-x  2 user user  4096 Oct 25 14:28 templates
drwxr-xr-x  2 user user  4096 Oct 25 14:28 tests
-rw-r--r--  1 user user   143 Oct 25 14:28 tox.ini
-rw-r--r--  1 user user    87 Oct 25 14:28 .travis.yml
-rw-r--r--  1 user user     0 Oct 25 14:31 venv
-rw-r--r--  1 user user    21 Oct 25 16:54 version.json
jarpy commented

Is the venv directory in the build directory fully operational?

What happens when you source venv/bin/activate?

@jarpy at moment you are building the container after having done the following:

activate venv
bash user@ubuntu:~/python-virtual-environments$ source env/bin/activate

install again jinja2 into virtualenv

(env) user@ubuntu:~/python-virtual-environments$ pip install jinja2
Collecting jinja2
  Downloading https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
    100% |████████████████████████████████| 133kB 126kB/s 
Collecting MarkupSafe>=0.23 (from jinja2)
  Downloading https://files.pythonhosted.org/packages/4d/de/32d741db316d8fdb7680822dd37001ef7a448255de9699ab4bfcbdf4172b/MarkupSafe-1.0.tar.gz
Building wheels for collected packages: MarkupSafe
  Running setup.py bdist_wheel for MarkupSafe ... error
  Complete output from command /home/user/python-virtual-environments/env/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-707canx1/MarkupSafe/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmpnlkrdzfjpip-wheel- --python-tag cp36:
  usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: -c --help [cmd1 cmd2 ...]
     or: -c --help-commands
     or: -c cmd --help
  
  error: invalid command 'bdist_wheel'
  
  ----------------------------------------
  Failed building wheel for MarkupSafe
  Running setup.py clean for MarkupSafe
Failed to build MarkupSafe
Installing collected packages: MarkupSafe, jinja2
  Running setup.py install for MarkupSafe ... done
Successfully installed MarkupSafe-1.0 jinja2-2.10

Validation jinja2 install

(env) user@ubuntu:~/python-virtual-environments$ pip show jinja2
Name: Jinja2
Version: 2.10
Summary: A small but fast and easy to use stand-alone template engine written in pure python.
Home-page: http://jinja.pocoo.org/
Author: Armin Ronacher
Author-email: armin.ronacher@active-4.com
License: BSD
Location: /home/user/python-virtual-environments/env/lib/python3.6/site-packages
Requires: MarkupSafe

exec build command Ok

(env) user@ubuntu:~/python-virtual-environments$ ELASTIC_VERSION=6.3.1 make
Running virtualenv with interpreter /usr/bin/python3.6
unning virtualenv with interpreter /usr/bin/python3.6
Using base prefix '/usr'
New python executable in /home/user/python-virtual-environments/venv/bin/python3.6
Also creating executable in /home/user/python-virtual-environments/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
Collecting docker-compose==1.11.2 (from -r requirements.txt (line 1))
  Downloading https://files.pythonhosted.org/packages/66/d4/2b69141a0b5e685659891c04748bbb414bef67c79a46333d915fbda3731d/docker_compose-1.11.2-py2.py3-none-any.whl (85kB)
    100% |████████████████████████████████| 92kB 164kB/s 
Collecting flake8==3.4.1 (from -r requirements.txt (line 2))
  Downloading https://files.pythonhosted.org/packages/b2/d3/e80ea2616e3871ae22459c415c63d64b9bd744c92267482670fcdbcea72d/flake8-3.4.1-py2.py3-none-any.whl (68kB)
    100% |████████████████████████████████| 71kB 207kB/s 
Collecting jinja2-cli[yaml]==0.6.0 (from -r requirements.txt (line 3))
  Downloading https://files.pythonhosted.org/packages/ca/0f/dd461ac244fec4310c425e9cec9553d8ae8523c36837792e7666c6e53393/jinja2_cli-0.6.0-py2.py3-none-any.whl
Collecting jinja2==2.9.5 (from -r requirements.txt (line 4))
  Downloading https://files.pythonhosted.org/packages/3c/d1/49d69bc23d0e0c7612248dd8f5391bd043648865132309616c280ca1c837/Jinja2-2.9.5-py2.py3-none-any.whl (340kB)
    100% |████████████████████████████████| 348kB 137kB/s 

.
.
.

jarpy commented

Yes. I figured there was something wrong with the venv. Did you try deleting it completely and letting it rebuild?

No, what happened was that it was necessary to install jinja2 within the venv. I had installed on the same host, but not in the venv. Then execute the build from the venv

jarpy commented

I understand. Normally, Jinja2 is installed by the Makefile.

You probably had a transient error on your first run.

@jarpy thankz for your support. I was able to create the image and create an instance from docker.

Regards.