/tensorflow-python3-jupyter

tensorflow-python3-jupyter

Primary LanguageJupyter NotebookApache License 2.0Apache-2.0

GPU/CUDA Supported: Tensorflow + Python3 + Jupyter + PySpark + Spark 3.0 + Hadoop 3.2

* (NEW) Auto detect & enable GPU/CUDA see CUDA Detection in Notebook into Container and Jupyter Notebook!

* (New) Added R Kernel support.

NOTE: The latest update supported HTTPS for Jupyter Notebook to increase security:

  • Launch the Tensorflow-Python3-Jupyter server (http://<ip>:28888/ - default or https://<ip>:28888/) -- To change HTTPS or HTTP, see next paragraph
    • Remember to ".env" setup for using HTTPS since the default is HTTP (again!)
  • Launch the Tensorboard Web GUI server (http://<ip>:26006/ as default)

To disable/enable HTTPS for Jupyter Notebook

Just edit the .env file's entry "ENABLE_HTTPS". The following changes back to HTTP instead of HTTPS.

## -- true to enable HTTPS; false to disable HTTPS:
ENABLE_HTTPS=false
#ENABLE_HTTPS=true

Zeppelin Notebooks as alternative to Jupyter Notebooks

We also provide Zeppelin notebook openkbs/docker-spark-bde2020-zeppelin. If you want Scala/Java Spark ML/Mlib etc, you might try that as well. In all, we support both Zeppelin Notebooks and Jupyter Notebooks technologies for both Spark Scala/Java and Python/PySpark data science users' needs.

Components:

Run (GPU/Nvidia - Auto Enable)

  • To run GPU/Nvidia, you need to install the Nvidia Driver in your HOST machine first and then install nvidia-docker2.
  • Please refer to Nvidia Container Toolkit documentation for how to install properly
  • You also need to setup environment variables once you have successfully install Nvidia driver and Nvidia-docker2 Container Toolkit before you run Docker (trying to use nvidia-docker2). It's recommended to setup in your HOST VM or Machine's user account's .bashrc profile.
export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

./run.sh -g
or, let it auto check and use Nvidia GPU if available:
./run.sh

Run (If choose only CPU!)

  • It will download 'yolov5s.pt' on-the-fly to use if not existing.
./run.sh
or, explicitly disable GPU to use CPU.
./run.sh -c

Run (using docker-compose)

docker pull openkbs/tensorflow-python3-jupyter
docker-compose up -d

Once it is up, you can access it with:

  • The default Jupyter password is (unless you change it in .env file): ChangeMe!
http://<IP_Address>:28888/ (Juypter Notebook)  -- if you disable HTTPS
https://<IP_Address>:28888/ (Juypter Notebook)  -- if you enable HTTPS
http://<IP_Address>:26006/ (Jupyter Tensorboard)

Where <IP_Address> will be:

  • If running locally: localhost, 0.0.0.0, 127.0.0.1, or host's IP address.
  • If running with Openshift: then find out the domain name that OpenShift has created for this container.

How to change the default password?

Edit the ./docker-run.env file, say, to change to MyNewPassword!

#### Don't use any quote to enclose the password since it (quotes) will be treated as part of the value!
PASSWORD=StrongPassword

What if I need more Python/PIP packages?

  1. Enter the Container
./shell.sh
  1. Install the new Python/PIP package, e.g.,
sudo python3 -m pip install <package>
or 
sudo pip install <package>
e.g.
    sudo pip install nose
    sudo pip install IPython
    sudo pip install pyGtk
    sudo pip install Scrapy
    . . . etc.

If you are using OpenShift / Origin Contaier Platform, you can use the Web UI to enter the Container's Console and then run the above Python package installation.

  1. Or, you can add your packages into the file below and restart "./run.sh" your Container. It will install additional packages in the file, "./requirements-extra.txt", for you.
# Bert pre-trained model
pytorch-pretrained-bert

(Other way) to install more packages - Inside Jupyter Notebook

# Install a pip package in the current Jupyter kernel
import sys
!{sys.executable} -m pip install <package>
e.g.
    import sys
    !{sys.executable} -m pip install numpy

Problems in Login

Few reasons that your login password might not work:

  1. You have multiple place set password and they are inconsistent: just search "password" and "PASSWORD" for all the files in the folder - basically, docker-compose.yml, docker-run.sh, run-sh (if you modify it) files are key sources for setting up password for login.
  2. It might be your Chrome or browsers auto feed old password as cache. To solve this problem, you just launch a new incognito windows and then type URL as usuall:
http://<ip>:28888/  (if you disable HTTPS, i.e., you want HTTP://...)
or
https://<ip>:28888/  (if you enable HTTPS, i.e., you want HTTPS://...)

Resources & References

Note: the jupyter notebooks are only based on Python 3 kernels as default.

See Also - docker-based IDE

Python Packages List

You can use either

import sys
!{sys.executable} -m pip list

or

developer@cace1316ae93:~$ pip freeze
absl-py==0.10.0
appdirs==1.4.4
argon2-cffi==20.1.0
asn1crypto==0.24.0
astunparse==1.6.3
async-generator==1.10
attrs==20.2.0
backcall==0.2.0
beautifulsoup4==4.6.0
bleach==3.2.1
blis==0.4.1
cachetools==4.1.1
catalogue==1.0.0
certifi==2020.6.20
cffi==1.14.3
chardet==3.0.4
click==7.1.2
cloudpickle==1.6.0
cryptography==2.1.4
cycler==0.10.0
cymem==2.0.3
decorator==4.4.2
defusedxml==0.6.0
distlib==0.3.1
entrypoints==0.3
filelock==3.0.12
findspark==1.4.2
fire==0.3.1
Flask==1.1.2
funcy==1.15
future==0.18.2
gast==0.3.3
gensim==3.8.3
google-auth==1.22.1
google-auth-oauthlib==0.4.1
google-pasta==0.2.0
grpcio==1.32.0
h5py==2.10.0
html5lib==0.999999999
httpie==2.2.0
hyperopt==0.2.5
idna==2.10
importlib-metadata==2.0.0
importlib-resources==3.0.0
iniconfig==1.1.1
ipaddress==1.0.23
ipykernel==5.3.4
ipython==7.16.1
ipython-genutils==0.2.0
ipywidgets==7.5.1
itsdangerous==1.1.0
j2cli==0.3.10
jedi==0.17.2
Jinja2==2.11.2
joblib==0.17.0
json-lines==0.5.0
jsonschema==3.2.0
jupyter==1.0.0
jupyter-client==6.1.7
jupyter-console==6.2.0
jupyter-core==4.6.3
jupyterlab-pygments==0.1.2
Keras==2.4.3
Keras-Preprocessing==1.1.2
keras2onnx==1.7.0
keyring==10.6.0
keyrings.alt==3.0
kiwisolver==1.2.0
langdetect==1.0.8
lxml==4.2.1
Markdown==3.3.2
MarkupSafe==1.1.1
matplotlib==3.3.2
mistune==0.8.4
murmurhash==1.0.2
nbclient==0.5.1
nbconvert==6.0.7
nbformat==5.0.8
nest-asyncio==1.4.1
networkx==2.5
nltk==3.5
notebook==6.1.4
numexpr==2.7.1
numpy==1.19.2
oauthlib==3.1.0
olefile==0.45.1
onnx==1.7.0
onnxconverter-common==1.7.0
onnxmltools==1.7.0
onnxruntime==1.5.2
opt-einsum==3.3.0
packaging==20.4
panda==0.3.1
pandas==1.1.3
pandasql==0.7.3
pandocfilters==1.4.2
parso==0.7.1
pbr==5.5.0
pexpect==4.8.0
pickleshare==0.7.5
Pillow==8.0.0
pkgconfig==1.5.1
plac==1.1.3
pluggy==0.13.1
preshed==3.0.2
prometheus-client==0.8.0
prompt-toolkit==3.0.8
protobuf==3.13.0
ptyprocess==0.6.0
py==1.9.0
py4j==0.10.9
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pycrypto==2.6.1
Pygments==2.7.1
pygobject==3.26.1
pyLDAvis==2.1.2
pyparsing==2.4.7
pyrsistent==0.17.3
pyspark==3.0.1
pytest==6.1.1
python-apt==1.6.5+ubuntu0.3
python-dateutil==2.8.1
python-git==2018.2.1
pytz==2020.1
pyxdg==0.25
PyYAML==5.3.1
pyzmq==19.0.2
qtconsole==4.7.7
QtPy==1.9.0
regex==2020.10.15
requests==2.24.0
requests-oauthlib==1.3.0
rsa==4.6
scikit-learn==0.23.2
scipy==1.5.3
seaborn==0.11.0
SecretStorage==2.3.1
Send2Trash==1.5.0
six==1.15.0
skl2onnx==1.7.0
smart-open==3.0.0
spacy==2.3.2
SQLAlchemy==1.3.20
srsly==1.0.2
stevedore==3.2.2
tables==3.4.2
tensorboard==2.3.0
tensorboard-plugin-wit==1.7.0
tensorflow==2.3.1
tensorflow-estimator==2.3.0
termcolor==1.1.0
terminado==0.9.1
testpath==0.4.4
tf2onnx==1.7.1
thinc==7.4.1
threadpoolctl==2.1.0
toml==0.10.1
torch==1.6.0
tornado==6.0.4
tqdm==4.50.2
traitlets==4.3.3
typing-extensions==3.7.4.3
unattended-upgrades==0.1
urllib3==1.25.11
virtualenv==20.0.35
virtualenv-clone==0.5.4
virtualenvwrapper==4.8.4
wasabi==0.8.0
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==1.0.1
widgetsnbextension==3.5.1
wrapt==1.12.1
yml2json==1.1.3
zipp==3.3.1

Releases information

See Release information