Install numba in DECODE environment?
cleterrier opened this issue · 5 comments
Hi,
I am trying to locally install the COMET drift-correction notebook (https://github.com/gpufit/Comet/blob/master/Colab_notebooks/COMET.ipynb) to chain it with DECODE output and correct drift on localization data.
The "from numba import cuda" fails because numba is not intalled in the DECODE environment. I tried a simple "conda install numba" but it does not seem to resolve properly. Any idea of what I should do?
Here is everything that is installed in my current DECODE environment:
absl-py 0.12.0 pyhd8ed1ab_0 conda-forge
aiohttp 3.7.4 py38h294d835_0 conda-forge
anyio 2.2.0 py38haa244fe_0 conda-forge
appdirs 1.4.4 pyh9f0ad1d_0 conda-forge
argon2-cffi 20.1.0 py38h294d835_2 conda-forge
async-timeout 3.0.1 py_1000 conda-forge
async_generator 1.10 py_0 conda-forge
atomicwrites 1.4.0 pyh9f0ad1d_0 conda-forge
attrs 20.3.0 pyhd3deb0d_0 conda-forge
babel 2.9.0 pyhd3deb0d_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
blas 2.106 mkl conda-forge
bleach 3.3.0 pyh44b312d_0 conda-forge
blinker 1.4 py_1 conda-forge
blosc 1.21.0 h0e60522_0 conda-forge
brotli 1.0.9 h0e60522_4 conda-forge
brotlipy 0.7.0 py38h294d835_1001 conda-forge
bzip2 1.0.8 h8ffe710_4 conda-forge
ca-certificates 2020.12.5 h5b45459_0 conda-forge
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cachetools 4.2.1 pyhd8ed1ab_0 conda-forge
certifi 2020.12.5 py38haa244fe_1 conda-forge
cffi 1.14.5 py38hd8c33c5_0 conda-forge
chardet 4.0.0 py38haa244fe_1 conda-forge
charls 2.2.0 h39d44d4_0 conda-forge
click 7.1.2 pyh9f0ad1d_0 conda-forge
cloudpickle 1.6.0 py_0 conda-forge
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
cryptography 3.4.7 py38hd7da0ea_0 conda-forge
cudatoolkit 11.0.3 h3f58a73_8 conda-forge
cycler 0.10.0 py_2 conda-forge
cytoolz 0.11.0 py38h294d835_3 conda-forge
dask-core 2021.4.1 pyhd8ed1ab_0 conda-forge
decode 0.10.0 np119py_0 turagalab
decorator 4.4.2 py_0 conda-forge
defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge
deprecated 1.2.12 pyh44b312d_0 conda-forge
deprecation 2.1.0 pyh9f0ad1d_0 conda-forge
entrypoints 0.3 pyhd8ed1ab_1003 conda-forge
freetype 2.10.4 h546665d_1 conda-forge
fsspec 2021.4.0 pyhd8ed1ab_0 conda-forge
giflib 5.2.1 h8d14728_2 conda-forge
gitdb 4.0.7 pyhd8ed1ab_0 conda-forge
gitpython 3.1.15 pyhd8ed1ab_0 conda-forge
google-auth 1.28.0 pyh44b312d_0 conda-forge
google-auth-oauthlib 0.4.1 py_2 conda-forge
grpcio 1.37.0 py38he5377a8_0 conda-forge
h5py 3.2.1 nompi_py38he6c2248_100 conda-forge
hdf5 1.10.6 nompi_h5268f04_1114 conda-forge
icu 68.1 h0e60522_0 conda-forge
idna 2.10 pyh9f0ad1d_0 conda-forge
imagecodecs 2021.3.31 py38hccd6b70_0 conda-forge
imageio 2.9.0 py_0 conda-forge
importlib-metadata 4.0.1 py38haa244fe_0 conda-forge
importlib_resources 5.1.2 py38haa244fe_0 conda-forge
iniconfig 1.1.1 pyh9f0ad1d_0 conda-forge
intel-openmp 2021.2.0 h57928b3_616 conda-forge
ipykernel 5.5.3 py38h43734a8_0 conda-forge
ipython 7.22.0 py38h43734a8_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
ipywidgets 7.6.3 pypi_0 pypi
jedi 0.18.0 py38haa244fe_2 conda-forge
jinja2 2.11.3 pyh44b312d_0 conda-forge
joblib 1.0.1 pyhd8ed1ab_0 conda-forge
jpeg 9d h8ffe710_0 conda-forge
json5 0.9.5 pyh9f0ad1d_0 conda-forge
jsonschema 3.2.0 pyhd8ed1ab_3 conda-forge
jupyter-packaging 0.9.2 pyhd8ed1ab_0 conda-forge
jupyter_client 6.1.12 pyhd8ed1ab_0 conda-forge
jupyter_core 4.7.1 py38haa244fe_0 conda-forge
jupyter_server 1.6.4 py38haa244fe_0 conda-forge
jupyterlab 3.0.14 pyhd8ed1ab_0 conda-forge
jupyterlab-widgets 1.0.0 pypi_0 pypi
jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge
jupyterlab_server 2.4.0 pyhd8ed1ab_0 conda-forge
jxrlib 1.1 h8ffe710_2 conda-forge
kiwisolver 1.3.1 py38hbd9d945_1 conda-forge
krb5 1.17.2 hbae68bd_0 conda-forge
lcms2 2.12 h2a16943_0 conda-forge
lerc 2.2.1 h0e60522_0 conda-forge
libaec 1.0.4 h39d44d4_1 conda-forge
libblas 3.9.0 6_mkl conda-forge
libcblas 3.9.0 6_mkl conda-forge
libclang 11.1.0 default_h5c34c98_0 conda-forge
libcurl 7.76.1 hf1763fc_1 conda-forge
libdeflate 1.7 h8ffe710_5 conda-forge
liblapack 3.9.0 6_mkl conda-forge
liblapacke 3.9.0 6_mkl conda-forge
libpng 1.6.37 h1d00b33_2 conda-forge
libprotobuf 3.15.8 h7755175_0 conda-forge
libsodium 1.0.18 h8d14728_1 conda-forge
libssh2 1.9.0 h680486a_6 conda-forge
libtiff 4.2.0 hc10be44_1 conda-forge
libuv 1.41.0 h8ffe710_0 conda-forge
libwebp-base 1.2.0 h8ffe710_2 conda-forge
libzopfli 1.0.3 h0e60522_0 conda-forge
locket 0.2.0 py_2 conda-forge
lz4-c 1.9.3 h8ffe710_0 conda-forge
m2w64-gcc-libgfortran 5.3.0 6 conda-forge
m2w64-gcc-libs 5.3.0 7 conda-forge
m2w64-gcc-libs-core 5.3.0 7 conda-forge
m2w64-gmp 6.1.0 2 conda-forge
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 conda-forge
markdown 3.3.4 pyhd8ed1ab_0 conda-forge
markupsafe 1.1.1 py38h294d835_3 conda-forge
matplotlib 3.4.1 py38haa244fe_0 conda-forge
matplotlib-base 3.4.1 py38heae8d8c_0 conda-forge
mistune 0.8.4 py38h294d835_1003 conda-forge
mkl 2020.4 hb70f87d_311 conda-forge
more-itertools 8.7.0 pyhd8ed1ab_1 conda-forge
msys2-conda-epoch 20160418 1 conda-forge
multidict 5.1.0 py38h294d835_1 conda-forge
nbclassic 0.2.7 pyhd8ed1ab_0 conda-forge
nbclient 0.5.3 pyhd8ed1ab_0 conda-forge
nbconvert 6.0.7 py38haa244fe_3 conda-forge
nbformat 5.1.3 pyhd8ed1ab_0 conda-forge
nd2reader 3.3.0 pypi_0 pypi
nest-asyncio 1.5.1 pyhd8ed1ab_0 conda-forge
networkx 2.5.1 pyhd8ed1ab_0 conda-forge
ninja 1.10.2 h5362a0b_0 conda-forge
notebook 6.3.0 pyha770c72_1 conda-forge
numpy 1.19.5 py38h0cc643e_1 conda-forge
oauthlib 3.0.1 py_0 conda-forge
olefile 0.46 pyh9f0ad1d_1 conda-forge
openjpeg 2.4.0 h48faf41_0 conda-forge
openssl 1.1.1k h8ffe710_0 conda-forge
packaging 20.9 pyh44b312d_0 conda-forge
pandas 1.2.4 py38h60cbd38_0 conda-forge
pandoc 2.13 h8ffe710_0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
parso 0.8.2 pyhd8ed1ab_0 conda-forge
partd 1.2.0 pyhd8ed1ab_0 conda-forge
patsy 0.5.1 py_0 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 8.1.2 py38h9273828_1 conda-forge
pims 0.5 pypi_0 pypi
pip 21.1 pyhd8ed1ab_0 conda-forge
pluggy 0.13.1 py38haa244fe_4 conda-forge
pooch 1.3.0 pyhd8ed1ab_0 conda-forge
prometheus_client 0.10.1 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.18 pyha770c72_0 conda-forge
protobuf 3.15.8 py38h885f38d_0 conda-forge
py 1.10.0 pyhd3deb0d_0 conda-forge
pyasn1 0.4.8 py_0 conda-forge
pyasn1-modules 0.2.7 py_0 conda-forge
pycparser 2.20 pyh9f0ad1d_2 conda-forge
pygments 2.8.1 pyhd8ed1ab_0 conda-forge
pyjwt 2.0.1 pyhd8ed1ab_1 conda-forge
pyopenssl 20.0.1 pyhd8ed1ab_0 conda-forge
pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge
pyqt 5.12.3 py38haa244fe_7 conda-forge
pyqt-impl 5.12.3 py38h885f38d_7 conda-forge
pyqt5-sip 4.19.18 py38h885f38d_7 conda-forge
pyqtchart 5.12 py38h885f38d_7 conda-forge
pyqtwebengine 5.12.1 py38h885f38d_7 conda-forge
pyreadline 2.1 py38haa244fe_1003 conda-forge
pyrsistent 0.17.3 py38h294d835_2 conda-forge
pysocks 1.7.1 py38haa244fe_3 conda-forge
pytest 6.2.3 py38haa244fe_0 conda-forge
python 3.8.8 h7840368_0_cpython conda-forge
python-dateutil 2.8.1 py_0 conda-forge
python_abi 3.8 1_cp38 conda-forge
pytorch 1.7.1 py3.8_cuda110_cudnn8_0 pytorch
pytz 2021.1 pyhd8ed1ab_0 conda-forge
pywavelets 1.1.1 py38h347fdf6_3 conda-forge
pywin32 300 py38h294d835_0 conda-forge
pywinpty 0.5.7 py38haa244fe_1 conda-forge
pyyaml 5.4.1 py38h294d835_0 conda-forge
pyzmq 22.0.3 py38h09162b1_1 conda-forge
qt 5.12.9 h5909a2a_4 conda-forge
requests 2.25.1 pyhd3deb0d_0 conda-forge
requests-oauthlib 1.3.0 pyh9f0ad1d_0 conda-forge
rsa 4.7.2 pyh44b312d_0 conda-forge
scikit-image 0.18.1 py38h4c96930_0 conda-forge
scikit-learn 0.24.1 py38ha09990b_0 conda-forge
scipy 1.6.2 py38he847743_0 conda-forge
seaborn 0.10.1 py_0 conda-forge
send2trash 1.5.0 py_0 conda-forge
setuptools 49.6.0 py38haa244fe_3 conda-forge
six 1.15.0 pyh9f0ad1d_0 conda-forge
slicerator 1.0.0 pypi_0 pypi
smmap 3.0.5 pyh44b312d_0 conda-forge
snappy 1.1.8 ha925a31_3 conda-forge
sniffio 1.2.0 py38haa244fe_1 conda-forge
spline 0.10.0 np119py38ha925a31_1 turagalab
sqlite 3.35.5 h8ffe710_0 conda-forge
statsmodels 0.12.2 py38h347fdf6_0 conda-forge
tensorboard 2.4.1 pyhd8ed1ab_0 conda-forge
tensorboard-plugin-wit 1.8.0 pyh44b312d_0 conda-forge
terminado 0.9.4 py38haa244fe_0 conda-forge
testpath 0.4.4 py_0 conda-forge
threadpoolctl 2.1.0 pyh5ca1d4c_0 conda-forge
tifffile 2021.4.8 pyhd8ed1ab_0 conda-forge
tk 8.6.10 h8ffe710_1 conda-forge
toml 0.10.2 pyhd8ed1ab_0 conda-forge
tomlkit 0.7.0 py38haa244fe_3 conda-forge
toolz 0.11.1 py_0 conda-forge
tornado 6.1 py38h294d835_1 conda-forge
tqdm 4.60.0 pyhd8ed1ab_0 conda-forge
traitlets 5.0.5 py_0 conda-forge
typing-extensions 3.7.4.3 0 conda-forge
typing_extensions 3.7.4.3 py_0 conda-forge
urllib3 1.26.4 pyhd8ed1ab_0 conda-forge
vc 14.2 hb210afc_4 conda-forge
vs2015_runtime 14.28.29325 h5e1d092_4 conda-forge
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
webencodings 0.5.1 py_1 conda-forge
werkzeug 1.0.1 pyh9f0ad1d_0 conda-forge
wheel 0.36.2 pyhd3deb0d_0 conda-forge
widgetsnbextension 3.5.1 pypi_0 pypi
win_inet_pton 1.1.0 py38haa244fe_2 conda-forge
wincertstore 0.2 py38haa244fe_1006 conda-forge
winpty 0.4.3 4 conda-forge
wrapt 1.12.1 py38h294d835_3 conda-forge
xmltodict 0.12.0 pypi_0 pypi
xz 5.2.5 h62dcd97_1 conda-forge
yaml 0.2.5 he774522_0 conda-forge
yarl 1.6.3 py38h294d835_1 conda-forge
zeromq 4.3.4 h0e60522_0 conda-forge
zfp 0.5.5 h0e60522_5 conda-forge
zipp 3.4.1 pyhd8ed1ab_0 conda-forge
zlib 1.2.11 h62dcd97_1010 conda-forge
zstd 1.4.9 h6255e5f_0 conda-forge
Here's the end of the output of "conda install numba":
The following specifications were found to be incompatible with your system:
- feature:/win-64::__win==0=0
- feature:|@/win-64::__win==0=0
- conda-forge/noarch::google-auth-oauthlib==0.4.1=py_2 -> click -> __unix
- conda-forge/noarch::google-auth-oauthlib==0.4.1=py_2 -> click -> __win
- conda-forge/noarch::jupyterlab==3.0.14=pyhd8ed1ab_0 -> ipython -> __linux
- conda-forge/noarch::jupyterlab==3.0.14=pyhd8ed1ab_0 -> ipython -> __win
- conda-forge/noarch::notebook==6.3.0=pyha770c72_1 -> ipykernel -> __linux
- conda-forge/noarch::notebook==6.3.0=pyha770c72_1 -> ipykernel -> __win
- conda-forge/noarch::urllib3==1.26.4=pyhd8ed1ab_0 -> pysocks[version='>=1.5.6,<2.0,!=1.5.7'] -> __unix
- conda-forge/noarch::urllib3==1.26.4=pyhd8ed1ab_0 -> pysocks[version='>=1.5.6,<2.0,!=1.5.7'] -> __win
- conda-forge/win-64::ipykernel==5.5.3=py38h43734a8_0 -> ipython[version='>=5.0'] -> __linux
- conda-forge/win-64::ipykernel==5.5.3=py38h43734a8_0 -> ipython[version='>=5.0'] -> __osx
- conda-forge/win-64::ipykernel==5.5.3=py38h43734a8_0 -> ipython[version='>=5.0'] -> __win
- conda-forge/win-64::jupyter_server==1.6.4=py38haa244fe_0 -> send2trash -> __linux
- conda-forge/win-64::jupyter_server==1.6.4=py38haa244fe_0 -> send2trash -> __win
- conda-forge/win-64::pysocks==1.7.1=py38haa244fe_3 -> win_inet_pton -> __win
- conda-forge/win-64::setuptools==49.6.0=py38haa244fe_3 -> wincertstore[version='>=0.2'] -> __win
- turagalab/noarch::decode==0.10.0=np119py_0 -> click -> __unix
- turagalab/noarch::decode==0.10.0=np119py_0 -> click -> __win
Your installed version is: 0
Note that strict channel priority may have removed packages required for satisfiability.
@cleterrier hmm, that's a good point. I don't think we can do anything about it in the released DECODE version, as conda states that the packages are, in fact, incompatible; probably because of some side requirements.
As a workaround you could save your localizations and then load them in a new environment with for the drift correction.
The .h5 files are fairly easy to read, you can just take a look at the DECODE .h5 reader.
In a future release I might be able to change the requirements and/or relax them, such that combining both DECODE and the drift correction is possible.
@Haydnspass thanks for the tip, I'm trying to set up an independent environment for this. I'm wondering how I can get the relevant variables from an emitter set (x,y,z coordinates) into arrays that are the entry format of COMET? Relevant import code is currently this:
import numpy as np
import pandas as pd
import io
filepath = 'C:/Users/chris/Desktop/test.csv'
data = pd.read_csv(filepath)
localizations = np.zeros((len(data['frame']), 4))
localizations[:, 0] = np.asarray(data['x [nm]'])
localizations[:, 1] = np.asarray(data['y [nm]'])
localizations[:, 2] = np.asarray(data['z [nm]'])
localizations[:, 3] = np.asarray(data['frame'])
frames = np.unique(localizations[:, -1])
n_frames = len(frames)
print(f"{filepath} import successful, {len(localizations[:, 0])} localizations, {n_frames} frames")
So basically I'd need a way to feed x,y,z coordinates as arrays after opening the h5 file. Is there a command for that?
I will have to look into that
@cleterrier This should put x, y, z, and frame index from a .h5 file into a Nx4 numpy array:
import h5py
filename = <path_to_emitters.h5>
with h5py.File(filename, "r") as f:
x, y, z = np.array(f["data"]["xyz"]).T
f = np.array(f["data"]["frame_ix"])
localizations = np.stack([x, y, z, f]).T
Does this help you?