torchdata not compatible with torch 2.3.0
mmdanziger opened this issue ยท 3 comments
๐ Describe the bug
Describe the bug
torchdata
does not work with torch 2.3.0 because DILL_AVAILABLE
is not available where expected:
Python 3.12.2 (main, Feb 6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torchdata
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/private/tmp/test/lib/python3.12/site-packages/torchdata/__init__.py", line 9, in <module>
from . import datapipes
File "/private/tmp/test/lib/python3.12/site-packages/torchdata/datapipes/__init__.py", line 9, in <module>
from . import iter, map, utils
File "/private/tmp/test/lib/python3.12/site-packages/torchdata/datapipes/iter/__init__.py", line 79, in <module>
from torchdata.datapipes.iter.util.cacheholder import (
File "/private/tmp/test/lib/python3.12/site-packages/torchdata/datapipes/iter/util/cacheholder.py", line 24, in <module>
from torch.utils.data.datapipes.utils.common import _check_unpickable_fn, DILL_AVAILABLE
ImportError: cannot import name 'DILL_AVAILABLE' from 'torch.utils.data.datapipes.utils.common' (/private/tmp/test/lib/python3.12/site-packages/torch/utils/data/datapipes/utils/common.py)
>>>
This was fixed in pytorch/pytorch#122616 but the fix did not make it into the torch 2.3.0 release
https://github.com/pytorch/pytorch/blob/v2.3.0/torch/utils/data/datapipes/utils/common.py
I would assume 2.3.1 will be ok because the fix is in main
. For now though, a clean install of torchdata
is broken.
I would recommend adding a torch>=2,!=2.3.0
to setup.py
To Reproduce
pip install torchdata
import torchdata
Expected behavior
successful import
Environment
fresh install
Versions
Collecting environment information...
PyTorch version: 2.3.0
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: macOS 14.4.1 (arm64)
GCC version: Could not collect
Clang version: 15.0.0 (clang-1500.3.9.4)
CMake version: version 3.27.1
Libc version: N/A
Python version: 3.12.2 (main, Feb 6 2024, 20:19:44) [Clang 15.0.0 (clang-1500.1.0.2.5)] (64-bit runtime)
Python platform: macOS-14.4.1-arm64-arm-64bit
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Apple M1 Max
Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] torch==2.3.0
[pip3] torchdata==0.7.1
[conda] Could not collect
I encountered the same issue. Is there any ETA to merge the fix into torch 2.3?
We'd also need this as soon as feasible for rolling out pytorch 2.3 support in conda-forge.
This is duplicate of pytorch/pytorch#124907