pull: _get_file() got an unexpected keyword argument 'max_concurrency'
BradyJ27 opened this issue · 4 comments
Bug Report
Issue name
pull: _get_file() got an unexpected keyword argument 'max_concurrency'
Description
When running dvc pull
, I am getting the following error:
ERROR: failed to transfer 'da9e3e6905fddf1bd672718f26702bb4' - _get_file() got an unexpected keyword argument 'max_concurrency'
Traceback (most recent call last):
File "/home/runner/.venv/lib/python3.8/site-packages/dvc_objects/executors.py", line 135, in batch_coros
result = fut.result()
File "/home/runner/.venv/lib/python3.8/site-packages/dvc_objects/fs/generic.py", line 219, in _get_one_coro
return await get_coro(
File "/home/runner/.venv/lib/python3.8/site-packages/dvc_objects/fs/callbacks.py", line 140, in async_wrapper
res = await fn(*args, **kwargs)
File "/home/runner/.venv/lib/python3.8/site-packages/dvc_objects/fs/callbacks.py", line 159, in async_wrapper
return await fn(path1, path2, **kwargs)
TypeError: _get_file() got an unexpected keyword argument 'max_concurrency'
Reproduce
- pip install dvc[s3]
- dvc pull
Expected
Dvc pull should pull files without errors.
Environment information
I am running inside of a docker container on AWS Linux. Python 3.8, dvc 3.42.0 with no other packages installed. The only installation I ran was pip install dvc[s3]
in a fresh environment. I am thinking this is happening due to some weird upstream dependency that got updated, but I am unsure.
Output of dvc doctor
:
DVC version: 3.42.0 (pip)
-------------------------
Platform: Python 3.8.10 on Linux-4.14.336-257.562.amzn2.x86_64-x86_64-with-glibc2.29
Subprojects:
dvc_data = 3.8.0
dvc_objects = 3.0.6
dvc_render = 1.0.1
dvc_task = 0.3.0
scmrepo = 2.0.4
Supports:
http (aiohttp = 3.9.3, aiohttp-retry = 2.8.3),
https (aiohttp = 3.9.3, aiohttp-retry = 2.8.3),
s3 (s3fs = 2024.3.0, boto3 = 1.34.51)
Config:
Global: /root/.config/dvc
System: /etc/xdg/dvc
Cache types: <https://error.dvc.org/no-dvc-cache>
Caches: local
Remotes: s3
Workspace directory: overlay on overlay
Repo: dvc, git
Repo.site_cache_dir: /var/tmp/dvc/repo/0dbc255b3ab01d2339a3c6032dfca3f9
Additional Information (if any):
One thing I noticed was that there is no cache types (seen in output of dvc doctor), I have tried both mkdir .dvc/cache
and dvc config cache.type hardlink,symlink
but to no avail.
I am running into the same issue. I noticed that on Python 3.8 it downloads an older version 3.42. On Python 3.9 it downloads the latest version. The repo badges indicate that DVC is 3.9+, but the docs say it's 3.8+.
Duplicate of iterative/dvc-s3#80. This has been fixed upstream in s3fs, and we are waiting for a new release. For now, please pin s3fs to <=2024.2
.
Thanks! Sorry for the duplicate.
s3fs==2024.3.1
was released with a fix.