iterative/dvc

Gett Error with pushing large file (data.csv ~ 280MB size)

ammaryasirnaich opened this issue · 2 comments

ammaryasirnaich commented 
dvc --version = 3.43.1
Distributor ID:	Ubuntu
Description:	Ubuntu 22.04.4 LTS
Release:	22.04
Codename:	jammy

I am trying to push the dataset on dagshub, however i am getting this error for large files only.

024-02-28 17:40:21,970 ERROR: failed to transfer 'b5d26507afbb0' - 400, message='Bad Request', url=URL('https://dagshub.com/<user>/my-first-repo.dvc/files/md5/b5/d26507af46acc6b55a3326cf41fbb0')    

ERROR: failed to push data to the cloud - 2 files failed to upload           
Traceback (most recent call last):
  File "/.local/lib/python3.10/site-packages/dvc/commands/data_sync.py", line 64, in run
    processed_files_count = self.repo.push(
  File "/.local/lib/python3.10/site-packages/dvc/repo/__init__.py", line 58, in wrapper
    return f(repo, *args, **kwargs)
  File "/.local/lib/python3.10/site-packages/dvc/repo/push.py", line 167, in push
    raise UploadError(failed_count)
dvc.exceptions.UploadError: 2 files failed to upload

2024-02-28 17:40:21,979 DEBUG: Analytics is enabled.
2024-02-28 17:40:22,011 DEBUG: Trying to spawn ['daemon', 'analytics', '/tmp/tmp_83sw124', '-v']
2024-02-28 17:40:22,019 DEBUG: Spawned ['daemon', 'analytics', '/tmp/tmp_83sw124', '-v'] with pid 607

Any suggestion or help would be appreciated

shcheklein commented

@ammaryasirnaich unfortunately I think you need to ask Dagshub folks for some help on this. They have their own implementation of the remote storage and night be related to that (can be some more generic as well though).

  • 👍1
ammaryasirnaich commented

@ammaryasirnaich unfortunately I think you need to ask Dagshub folks for some help on this. They have their own implementation of the remote storage and night be related to that (can be some more generic as well though).

Many thanks for the response 👍