HF_HUB_ENABLE_HF_TRANSFER=1 - last few hundred MBs becomes so so so slow

[FurkanGozukara]

Describe the bug

It downloads with 100MB+ the early part

but as gets closer to the end it becomes way way slower 3 mb per/second and it destroys the whole purpose of

HF_HUB_ENABLE_HF_TRANSFER=1

What could be reason ?

Download method

snapshot_download(local_dir=current_dir,repo_id=repo_id)

Reproduction

No response

Logs

No response

System info

(venv) G:\Joy_Caption_v23\venv\Scripts>huggingface-cli env

Copy-and-paste the text below in your GitHub issue.

- huggingface_hub version: 0.26.2
- Platform: Windows-10-10.0.22631-SP0
- Python version: 3.10.11
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Running in Google Colab Enterprise ?: No
- Token path ?: C:\Users\Furkan\.cache\huggingface\token
- Has saved token ?: True
- Configured git credential helpers: manager
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.4.1+cu124
- Jinja2: 3.1.4
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 11.0.0
- hf_transfer: 0.1.8
- gradio: 5.6.0
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: 2.10.1
- aiohttp: N/A
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: C:\Users\Furkan\.cache\huggingface\hub
- HF_ASSETS_CACHE: C:\Users\Furkan\.cache\huggingface\assets
- HF_TOKEN_PATH: C:\Users\Furkan\.cache\huggingface\token
- HF_STORED_TOKENS_PATH: C:\Users\Furkan\.cache\huggingface\stored_tokens
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

[Wauplin]

Hi @FurkanGozukara I'm sorry you're facing this but unfortunately there can be a lot of reasons for this to happen and it'll probably be hard to investigate. Maybe your network gets limited by your provider? Maybe it's a temporary outage on the network? Maybe it's an error in the estimation by tqdm? Maybe another process started on your machine in the meatnime? etc. In any case, I can assure you it's not a deliberate limitation server-side and it's not a problem that has ever been reported for hf_transfer.

If you find a way to repeatedly reproduce it, I'd be interested to investigate it though

[FurkanGozukara]

@Wauplin ty for reply