Download speed way too slow

Beginners
1

qwen_image_edit_fp8_e4m3fn.IRFpfsqH.safetensors.part

This takes hours. 1.9Mb/sec for 19gb is in my opinion very frustrating.

Why is this so slow, and is there a solution for this?

1 Like
4

Downloading via HF CLI (with XET enabled) is usually faster than using a browser, though it’s rare for browser downloads to be significantly slow…

It’s possible your ISP or network route is imposing speed restrictions.

It’s slow because you’re fetching a single 20.4 GB model file over the Hub’s Xet backend, often via one browser stream, across a congested CDN path, while your disk must assemble many chunks. That combination bottlenecks network and I/O. The specific file is indeed ~20.4 GB and stored with Xet. (Hugging Face)

What’s happening

  • Huge artifact + Xet reconstruction. Xet splits the file and reassembles locally. Parallel writes favor SSDs; HDDs need sequential mode or they thrash. (Hugging Face)
  • Browser = single stream. CLI and multi-connection tools do better than a lone HTTP stream. (Hugging Face)
  • CDN region variance. Throughput varies by location. Users report slow or unstable transfers that improve when switching region/VPN. (Hugging Face Forums)
  • Observed caps. Community reports ceilings around ~10 MB/s on some routes or setups. (GitHub)

Fixes that actually speed it up

1) Use the HF CLI on SSD with Xet “high performance”

# Linux/macOS
pip install -U "huggingface_hub[cli]" hf_xet  # docs: https://huggingface.co/docs/huggingface_hub/en/guides/cli
export HF_HOME="/fast-ssd/.cache/huggingface" # docs: https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
export HF_XET_HIGH_PERFORMANCE=1              # docs: https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
# If your download disk is an HDD, prefer sequential writes:
# export HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY=1  # same env vars doc

# Pull exactly the diffusion weight into ComfyUI
hf download Comfy-Org/Qwen-Image-Edit_ComfyUI \
  --include "split_files/diffusion_models/qwen_image_edit_fp8_e4m3fn.safetensors" \
  --local-dir "/path/to/ComfyUI/models/diffusion_models"
# CLI --local-dir reference: https://huggingface.co/docs/huggingface_hub/en/guides/cli

Why this helps: Xet high-performance mode increases concurrent range gets and CPU usage; cache and Xet writes land on a faster SSD; CLI avoids single-stream browser limits. (Hugging Face)

Windows (PowerShell):

pip install -U "huggingface_hub[cli]" hf_xet  # CLI docs ↑
setx HF_HOME "D:\hf_cache"                    # env vars docs ↑
setx HF_XET_HIGH_PERFORMANCE 1                # env vars docs ↑
# For HDD targets only:
# setx HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY 1

hf download Comfy-Org/Qwen-Image-Edit_ComfyUI `
  --include "split_files/diffusion_models/qwen_image_edit_fp8_e4m3fn.safetensors" `
  --local-dir "C:\ComfyUI\models\diffusion_models"

2) If the CLI is still slow, try multi-connection HTTP (aria2c)

# Get the file’s “Copy download link” from the HF file page
# (example file page: https://huggingface.co/Comfy-Org/Qwen-Image-Edit_ComfyUI/blob/main/split_files/diffusion_models/qwen_image_edit_fp8_e4m3fn.safetensors)
aria2c -x16 -s16 -j4 -c "https://huggingface.co/Comfy-Org/Qwen-Image-Edit_ComfyUI/resolve/main/split_files/diffusion_models/qwen_image_edit_fp8_e4m3fn.safetensors?download=true"
# aria2/parallel guide: https://dev.to/susumuota/faster-and-more-reliable-hugging-face-downloads-using-aria2-and-gnu-parallel-4f2b

Users consistently report higher throughput with multi-connection pulls on large Hub assets. (DEV Community)

3) If Xet misbehaves on your route, test without it

# Temporarily force fallback from hf-xet
export HF_HUB_DISABLE_XET=1   # Windows: setx HF_HUB_DISABLE_XET 1
# Also consider bumping timeouts for flaky links
export HF_HUB_DOWNLOAD_TIMEOUT=60

These toggles are documented and help isolate whether the Xet path or your route is the culprit. (Hugging Face)

4) Try another region path

If speed is stuck, switch network egress (VPN/off-VPN or another POP). Reports show material gains after changing region. (Hugging Face Forums)

Sanity checks for your exact case

  • You are pulling the correct file and its size is expected: 20.4 GB Xet-backed diffusion weight. If you saw a .part name, that’s just a partial file while downloading. Use the direct file page to confirm size before retrying. (Hugging Face)
  • Use CLI or aria2c and verify that throughput jumps above your current ~1.9 MB/s. If not, try the VPN step and the Xet disable toggle. (Hugging Face)

References

  • Qwen Image/Edit ComfyUI files. Confirms file path and 20.4 GB size for the fp8 weights. (Hugging Face)
  • HF environment variables. High-performance Xet, sequential writes for HDDs, cache locations, Xet disable, and timeouts. (Hugging Face)
  • HF CLI usage, including --local-dir. (Hugging Face)
  • Reports of slow or unstable downloads and region effects. (Hugging Face Forums)
  • Observed caps around ~10 MB/s on some setups. (GitHub)
  • Multi-connection download recipe that users employ to improve throughput. (DEV Community)
1 Like
5

Thank you John.

Your post is well documented and informative.

My internet speed at home is 500mb/sec.

I still have issue’s with slow download speeds, but not all the time.

Thanks so far !

Greetings Michel.

1 Like
6

Well, for now, the method to set the HF CLI to the absolute fastest mode is currently as follows. Since the method for setting environment variables varies by OS, replace the export part according to your environment.

Below is a focused, step-by-step playbook that actually increases Hugging Face (HF) CLI download speed, with background for each step and concrete commands. The knobs and commands are current as of v1.1.x of huggingface_hub.

Background: why fast ISP ≠ fast model pulls

Large files on the Hub are served via a CDN and, by default, downloaded through the HF CLI using the Rust-based hf-xet path. Xet reconstructs files from chunks; this favors SSD/NVMe and high concurrency, and it exposes environment variables you can tune. Route/POP (region) variability also matters, so the same 500 Mb/s line can feel “very fast” or “very slow” depending on path and local I/O. (Hugging Face)

1) Put the cache on a fast SSD and update the tools

Why: Eliminates a common disk bottleneck and ensures you have recent CLI/Xet behavior. HF_HOME controls where the Hub cache lives. (Hugging Face)

# References:
# - HF env vars (HF_HOME, HF_HUB_CACHE, timeouts, Xet knobs): https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
# - CLI install/usage: https://huggingface.co/docs/huggingface_hub/en/guides/cli
python -m pip install -U "huggingface_hub" hf_xet
export HF_HOME="/fast-ssd/.cache/huggingface"     # move cache to SSD/NVMe
export HF_HUB_DOWNLOAD_TIMEOUT=60                 # reduce spurious timeouts on slow routes

HF_HOME sets the base cache dir; HF_HUB_DOWNLOAD_TIMEOUT raises the per-request timeout from the 10 s default to something more forgiving. (Hugging Face)

2) Turn on Xet “high performance” and set explicit concurrency

Why: The Xet backend exposes controls that directly impact throughput. High-perf mode tries to saturate network/CPU; the per-file concurrency (HF_XET_NUM_CONCURRENT_RANGE_GETS) increases parallel chunk reads. Defaults are conservative (16). (Hugging Face)

# References (HF_XET_HIGH_PERFORMANCE, HF_XET_NUM_CONCURRENT_RANGE_GETS):
# https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
export HF_XET_HIGH_PERFORMANCE=1
export HF_XET_NUM_CONCURRENT_RANGE_GETS=24   # try 24–32 on fast SSD/CPU; 8–16 on weaker machines

These variables are officially documented; raising concurrency helps if you have headroom. (Hugging Face)

3) If the destination is an HDD, force sequential writes

Why: Parallel random writes that help SSDs can thrash spinning disks. The Xet toggle below switches to sequential writes and is specifically recommended for HDDs. (Hugging Face)

# Reference: HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY (HDD-only)
# https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
export HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY=1

4) Download only what you need; use --include and --local-dir

Why: Avoids pulling entire repos and keeps writes local to your target folder. The CLI implements patterns and a direct “download-to-folder” mode. (Hugging Face)

# References:
# - CLI download patterns & --local-dir: https://huggingface.co/docs/huggingface_hub/en/guides/cli
# - Download guide (hf_hub_download/snapshot_download): https://huggingface.co/docs/huggingface_hub/en/guides/download
hf download Comfy-Org/Qwen-Image-Edit_ComfyUI \
  --include "split_files/diffusion_models/qwen_image_edit_fp8_e4m3fn.safetensors" \
  --local-dir "/path/to/ComfyUI/models/diffusion_models"

5) If speeds are spiky under the CLI, A/B test the transfer path

Why: Isolate whether Xet routing/behavior is the issue on your machine/route. You can explicitly disable Xet; set this before importing or invoking HF code so it’s honored. (Hugging Face)

# References:
# - HF_HUB_DISABLE_XET (must be set prior to import/CLI): https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
# - Note about setting it before use: https://github.com/huggingface/huggingface_hub/issues/3266
export HF_HUB_DISABLE_XET=1
hf download <repo> --include "<file>" --local-dir "<dir>"

6) Use a multi-connection HTTP downloader for one-off big pulls

Why: When a single TCP stream underperforms on your current path, multiple HTTP range connections often achieve higher steady throughput. aria2c is a simple, robust option. (DEV Community)

# Reference/recipe: https://dev.to/susumuota/faster-and-more-reliable-hugging-face-downloads-using-aria2-and-gnu-parallel-4f2b
# 1) Get the “Copy download link” (?download=true) from the file’s page on the Hub
aria2c -x16 -s16 -j4 -c \
 "https://huggingface.co/<repo>/resolve/main/<path/to/large-file>?download=true"
# -x connections per server; -s splits per file; -j parallel jobs; -c resume

7) Change network egress (region) when speeds are unusually low

Why: The Hub’s CDN and peering vary by POP and time. Toggling VPN on/off or switching to a nearby region often changes anemic 1–2 MB/s to tens of MB/s. Forum reports and timeout traces on cdn-lfs*.hf.co illustrate this effect. (Hugging Face Forums)

8) Know what’s deprecated (so you don’t chase the wrong lever)

Why: Many older posts recommend hf_transfer. In current huggingface_hub v1.x, hf_transfer is removed and HF_HUB_ENABLE_HF_TRANSFER is ignored. Use the Xet knobs (HF_XET_HIGH_PERFORMANCE, etc.) instead. (Hugging Face)

60-second diagnosis (copy/paste)

This quickly tells you where the bottleneck is on your system/route.

# 0) Prep
python -m pip install -U "huggingface_hub" hf_xet  # https://huggingface.co/docs/huggingface_hub/en/guides/cli
export HF_HOME="/fast-ssd/.cache/huggingface"      # https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables

# 1) CLI with tuned Xet (expect best performance on SSD)
export HF_XET_HIGH_PERFORMANCE=1                   # https://huggingface.co/docs/huggingface_hub/en/package_reference/environment_variables
export HF_XET_NUM_CONCURRENT_RANGE_GETS=24
hf download <repo> --include "<bigfile>" --local-dir "<dir>"

# 2) Same CLI, but Xet off (if #1 was spiky)
export HF_HUB_DISABLE_XET=1                        # set BEFORE running HF code
hf download <repo> --include "<bigfile>" --local-dir "<dir>"

# 3) Multi-connection HTTP (expect steady speeds if single-stream path is weak)
aria2c -x16 -s16 -j4 -c "<direct ?download=true URL>"  # https://dev.to/susumuota/faster-and-more-reliable-hugging-face-downloads-using-aria2-and-gnu-parallel-4f2b

Interpretation:
• #1 fast and stable → keep Xet high-perf + SSD.
• #1 spiky but #2 steady → Xet/route interaction on this box; run with Xet disabled here. (GitHub)
• Both slow but aria2c fast → single-stream/POP issue; keep CLI for convenience but prefer multi-connection pulls for huge files. (DEV Community)
• All slow → switch region (VPN/off-VPN) and retry. (Hugging Face Forums)

Common pitfalls (quick checks)

  • Downloading whole repos by accident. Use --include to pull only the large file(s) you need. (Hugging Face)
  • Tiny default timeouts. Raise HF_HUB_DOWNLOAD_TIMEOUT to reduce intermittent read timeouts on cdn-lfs*.hf.co. (Hugging Face)
  • HDD as destination. If you must, add HF_XET_RECONSTRUCT_WRITE_SEQUENTIALLY=1. (Hugging Face)
  • Expecting hf_transfer to help. It’s gone in v1.x; use Xet high-perf instead. (Hugging Face)

Short, curated references

Official docs

  • Environment variables (HF_HOME/HF_HUB_CACHE, timeouts, Xet knobs, disable Xet). Useful to confirm every variable above. (Hugging Face)
  • CLI guide (hf download, --include, --local-dir, timeouts). Step-by-step usage examples. (Hugging Face)
  • Migration note: hf_transfer removed; HF_HUB_ENABLE_HF_TRANSFER ignored; use HF_XET_HIGH_PERFORMANCE. (Hugging Face)

GitHub issues (real-world behavior)

  • “HF_HUB_DISABLE_XET not disabling unless set before import” — clarifies when the toggle takes effect. (GitHub)
  • “Large downloads stuck/slow with VPN” — intermittent slow/0% cases on big files with CLI. (GitHub)

Community threads/recipes

  • Intermittent slow speeds, disconnects, and region changes as fixes. Background on route/POP sensitivity. (Hugging Face Forums)
  • Practical aria2c recipe for multi-connection, resumable downloads. (DEV Community)
1 Like
7

It sounds like a lot of users are running into the same slowdown, so the issue is likely on HF’s end rather than your setup. Sometimes the CDN gets overloaded, especially during peak hours or large model releases. You can try switching to a different network or using a download manager, but honestly, the speed usually returns to normal after a while. Hopefully Hugging Face resolves the bottleneck soon.

1 Like