Deploying huge numbers of files (> 10k)
Closed this issue · 3 comments
- Do you want to request a feature or report a bug?
Feature
- What is the current behavior?
CLI is super slow for huge amount of pages.
- Please mention your node.js, and operating system version.
Node 10, Mac latest
I'd love to see parallel uploading with multiple threads implemented. Node workers are fairly easy to setup and could make sure that maximum resources are used for deployment.
Currently, deployment is unusable for that many pages.
Uploads are performed in parallel (up to 15 files at a time, which helps improve speeds for lots of small files).
Initial uploads can be a bit slow the first time, but subsequent deploys should be much faster due to content de-duplication.
Since this is no issue with CLI but with the js client, I'll close this.
It just took 27 minutes to upload 12k files. Is that considered fast? It's not fast in my opinion