ipfs/ipget

Directory resource: not possible to resume

Closed this issue · 6 comments

cryptotester commented
  • Try to download a resource directory
  • Wait for some progress, so that inside the local directory some files have been downloaded
  • Press Ctrl-C to stop
  • Run the command again

Result: ipget tells that it can't overwrite existing local files

Expected result: it should resume and download what's new, files not fully downloaded should either resume or being re-downloaded.

welcome commented

Thank you for submitting your first issue to this repository! A maintainer will be here shortly to triage and review.
In the meantime, please double-check that you have provided all the necessary information to make this process easy! Any information that can help save additional round trips is useful! We currently aim to give initial feedback within two business days. If this does not happen, feel free to leave a comment.
Please keep an eye on how this issue will be labeled, as labels give an overview of priorities, assignments and additional actions requested by the maintainers:

  • "Priority" labels will show how urgent this is for the team.
  • "Status" labels will show if this is ready to be worked on, blocked, or in progress.
  • "Need" labels will indicate if additional input or analysis is required.

Finally, remember to use https://discuss.ipfs.io if you just need general support.

Stebalien commented

If you have a full IPFS node installed, ipget will use it and the chunks will still be available in your main repo (so you can resume).

Otherwise, this is technically possible but difficult as there are different ways to "chunk" files, resulting in different CIDs. You'd need to:

  1. Re-start the download, re-downloading the indirect (not data) blocks.
  2. Before downloading a block, use the offset/size (usually included in the indirect blocks) to check to see if that chunk of the file matches.

However, this would require some new traversal logic. I'd suggest implementing a feature like this in a fork although I can't guarantee that someone will be able to review your code in the near future.

cryptotester commented

Thanks for the answer. I could already sort my issue using not a full node, but IPFS Desktop + a script in python to download from localhost:8080 (ipfs desktop) so I assume this could work also using IPFS Desktop from what you know?

Stebalien commented

Yes. If the node is running and ipget can find it, it'll use it.

guseggert commented

This is difficult enough that we're not going to prioritize it.

stdedos commented

Closed as incomplete?