KhronosGroup/glTF-Sample-Models

Alternative download method like bittorrent or archive.org?

travisfw opened this issue · 11 comments

Github isn't really the best place to host big files. (Because git isn't designed for big files.) I understand the convenience, but I've had a few git clones of this repo fail, and am now waiting (it's been over an hour) for a zip to download. I don't have any idea when it will finish because the size is unknown.
I suspect the licensing on these public assets would make the Internet Archive a good home for them. I see there are already some KhronosGroup collections there.
Why not glTF samples too? The Archive generates a .torrent for collections "automagically".

So I got confirmation that Khronos already has a paid GitHub account for hosting Git LFS.

What do folks think about moving some of the heavy lifting on this repo to LFS?

Pros

  • Can add lots of huge new "PBR Next" type sample and test models.
  • We can still review new sample model submissions via the normal GitHub PR system.
  • Don't need to download a large Git history to each client (But would we reset the repo for this? Make a new LFS repo?)

Cons

  • Users have to install and learn LFS... It's not that bad, but it's an extra command to remember to run.
  • Might have to move this repo to some sort of archive, and start with a blank repo, to get the size benefits in full.

We just had exactly the same discussion on the model-viewer repro (it's huge because of golden render images). I was also thinking of rewriting the history to LFS, but then someone mentioned you can just remind people to git clone depth=1, which skips downloading the history. At that point it's nearly identical to LFS, but without any effort. I would just put that information prominently in the README.

While we're at it, we should re-org the layout here. How do people feel about...

  • Organizational folders by category, such as:

    • Minimal example models, to help with early implementations
    • High-detail, complex example models, to test art of the possible
    • Feature test models, to test individual glTF features in engines
    • Separate core-only models from ones using extension(s)?
  • Getting rid of 1.0 samples?

Organizational folders by category...

I think I'd prefer tags in the index .json file or a naming convention, and not additional subfolders... It is easier to find things with flatter structure. Something similar to the structure in the readme here, perhaps: https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0#gltf-20-sample-models

Getting rid of 1.0 samples?

No objection — we could put these on a branch, maybe?


It's a bit jankier than anything I'd want to advertise on the homepage, but I usually download things with https://kinolien.github.io/gitzip/. You can just paste the GitHub URL of a subfolder and it'll download a .zip of that specific model. Works well as long as there aren't ~hundreds of files in the subfolder, at which point it's better to just clone the repo.

I think I'd prefer tags in the index .json file or a naming convention

I suppose each model could have a "category" listed in its documentation, such that the generator (Python script) could find it and tag it in model-index.json. Maybe that generator could even rebuild the README with that info while it's at it.

I think we should be careful to pick categories that don't overlap (or as little as possible). I think a generic tagging system, with multiple tags per model, is too free-form for what these models are being used for.

@cx20 One option for you, if we do switch to Git LFS, is you could have gltf-test import this repo as a git submodule. This would let you reference all of the official samples from wherever they're hosted, even if it's the paid Khronos Git LFS hosting. You'd still have the option to include some of your own models to test, by keeping them in a normal Git folder alongside (but not inside) the submodule.

cx20 commented

@emackey Thank you for the information.

This would let you reference all of the official samples from wherever they're hosted, even if it's the paid Khronos Git LFS hosting.

That would be great. However, if I go LFS, will I still be able to publish in GitHub Pages?
I don't know if that's possible, as I don't know enough about LFS to know.


The gltf-test uses the models from this repository in GitHub Pages, but lately I'm seeing more and more GitHub Pages builds fail, probably because of the large repository size. I've found that if I retry or take some time to try, it seems to work.
Even if the LFS repository can be used as a submodule, I think that issuing GitHub Pages will fail if the size is still large.

You're right, looks like GitHub Pages doesn't support Git LFS at all, and there have been issues opened on that. That seems like a show-stopper issue unless a solution can be found.

cx20 commented

I agree with @donmccurdy's comments. #272 (comment)

I tried to separate the folders by category in gltf-test, For example, sampleModels, tutorialModels, etc. but quickly realized it was not a good idea. However, I can't change the folder configuration flatly because I think changing the configuration would create a massive repository history.

Just a thought, but it might be useful to be able to search for samples and download the results in one place, like glTF-Project-Explorer.
For example, I thought it might be useful for us to be able to type in an glTF extension name or keyword and download only the samples we want.

DRx3D commented

There have been no comments in the last 3 years. Since then the new Sample Assets repo has addressed many (most?) of these issues. If there is anything else that needs to be addressed, please open an issue in Sample Assets and reference this issue.

This issue will be closed by 27 Nov.

Given the existence of glTF-Sample-Assets, this issue is OBE. There are still downsides to keeping these files in a git repo, but for now it appears Khronos is moving forward with the new repo.