be url rather than file centric
mccalluc opened this issue · 1 comments
mccalluc commented
Consider passing in URLs instead of paths: Right now there are actually two passes, first downloading the data, and then reading the files in. There's no reason to keep the files on disk since we really just need them in memory.
mccalluc commented
.... but, that would probably make development more awkward? Right now when you develop with local files you're working in a kernel, and the behavior is the same as if it came from a url originally.
Closing, but reconsider if we see disk space becoming an issue. @scottx611x : sound ok?