Big Dataset Support
Opened this issue · 1 comments
matheus23 commented
Summary
Widening the possible use cases to include bigger data sets.
Problem / Need / Opportunity
Due to practical IPFS block size restrictions, WNFS can only safely encode up to 1000-2000 directory entries in public and private directories.
Success Scenario
It's possible to create huge directories! We enable use cases like on-boarding bigger data sets comparable to all of wikipedia, the linux kernel or a nix cache.
Children
ETA: 2023-09-30
Notes
RangerMauve commented
Would be cool to talk about Prolly Trees around when this happens. We're hoping to have some tests figured out by april on prolly trees to see how the performance works.