datonic/datadex

Improve integration between Parquet and IPFS

Closed this issue · 1 comments

davidgasquez commented
  • Partition parquet files in smaller chunks to fit with IPLD
  • Use a trickle DAG: ipfs add --cid-version 1 --raw-leaves -r --trickle
davidgasquez commented

This won't scale, since it puts more weight on users. It might be interesting to explore it in a different setting, a decentralized data warehouse on top of IPFS.