The ChunkedSet stores its elements in Chunks of size k. It behaves as a set and contains only unique values in all chunks combined.
The implementation works efficiently for any number of chunks.
ChunkedSet is implemented with 3 hash_tables. Each of which helps make it behave as a set of sets.
Data_table -
hash_table1 : key : datapoint
value : (chunk, slots)
Slot_table -
hash_table2 : key : slots
value : [list of chunks]
Node_table -
hash_table3 : key : chunks
value : datapoints in tuple
- Join : When a new chunk joins the ChunkedSet, all the duplicate values are deleted
- Leave : When a chunk leaves the ChunkedSet, all the values are migrated to the available slots in the rest of the chunks
- flask
- Clone the repo
git clone https://github.com/anaswaratrajan/Chunkedset.git
- Enter directory chunkedset
cd chunked set
- Install requirements
pip install -r requirements.txt
- Run the app
flask run
- Set data to chunk - POST
curl --header "Content-Type: application/json" \
--request POST \
--data '{"chunk":1,"data":[1,2,3,4,5]}' \
http://127.0.0.1:5000/chunk1/setdata
- Send join request to server - GET
http://127.0.0.1:5000/chunk1/join
- Send leave request to server - GET
http://127.0.0.1:5000/chunk1/leave
- Check server for updates
http://127.0.0.1:5000/server
Similar for other chunks ( chunk2, chunk3)