Hello there! This repository contains some demonstration code and comparisons for answering discrete problems you might encounter in your day to day work using both brute force methods in SQL and Redis, and using Probabilistic Data Structures in Redis. The problems we seek to evaluate are the following:
- Set Presence - Has an item been added to a set?
- Count of Items in a Stream - How many times has a given item been added in a stream?
- Heavy Hitters in a Stream - what are the most frequent items added to a stream
- Set Cardinality - How many unique items have been added to a set.
In this example, we break down the individual words in Herman Melville's Epic - "Moby Dick" and insert them into 4 data sources.
- An Unindexed Postgres table
- An Indexed Postgres table
- A Redis Sorted Set
- A set of Redis Probabilistic Data Structures Including a Bloom Filter, a TopK, a Count-Min-Sketch, and a HyperLogLog.
- The .NET 6 SDK
- Some means of running Redis Stack and Postgres (preferably with Docker)
Start up docker with docker-compose up
To run the App, simply run dotnet run