System Design

Relational database (joins) vs NoSQL database
NoSQL vs SQL
SQL: tables, foreign key, primary key, joins,
Scaling: a million users, solve auto-scaling groups?
Latency: how quickly will for example a post show up in a user's feed
Cloud storage: S3, Firebase Storage, .....
Status codes: 100, 200, 300, 400, 500
Load balancing
Request -> Response
HTTP vs Websockets
HTTP methods: GET, POST, DELETE, ....
Caching: pre-cache, background, (e.g. Redis, MemCache)
worker
job
bottle necks: solution maybe regions, geolocating, cdn (akamai)
CDN: support 10x traffic, cached locally so all users are grabbing it are grabbing it from the CDN
Regional data centers
API endpoints behind a CDN to scale traffic
5mb per video?? 1 minute video??
shards - helps split load between databases???
microservices - e.g load balancing etc....
object storage - data storage architecture: object, metadata, identifier
SQL database(support joins): PostGres, MySQL....
NoSQL(document based-similar to JSON): MongoDB, Cassandra.....
Reliability and Redundancy (Backup to prevent lost of data)
Least Recently Used (LRU) - reasonable cache eviction policy
eighty-twenty rule, i.e., 20% of daily read volume for photos is generating 80% of the traffic
message queue - asynchronous service to service communication used in serverless microservice architecture (e.g Kafka)

Sharding and partitioning are both about breaking up a large data set into smaller subsets. The difference is that sharding implies the data is spread across multiple computers while partitioning does not. Partitioning is about grouping subsets of data within a single database instance.

How can we build a more intelligent cache? If we go with the eighty-twenty rule, i.e., 20% of daily read volume for photos is generating 80% of the traffic, which means that certain photos are so popular that most people read them. This dictates that we can try caching 20% of the daily read volume of photos and metadata.