How your computer works, why you should care, and how that will change in the future.
Intro
This started with a post I made on FunFunForum regarding how immutability is a very scalable approach to parallelism from a hardware perspective, and some people seemed to want a more thorough explanation of microarchitecture. So that's what this is.
I decided not to post this directly to FunFunForum though, as it's behind a paywall. If I'm going to help people with this information, I think it's best to make it freely available. I considered putting it on a blog, but decided Github would be better, as it would be more flexible and would allow for others more knowledgeable than me to contribute.
The explicit goal is this: Explain how your computer works (CPU, GPU, RAM, etc.), how this affects programming, and how that will change in the future.
So here's my plan: this will start out as a relatively brief explanation of microarchitecture, just going into the basics. However, much like wikipedia, this will be more of a collection of articles to provide more information on more specific subjects. Each article will be strewn with links to other articles where more information can be found.
Unlike wikipedia though, this will be more stripped down and to-the-point, more directed toward programmers, and trying to keep relavent to the overall goal of keeping to the aforementioned goal.
At the moment, the following is the Table of Contents. Most of these articles have not been started yet, but I'll get to them as I can. Also, due to the intended nonlinear nature of this (I don't know what exactly to call this), this will be more of a list of subjects covered with links to those that have been completed. Feel free to suggest more subjects or contribute. Just be aware that this will take some time to complete. Articles with "(?)" afterward are ones I'm not sure about doing, as I don't know how interested people will be in them.
Table of Contents
(Note: Some articles may be mentioned multiple times due to the nonlinear structure of this.)
Introduction (You are Here)
Misc.
- Why You Should Care About Hardware
- Clock Speed v. Core Count
- Amdahl's Law
- Major Bottlenecks
- Guidelines on Optimization
- Bits and Bytes
Scalability and Communication
- SMP vs. NUMA
- The Problem of Cache Coherency
- Advantages and Disadvantages of Immutability
- Atomic Operations
- Parallel Reads and Sequential Writes
- Uses of Parallel Writes
- Cache Oblivious Algorithms
Memory
- Why You Should Care About Memory
- RAM
- Common Types of Memory
- Why Memory Layout is Important
- The Stack and Heap
- Bunches of Bytes
- Memory Alignment
- Memory Bandwidth, Bus Size, and Latency
- Manual Memory Management
- Automatic Reference Counting
- Garbage Collection
- Why Caches
- Caches
- The Problem of Cache Coherency
- SMP vs. NUMA
- Virtual Memory
- Caches v. Scratchpad v. Register Files
- Prefetching
- Future of Memory
CPUs
- Why You Should Care About the CPU
- Clock Speed v. Core Count
- CPU Fundamentals
- Basics of Assembly
- Logic Gates
- Pipelines and Execution
- Branch Prediction
- Interacting With Memory
- Common Types of Memory
- Prefetching
- RISC v. CISC
- IPC and ILP
- Superscalar Processors
- SMT / Hyperthreading
- Out-of-Order Execution
- SIMD v. MIMD
- SMP vs. NUMA
- Heterogeous Architectures
- Instruction v. Data Caches
- Tiled Architectures (?)
- VLIW : From DSPs to Itanium (?)
- Stack Machines (?)
- FPGAs (?)
- Belt Machines (?)
- Future of CPUs
GPUs
- Why You Should Care About the GPU
- Clock Speed v. Core Count
- GPU caches v. CPU caches
- SIMD v. MIMD
- GPU Register Files
- Common Types of Memory
- Caches v. Scratchpad v. Register Files
- Basics of a Graphics Pipeline
- Future of GPUs
Programming This Crazy Hardware
- Minimizing Cache Misses
- Working with the Prefetcher
- Cache-Oblivious Programming
- Data Oriented Design
- Intro to Bit Twiddling : Crazy Speed Hacks (?)
The Compiler
- Various Optimizations
- Intro to Substructural Type Systems
- SSA Form : Functional Assembly
- SIMD Optimizations (?)
The Future of Hardware
- Manycore Systems
- Quantum Computing
- Heterogeous Architectures
- Tiled Architectures
- 2.5D and 3D Architectures
- Far-Future Computing Tech
Analysis of Architectures (?)
- Architecture Analysis : x86 Overview (?)
- Architecture Analysis : AMD's Zen (?)
- Architecture Analysis : Intel Xeon Phi (?)
- Architecture Analysis : ARM big.LITTLE (?)
- Architecture Analysis : HPE's Machine (?)
- Architecture Analysis : Intel's Itanium (?)
- Architecture Analysis : Adapteva's Epiphany (?)
- Architecture Analysis : The Mill (?)