A collection of my expolorations with CUDA on my NVIDIA Jetson Orin Nano 8gb Dev Kit
TO-DO:
- cuda-gdb exploration
- computer sanitizer
- set up nsight systems and compute from remote
- nvidia nvtx
- Use cuBLAS/CUTLASS to implement various numerical linear algebra operations/algorithms
- matrix inversion
- norms
- l1 norm
- l2 norm
- infinite norm
- frobenius norm
- SVD
- QR Factorization Algorithm
- Graham-Schmidt Ortogonalization
- Least-Squares Algorithms
- Arnoldi Iteration
- GMRES
- attention kernels
- standard attention
- ring attention
- group query attention
- latent multi-head attention
In Progress:
- SGEMM optimization based on https://siboehm.com/articles/22/CUDA-MMM
- er
Completed: