[API Enhancement] Project Simplification for those who is new to Common Lisp
hikettei opened this issue · 1 comments
hikettei commented
- TODO
- Step-by-step tutorial for those who aren't familiar with Deep Learning.
- Tutorial written in Jupyter Notebook
- Translate
tutorial_jp.lisp
into English - Visualized Explanation of cl-waffe2 (should i make a slide?)
- I think
deftrainer
usage should be much more documented, because it is unique and complicated.
hikettei commented
We need to simplify the list of instructions needed for backend extensions:
Cheat sheet for backend extension
A backend which is completely separated from the standard implementation can be established by implementing all of the following instructions:
;; Arithmetic
AddNode SubNode MulNode DivNode
ScalarAdd ScalarSub ScalarMul ScalarDiv
InverseNode
;; [TODO] should be reduced to =>
AddNode SubNode MulNode DivNode
;; Matrix Transformation
MoveTensorNode
MoveTensorNode but JITxxxBackend dedicated to arithmetic ops for fast permute computation.
;; Mathematical Functions
SinNode CosNode TanNode ...
;; Logical Operations
;; Neural Network specialized instructions
Im2ColNode
Col2ImNode
To put it bluntly, instructions needs to be reimplemented are divided to three types:
-
CFFI Interface Binding (generated by call-with-view)
- BLAS APIS (axpy, gemm etc...)
- CFFI Interface like cl-bmas
-
JIT Backend Extension just following the instruction of
solve-loop-order
- Example: JITCPUTensor
- almost 300 line JIT compiler works enough, just used to optimize the layout of memory. Example: JITCPUTensor
-
Device-specific (Neural Network) operations
- e.g.: Im2Col, Col2Im ReLUNode(optional)
Our goal is: just 1000 line CUDA Extension