hjinnkim/awesome-kan

A comprehensive collection of KAN(Kolmogorov-Arnold Network)-related resources, including libraries, projects, tutorials, papers, and more, for researchers and developers in the Kolmogorov-Arnold Network field.

Awesome KAN(Kolmogorov-Arnold Network)

A curated list of awesome libraries, projects, tutorials, papers, and other resources related to Kolmogorov-Arnold Network (KAN). This repository aims to be a comprehensive and organized collection that will help researchers and developers in the world of KAN!

Table of Contents

• Papers
  • Theorem
• Library
  • ConvKANs
  • Benchmark
• Project
• Discussion
• Tutorial
  • YouTube
• Contributing

Papers

• KAN: Kolmogorov-Arnold Networks : Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs). While MLPs have fixed activation functions on nodes ("neurons"), KANs have learnable activation functions on edges ("weights"). KANs have no linear weights at all -- every weight parameter is replaced by a univariate function parametrized as a spline. We show that this seemingly simple change makes KANs outperform MLPs in terms of accuracy and interpretability. For accuracy, much smaller KANs can achieve comparable or better accuracy than much larger MLPs in data fitting and PDE solving. Theoretically and empirically, KANs possess faster neural scaling laws than MLPs. For interpretability, KANs can be intuitively visualized and can easily interact with human users. Through two examples in mathematics and physics, KANs are shown to be useful collaborators helping scientists (re)discover mathematical and physical laws. In summary, KANs are promising alternatives for MLPs, opening opportunities for further improving today's deep learning models which rely heavily on MLPs.
• Chebyshev Polynomial-Based Kolmogorov-Arnold Networks : The document discusses a novel neural network architecture called the Chebyshev Kolmogorov-Arnold Network (Chebyshev KAN) for approximating complex nonlinear functions. It combines the theoretical foundations of the Kolmogorov-Arnold Theorem with the powerful approximation capabilities of Chebyshev polynomials. The Chebyshev KAN layer approximates a target multivariate function by representing it as a weighted sum of Chebyshev polynomials, leveraging the Kolmogorov-Arnold Theorem's guarantee of the existence of a superposition of univariate functions to represent any continuous multivariate function. The paper provides a detailed mathematical explanation of the implementation, training, and optimization of the Chebyshev KAN layer, as well as experimental results demonstrating its effectiveness in approximating complex fractal-like functions.
• TKAN: Temporal Kolmogorov-Arnold Networks ｜ code ｜
• Kolmogorov-Arnold Networks (KANs) for Time Series Analysis

Theorem

• 2009-On a constructive proof of Kolmogorov’s superposition theorem
• 2021-The Kolmogorov-Arnold representation theorem revisited
• 2021-The Kolmogorov Superposition Theorem can Break the Curse of Dimension When Approximating High Dimensional Functions

Library

• pykan : Offical implementation for Kolmogorov Arnold Networks ｜
• efficient-kan : An efficient pure-PyTorch implementation of Kolmogorov-Arnold Network (KAN). ｜
• FastKAN : Very Fast Calculation of Kolmogorov-Arnold Networks (KAN) ｜
• FasterKAN : FasterKAN = FastKAN + RSWAF bases functions and benchmarking with other KANs. Fastest KAN variation as of 5/13/2024, 2 times slower than MLP in backward speed. ｜
• TorchKAN : Simplified KAN Model Using Legendre approximations and Monomial basis functions for Image Classification for MNIST. Achieves 99.5% on MNIST using Conv+LegendreKAN. ｜
• FourierKAN : Pytorch Layer for FourierKAN. It is a layer intended to be a substitution for Linear + non-linear activation |
• ChebyKAN : Kolmogorov-Arnold Networks (KAN) using Chebyshev polynomials instead of B-splines. ｜
• GraphKAN : Implementation of Graph Neural Network version of Kolmogorov Arnold Networks (GraphKAN) ｜
• FCN-KAN : Kolmogorov–Arnold Networks with modified activation (using fully connected network to represent the activation) ｜
• X-KANeRF : KAN based NeRF with various basis functions like B-Splines, Fourier, Radial Basis Functions, Polynomials, etc ｜
• Large Kolmogorov-Arnold Networks : Variations of Kolmogorov-Arnold Networks (including CUDA-supported KAN convolutions) ｜
• FastKAN : Very Fast Calculation of Kolmogorov-Arnold Networks (KAN) ｜
• xKAN : Kolmogorov-Arnold Networks with various basis functions like B-Splines, Fourier, Chebyshev, Wavelets etc ｜
• JacobiKAN : Kolmogorov-Arnold Networks (KAN) using Jacobi polynomials instead of B-splines. ｜
• kan-polar : Kolmogorov-Arnold Networks in MATLAB ｜
• GraphKAN : Implementation of Graph Neural Network version of Kolmogorov Arnold Networks (GraphKAN) ｜
• OrthogPolyKAN : Kolmogorov-Arnold Networks (KAN) using orthogonal polynomials instead of B-splines. ｜
• kansformers : Kansformers: Transformers using KANs |
• Deep-KAN: Better implementation of Kolmogorov Arnold Network |
• RBF-KAN: RBF-KAN is a PyTorch module that implements a Radial Basis Function Kolmogorov-Arnold Network |
• KolmogorovArnold.jl : Very fast Julia implementation of KANs with RBF and RSWAF basis. Extra speedup is gained by writing custom gradients to share work between forward and backward pass. ｜

ConvKANs

• Convolutional-KANs : This project extends the idea of the innovative architecture of Kolmogorov-Arnold Networks (KAN) to the Convolutional Layers, changing the classic linear transformation of the convolution to non linear activations in each pixel. ｜
• Conv-KAN : This repository implements Convolutional Kolmogorov-Arnold Layers with various basis functions. ｜
• convkan : Implementation of convolutional layer version of KAN (drop-in replacement of Conv2d) ｜
• KA-Conv : Kolmogorov-Arnold Convolutional Networks with Various Basis Functions (Optimization for Efficiency and GPU memory usage) |

Benchmark

• KAN-benchmarking : Benchmark for efficiency in memory and time of different KAN implementations. |

Alternative

• high-order-layers-torch : High order piecewise polynomial neural networks using Chebyshev polynomials at Gauss Lobatto nodes (lagrange polynomials). Includes convolutional layers as well HP refinement for non convolutional layers, linear initialization and various applications in the linked repos with varrying levels of success. Euler equations of fluid dynamics, nlp, implicit representation and more |

Project

• KAN-GPT : The PyTorch implementation of Generative Pre-trained Transformers (GPTs) using Kolmogorov-Arnold Networks (KANs) for language modeling ｜
• KAN-GPT-2 : Training small GPT-2 style models using Kolmogorov-Arnold networks.(despite the KAN model having 25% fewer parameters!). ｜
• KANeRF : Kolmogorov-Arnold Network (KAN) based NeRF ｜
• Simple-KAN-4-Time-Series : A simple feature-based time series classifier using Kolmogorov–Arnold Networks ｜
• kanrl : Kolmogorov-Arnold Network for Reinforcement Leaning, initial experiments ｜
• kan-diffusion : Applying KANs to Denoising Diffusion Models with two-layer KAN able to restore images almost as good as 4-layer MLP (and 30% less parameters). ｜
• KAN4Rec : Implementation of Kolmogorov-Arnold Network (KAN) for Recommendations ｜
• X-KANeRF : X-KANeRF: KAN-based NeRF with Various Basis Functions to explain the the NeRF formula ｜
• KAN4Graph : Implementation of Kolmogorov-Arnold Network (KAN) for Graph Neural Networks (GNNs) and Tasks on Graphs ｜
• ImplicitKAN : Kolmogorov-Arnold Network (KAN) as an implicit function for images and other modalities ｜

Discussion

• KAN Hacker news discussion
• Can Kolmogorov–Arnold Networks (KAN) beat MLPs?
• Twitter thinks they killed MLPs. But what are Kolmogorov-Arnold Networks?
• [D] Kolmogorov-Arnold Network is just an MLP

Tutorial

• KAN Author's twitter introduction
• A Simplified Explanation Of The New Kolmogorov-Arnold Network (KAN) from MIT
• A from-scratch implementation of Kolmogorov-Arnold Networks (KAN)…and MLP | GitHub Code
• team-daniel/KAN : Implementation on how to use Kolmogorov-Arnold Networks (KANs) for classification and regression tasks.｜
• vincenzodentamaro/keras-FastKAN : Tensorflow Keras implementation of FastKAN Kolmogorov Arnold Network｜
• Official Tutorial Notebooks
• imodelsX examples with KAN : Scikit-learn wrapper for tabular data for KAN (Kolmogorov Arnold Network)
• What is the new Neural Network Architecture?(KAN) Kolmogorov-Arnold Networks Explained
• KAN: Kolmogorov–Arnold Networks — A Short Summary
• What is the significance of the Kolmogorov axioms for Mathematical Probability?
• Andrey Kolmogorov — one of the greatest mathematicians of the XXth century
• Unpacking Kolmogorov-Arnold Networks : Edge-Based Activation: Exploring the Mathematical Foundations and Practical Implications of KANs

YouTube

• KAN: Kolmogorov-Arnold Networks | Ziming Liu(KAN Author)
• Kolmogorov-Arnold Networks: MLP vs KAN, Math, B-Splines, Universal Approximation Theorem
• Didn't Graduate Guide to: Kolmogorov-Arnold Networks
• 超越谷歌DeepMind的最新大作：KAN全网最详细解读！
• Kolmogorov Arnold Networks (KAN) Paper Explained - An exciting new paradigm for Deep Learning?
• KAN: Kolmogorov-Arnold Networks Explained
• Kolmogorov-Arnold Networks (KANs) and Lennard Jones
• Simply explained! KAN: Kolmogorov–Arnold Networks is interpretable! Mathematics and Physics

Contributing

We welcome your contributions! Please follow these steps to contribute:

1. Fork the repo.
2. Create a new branch (e.g., feature/new-kan-resource).
3. Commit your changes to the new branch.
4. Create a Pull Request, and provide a brief description of the changes/additions.

Please make sure that the resources you add are relevant to the field of Kolmogorov-Arnold Network. Before contributing, take a look at the existing resources to avoid duplicates.

License

This work is licensed under a Creative Commons Attribution 4.0 International License.