kyegomez

Andromeda

An all-new Language Model That Processes Ultra-Long Sequences of 100,000+ Ultra-Fast

BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

Gemini

The open source implementation of Gemini, the model that will "eclipse ChatGPT" by Google

LongNet

Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"

MultiModalMamba

A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.

Sophia

Effortless plugin and play Optimizer to cut model training costs by 50%. New optimizer that is 2x faster than Adam on LLMs.

swarms

Orchestrate Swarms of Agents From Any Framework Like OpenAI, Langchain, and Etc for Real World Workflow Automation. Join our Community: https://discord.gg/DbjBMJTSWD

swarms-pytorch

Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊

tree-of-thoughts

Plug in and Play Implementation of Tree of Thoughts: Deliberate Problem Solving with Large Language Models that Elevates Model Reasoning by atleast 70%

zeta

Build high-performance AI models with modular building blocks

kyegomez/BitNet

Implementation of "BitNet: Scaling 1-bit Transformers for Large Language Models" in pytorch

kyegomez/swarms

Orchestrate Swarms of Agents From Any Framework Like OpenAI, Langchain, and Etc for Real World Workflow Automation. Join our Community: https://discord.gg/DbjBMJTSWD

kyegomez/MultiModalMamba

A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.

kyegomez/Gemini

The open source implementation of Gemini, the model that will "eclipse ChatGPT" by Google

kyegomez/zeta

Build high-performance AI models with modular building blocks

kyegomez/RT-X

Pytorch implementation of the models RT-1-X and RT-2-X from the paper: "Open X-Embodiment: Robotic Learning Datasets and RT-X Models"

kyegomez/swarms-pytorch

Swarming algorithms like PSO, Ant Colony, Sakana, and more in PyTorch 😊

kyegomez/awesome-multi-agent-papers

A compilation of the best multi-agent papers

kyegomez/NeoSapiens

The next evolution of Agents

kyegomez/Mixture-of-Depths

Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"

kyegomez/Lets-Verify-Step-by-Step

"Improving Mathematical Reasoning with Process Supervision" by OPENAI

kyegomez/Reka-Torch

Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch

kyegomez/swarms-cloud

Deploy your autonomous agents to production grade environments with 99% Uptime Guarantee, Infinite Scalability, and self-healing.

kyegomez/MLXTransformer

Simple Implementation of a Transformer in the new framework MLX by Apple

kyegomez/TeraGPT

Train a production grade GPT in less than 400 lines of code. Better than Karpathy's verison and GIGAGPT

kyegomez/Infini-attention

Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTORCH

kyegomez/SimplifiedTransformers

SimplifiedTransformer simplifies transformer block without affecting training. Skip connections, projection parameters, sequential sub-blocks, and normalization layers are removed. Experimental results confirm similar training speed and performance.

kyegomez/BRAVE-ViT-Swarm

Implementation of the paper: "BRAVE : Broadening the visual encoding of vision-language models"

kyegomez/swarms-platform

kyegomez/CELESTIAL-1

Omni-Modality Processing, Understanding, and Generation

kyegomez/ShallowFF

Zeta implemantion of "Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers"

kyegomez/AttnWithConvolutions

Interleaved Attention's with convolutions for text modeling

kyegomez/GiediPrime

An experimental architecture using Mixture of Attentions with sandwiched Maracron Feedforward's and other modules

kyegomez/SimpleUnet

An simple implementation of Unet because all the implementations i've seen are wayy tooo complicated.

kyegomez/synchro

Synchronize your requirement.txt and pyproject.toml at the bush of a button!

kyegomez/Kosmos-API

This repository holds the infrastructure to serve Kosmos to customers.

kyegomez/AoA-torch

Implementation of Attention on Attention in Zeta

kyegomez/Genie

Implementation of the paper: "Genie: Generative Interactive Environments" from Deepmind

kyegomez/HGRN2

Implementation of the paper: "HGRN2: Gated Linear RNNs with State Expansion" in PyTorch

kyegomez/SimpleDiffusion

Implementation of a simple Diffusion model so I can learn how they work, they are my weakness now.