/AIOS

AIOS: LLM Agent Operating System

Primary LanguagePythonMIT LicenseMIT

AIOS: LLM Agent Operating System

Code License

AIOS, a Large Language Model (LLM) Agent operating system, embeds large language model into Operating Systems (OS) as the brain of the OS, enabling an operating system "with soul" -- an important step towards AGI. AIOS is designed to optimize resource allocation, facilitate context switch across agents, enable concurrent execution of agents, provide tool service for agents, maintain access control for agents, and provide a rich set of toolkits for LLM Agent developers.

🏠 Architecture of AIOS

📰 News

✈️ Getting Started

Installation

Make sure you have Python >= 3.9
Install the required packages using pip

pip install -r requirements.txt

Usage

Set up huggingface token and cache directory

export HUGGING_FACE_HUB_TOKEN=<YOUR READ TOKEN>
export HF_HOME=<YOUR CACHE DIRECTORY>

Run the main.py to start

# Use Gemma-2b-it for example, replace the max_gpu_memory and eval_device with your own
python main.py --llm_name gemma-2b-it --max_gpu_memory '{"0": "24GB"}' --eval_device "cuda:0" --max_new_tokens 256

🌟 Join Us!

AIOS is dedicated to facilitating LLM agents' development and deployment in a systematic way, we are always looking for passionate collaborators to join us to foster a more cohesive, effective and efficient AIOS-Agent ecosystem!

📪 Contact

If you have any suggestions, or wish to contact us for any reason, feel free to email us at marknju2018@gmail.com