chunhui-shi/FATE-LLM

Federated Learning for LLMs.

FATE-LLM

FATE-LLM is a framework to support federated learning for large language models(LLMs).

Design Principle

• Federated learning for heterogeneous large model and small model.
• Promote training efficiency of federated LLMs using Parameter-Efficient methods.
• Protect the IP of LLMs using FedIPR.
• Protect data privacy during training and inference through privacy preserving mechanisms.

Quick Start

• Federated ChatGLM-6B Training
• GPT-2 Training