/ppl.llm.kernel.cuda

Primary LanguageC++Apache License 2.0Apache-2.0

PPL LLM Kernel CUDA

Overview

Primitive cuda kernel library for ppl.nn.llm

Currently, only Ampere and Hopper have been tested.

Prerequisites

  • Linux running on x86_64 or arm64 CPUs
  • GCC >= 9.4.0
  • CMake >= 3.18
  • Git >= 2.7.0
  • CUDA Toolkit >= 11.4. 11.6 recommended. (for CUDA)

Quick Start

  • Installing Prerequisites(on Debian or Ubuntu for example)

    apt-get install build-essential cmake git

  • Cloning Source Code

    git clone https://github.com/openppl-public/ppl.llm.kernel.cuda.git

  • Building from Source

    ./build.sh -DPPLNN_CUDA_ENABLE_NCCL=ON -DPPLNN_ENABLE_CUDA_JIT=OFF -DPPLNN_CUDA_ARCHITECTURES="'80;86;87'" -DPPLCOMMON_CUDA_ARCHITECTURES="'80;86;87'"

License

This project is distributed under the Apache License, Version 2.0.