/model-llama2-7b-dvc

⚗️ Llama2 7b model repository trained by meta managed by DVC

Primary LanguagePython

Task Tags
TextGeneration
TextGeneration
Llama2-7b

Model-Llama2-7b-dvc

🔥🔥🔥 Deploy Llama2-7b model on VDP.

This repository contains the Llama2-7b Text Completion Generation Model in the vLLM and Transformers format, managed using DVC. For information about available extra parameters, please refer to the documentation on SamplingParams in the vLLM library.

Notes:

  • Disk Space Requirements: 14G
  • Memory Requirements: 40G (for fp32 in cpu mode)
  • Following is an example of query parameters:
{
    "task_inputs": [
        {
            "text_generation": {
                "prompt": "The capital city of Franch is ",
                "max_new_tokens": "100",
                "temperature": "0.8",
                "top_k": "10",
                "seed": "42",
                "extra_params": {
                    "repetition_penalty": 1.8
                }
            }
        }
    ]
}