llm_learning all about LLM (large language model) 1. model basic and config clone llm bin files and utils code set run at gpu num 2. chatglm2 inference load base model for inference 3. chatglm2 ptuning ptuning steps 1GPU.fp16 1GPU.int4 3GPU.fp16 4. chatglm2 load ptuning model for inference load ptuning model checkpoint for inference 5. LLaMA-Factory steps to use LLaMA-Factory fine tuning framework fine tuning chatglm2.lora.1gpu.fp16 load model cli demo chatglm2.lora.1gpu support multiple dataset mulitple GPU on one node by accelerate 6. Prompt tech persona assignment 7. datasketch: lsh for data process, dup-remove Keep Chinese, English and underline, no punctuation minhash lsh for similarity lsh cluster 8. fastparquet read parquet file 9. dataset construct prompt prefix for sft dataset 10. FlagEmbedding BGE-M3 embedding for text 11. BERT bert for long text classification 12. datasets load and show data structure