This code was used to generate benchmarking results comparing text generation speeds on g5.12xlarge (AWS) and 1 x A100 (Vultr) hardware setups.
Detailed Results are located here: Benchmarking Llama 2 70B inference on AWS’s g5.12xlarge vs an A100
This code was used to generate benchmarking results comparing text generation speeds on g5.12xlarge (AWS) and 1 x A100 (Vultr) hardware setups.
Detailed Results are located here: Benchmarking Llama 2 70B inference on AWS’s g5.12xlarge vs an A100