vicuna-tools/vicuna-installation-guide

Installation failure: make -j gives error

sujitpal opened this issue · 2 comments

sujitpal commented

Getting the following error when running make -j.

(pytorch_p39) [ec2-user@ip-10-76-218-85 llama.cpp]$ make -j
I llama.cpp build info: 
I UNAME_S:  Linux
I UNAME_P:  x86_64
I UNAME_M:  x86_64
I CFLAGS:   -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -march=native -mtune=native
I CXXFLAGS: -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native
I LDFLAGS:  
I CC:       cc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-15)
I CXX:      g++ (GCC) 7.3.1 20180712 (Red Hat 7.3.1-15)

cc  -I.              -O3 -std=c11   -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wdouble-promotion -Wshadow -Wstrict-prototypes -Wpointer-arith -pthread -march=native -mtune=native   -c ggml.c -o ggml.o
g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -c llama.cpp -o llama.o
g++ -I. -I./examples -O3 -std=c++11 -fPIC -DNDEBUG -Wall -Wextra -Wpedantic -Wcast-qual -Wno-unused-function -Wno-multichar -pthread -march=native -mtune=native -c examples/common.cpp -o common.o
ggml.c: In function ‘ggml_vec_dot_q4_2_q8_0’:
ggml.c:3250:40: warning: implicit declaration of function ‘_mm256_set_m128’; did you mean ‘_mm256_set_epi8’? [-Wimplicit-function-declaration]
         const __m256 d = _mm256_mul_ps(_mm256_set_m128(d1, d0), _mm256_broadcast_ss(&y[i].d));
                                        ^~~~~~~~~~~~~~~
                                        _mm256_set_epi8
ggml.c:3250:40: error: incompatible type for argument 1 of ‘_mm256_mul_ps’
In file included from /usr/lib/gcc/x86_64-redhat-linux/7/include/immintrin.h:41:0,
                 from ggml.c:186:
/usr/lib/gcc/x86_64-redhat-linux/7/include/avxintrin.h:317:1: note: expected ‘__m256 {aka __vector(8) float}’ but argument is of type ‘int’
 _mm256_mul_ps (__m256 __A, __m256 __B)
 ^~~~~~~~~~~~~
ggml.c:3254:22: warning: implicit declaration of function ‘_mm256_set_m128i’; did you mean ‘_mm256_set_epi8’? [-Wimplicit-function-declaration]
         __m256i bx = _mm256_set_m128i(bx1, bx0);
                      ^~~~~~~~~~~~~~~~
                      _mm256_set_epi8
ggml.c:3254:22: error: incompatible types when initializing type ‘__m256i {aka __vector(4) long long int}’ using type ‘int’
make: *** [ggml.o] Error 1
make: *** Waiting for unfinished jobs....
llama.cpp: In function ‘size_t llama_set_state_data(llama_context*, const uint8_t*)’:
llama.cpp:2610:36: warning: cast from type ‘const uint8_t* {aka const unsigned char*}’ to type ‘void*’ casts away qualifiers [-Wcast-qual]
             kin3d->data = (void *) in;
                                    ^~
llama.cpp:2614:36: warning: cast from type ‘const uint8_t* {aka const unsigned char*}’ to type ‘void*’ casts away qualifiers [-Wcast-qual]
             vin3d->data = (void *) in;
                                    ^~
(pytorch_p39) [ec2-user@ip-10-76-218-85 llama.cpp]$

This is on a AWS p3.2xlarge with DL-AMI installed and the pytorch_p39 conda environment activated (although not sure if it matters).

Not a C/C++ programmer, so not sure what to make of this error, but guessing its something to do with the environment. Any help would be greatly appreciated, thanks!

fredi-python commented

The error message suggests that you're using GCC 7.3.1, which is a few years old (2018). Try updating your compiler:
Assuming, AWS is using a debian based operating system, try:

sudo apt-get update && sudo apt-get upgrade gcc

If you are already in a root environment, use:

apt-get update && apt-get upgrade gcc

Hope this helps!

sujitpal commented

Many thanks for the pointer. I am on a Amazon Linux EC2 instance so I had to download the gcc10 package using instructions on this SO Page

sudo yum install -y gcc10.x86_64 gcc10-c++.x86_64

and updating the CC and CXX environment variables to point to gcc10-cc and gcc10-g++ respectively. I am now able to compile and run vicuna through the CLI.

  • 🎉1