lib installation on Windows
codeyourweb opened this issue · 15 comments
Hi,
I'm trying to use your library on Windows. I'm not familiar with C/C++ and linker/compilation problems but i've searched a lot before posting here. Maybe you could help me. First, i've installed mingw64 and also the latest version of visual studio. I've cloned the official yara repository and successfuly compiled libyara with visual studio. I saw that i've got the following folders:
- C:\YARA\windows\libyara (with a lot of *.obj files)
- C:\YARA\libyara (with a lot of *.c files)
- C:\YARA\libyara\include (with *.h files)
- C:\YARA\windows\vs2017\libyara\Release (with a 22MB libyara64.lib)
I didn't understood what i need to put in CGO_LDFLAGS and CGO_CFLAGS env vars or if i need additional compilations to successfully go get github.com/hillu/go-yara.
Actually it just return the following error:
# runtime/cgo
C:/mingw64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.1.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -lyara
Any help would be appreciated :)
Sorry, I have no experience with using the Visual Studio buildsystem, so I can offer guesswork at best.
Assuming you want to use the VS-generated build using files from the paths you mention, I think that you'll have to
- disable pkg-config based configuration since no meaningful
yara.pc
has been generated. This can be done with passing the build tagyara_no_pkg_config
as described in README.md. - explicitly set
CGO_CFLAGS
so that the compiler looks in the correct directory for the header files:-IC:\YARA\libyra/include
should be correct. - explicitly set
CGO_LDFLAGS
so that the linker looks in the correct directory for the library:-LC:\YARA\windows\vs2017\libyara\Release
should be correct. - patch cgo so that
libyara64
is used in your case (with pkg-config disabled). Try this:Perhaps passing the absolute filename instead of// #cgo yara_no_pkg_config LDFLAGS: -lyara64
-lyara64
also works.
I hope this helps. If it does, I'd be very happy about a documentation pull request.
definitly, i know why i'll never dev something in C. sometimes you take more times to successfully compile third-party libraries than coding your own program...
Anyway thanks for your help, it's seems to be better (even if i doesn't work for the moment). the linker seems to correctly parse my .lib file. However there are a lot of errors that i really don't how to deal with:
Have you got any idea on how to solve this?
Uh-oh. I never tried this before, but it seems like the Go toolchain can't link objects built with Visual C++ (see here). I think you are going to have to build libyara using MinGW.
more complicated on windows than on linux but (with your help) i made progress. I installed MSYS2/mingw and managed to compile yara from bash. I gave up using pkg-config because it always returns an error, even from mingw. However running with tag "yara_no_pkg_config" is really better than before. It seems it's only missing one library. I've installed openssl and openssl-devel in mingw but it's not enough.
# go install --tags yara_static,yara_no_pkg_config
# github.com/hillu/go-yara/v4
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x1034): undefined reference to `MD5_Init'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x114e): undefined reference to `MD5_Update'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x119b): undefined reference to `MD5_Final'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3062): undefined reference to `EVP_sha1'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3092): undefined reference to `PKCS7_get0_signers'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x30bf): undefined reference to `OPENSSL_sk_num'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x30f6): undefined reference to `OPENSSL_sk_value'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3111): undefined reference to `X509_digest'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3165): undefined reference to `X509_get_issuer_name'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3176): undefined reference to `X509_NAME_oneline'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x31a2): undefined reference to `X509_get_subject_name'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x31b3): undefined reference to `X509_NAME_oneline'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x31e5): undefined reference to `X509_get_version'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3205): undefined reference to `X509_get_signature_nid'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x320c): undefined reference to `OBJ_nid2ln'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3251): undefined reference to `X509_get_serialNumber'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3263): undefined reference to `i2d_ASN1_INTEGER'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3277): undefined reference to `X509_get0_notBefore'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x329d): undefined reference to `X509_get0_notAfter'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x32cd): undefined reference to `OPENSSL_sk_num'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x32ee): undefined reference to `OPENSSL_sk_value'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3309): undefined reference to `OBJ_txt2nid'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3319): undefined reference to `X509at_get_attr_by_NID'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x3328): undefined reference to `X509at_get_attr'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x334e): undefined reference to `d2i_PKCS7'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x336c): undefined reference to `PKCS7_free'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x337e): undefined reference to `X509_ATTRIBUTE_get0_type'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x338d): undefined reference to `OPENSSL_sk_free'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x33b5): undefined reference to `i2d_ASN1_INTEGER'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x58f1): undefined reference to `OBJ_create'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x6d22): undefined reference to `PKCS7_free'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(pe.o):pe.c:(.text+0x6d55): undefined reference to `d2i_PKCS7'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xaa2): undefined reference to `MD5_Init'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xc32): undefined reference to `MD5_Update'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xd1e): undefined reference to `MD5_Final'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xecf): undefined reference to `MD5_Init'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xede): undefined reference to `MD5_Update'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xeee): undefined reference to `MD5_Final'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xfd5): undefined reference to `SHA256_Init'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xfe4): undefined reference to `SHA256_Update'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0xff4): undefined reference to `SHA256_Final'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x10df): undefined reference to `SHA1_Init'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x10ee): undefined reference to `SHA1_Update'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x10fe): undefined reference to `SHA1_Final'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x11fa): undefined reference to `SHA1_Init'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x1398): undefined reference to `SHA1_Update'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x148e): undefined reference to `SHA1_Final'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x164a): undefined reference to `SHA256_Init'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x17e8): undefined reference to `SHA256_Update'
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: C:/msys64/usr/lib/libyara.a(hash.o):hash.c:(.text+0x18de): undefined reference to `SHA256_Final'
collect2.exe: error: ld returned 1 exit status
You need to pass flags to the linker so that it can find OpenSSL's libcrypto
which contains the symbols mentioned in your error message: -L$PATH -lcrypto
. PATH is probably the openssl-$VERSION
directory that was created when unpacking the source distribution.
Can you post the error messages that pkg-config gave you?
Did the OpenSSL and YARA builds generate DLLs or only static libraries (lib*.a
)?
I'm currently at work, i'll check this when i'll be at home. Here's some elements that i remember:
- i don't remember the exact message of pkg-config but it throw an error because it can't found "m.pc".
- with mingw , yara was compiled as a *.a library (not *.lib like in VC++) , openssl il available in *.a and *.dll
Not sure to clearly understand your first comment. I need to pass this item to the linker like this, is it true?
CGO_LDFLAGS="-L${YARA_LIB_PATH} -lyara" "-L{$LCRYPTO_LIB_PATH -lcrypto}"
(with quotes and space separated or something like this?
Assuming you are setting CGO_LDFLAGS from within bash, use a single pair of double quotes, e.g.
export CGO_LDFLAGS="-L${YARA_LIB_PATH} -l yara -L${CRYPTO_LIB_PATH} -lcrypto"
First of all, thank you for your help, because this time it works :) ... to be more precise, it NEARLY works, but maybe it's just my mistake and my lack of understanding of C linking process:
-
compilation with CGO_LDFLAGS="-LC:/msys64/usr/lib -lyara -lcrypto" doesn't return any error :)
-
running the project in your _examples folder return the same error as while i try to install without "no_pkg_config" tag
-
About pkg-config, the exact error message is:
# pkg-config --cflags -- yara
Package m was not found in the pkg-config search path.
Perhaps you should add the directory containing `m.pc'
to the PKG_CONFIG_PATH environment variable
Package 'm', required by 'yara', not found
pkg-config: exit status 1
maybe you'll know how to solve this error
- if we can't solve pkg-config issue, do you know to avoid pkg-config to be used? I've tried the following things but maybe i missed something:
1/ add a comment on first line of "simple-yara.go":
// #cgo yara_no_pkg_config LDFLAGS: -lyara
2/ run it with:
go run simple-yara.go --tags yara_no_pkg_config
Alright, we're getting there. It seems that the pkg-config error happens because of a bad commit in YARA:
commit a94f6deb1d2c60e12c6246c4562399c2e86f5a86
Author: Hilko Bengen <bengen@hilluzination.de>
Date: Fri Nov 6 17:12:28 2020 +0100
Add m (math) library to libs.private (pkg-config) if it is used
This should get rid of "undefined reference to `log2'" errors when
using pkg-config and using the static library.
You should be able to work around this by removing the "m" from the Requires.private
line in the generated yara.pc
file that is read by pkg-config.
I'll get this fixed properly (VirusTotal/yara#1408).
Please post the error messages you get from go run
. I suspect that you didn't pass extra directories for the library search path (-L$path
parameters).
- i've compiled yara with your commit and removed the "m" in yara.pc. There isn't the previous error message anymore but an exit status 5-10 seconds after starting your example project:
C:\Users\myusername\go\src\github.com\hillu\go-yara\_examples\simple-yara> go run .
exit status 3221225781
- in order to run your example without pkg-config i've just edited the simple-yara.go like this:
package main
// #cgo !yara_no_pkg_config,!yara_static pkg-config: yara
// #cgo !yara_no_pkg_config,yara_static pkg-config: --static yara
// #cgo yara_no_pkg_config LDFLAGS: -lyara
import (
"github.com/hillu/go-yara/v4"
"bytes"
"flag"
"fmt"
"log"
"os"
"path/filepath"
"strconv"
"sync"
)
func printMatches(item string, m []yara.MatchRule, err error) {
[...]
and then tried to run it like this (but it return the same error as if it's trying to run it with pkg-config and return the exact same error):
C:\Users\myusername\go\src\github.com\hillu\go-yara\_examples\simple-yara> go run . --tags yara_no_pkg_config
exit status 3221225781
could mean STATUS_DLL_NOT_FOUND (0xc0000135)
. This probably means that YARA or OpenSSL has been built as a DLL and not linked statically into the binary generated by go run
. Try just running go build
and inspect the import directory of the binary.
You can either pass -static
to the linker or prevent the YARA and OpenSSL build scripts from generating DLLs in the first place.
PS C:\Users\myusername\go\src\github.com\hillu\go-yara\_examples\simple-yara> go run .
2020/12/06 08:08:03 no rules specified
exit status 1
I've never been so happy to see an error from a golang code 😂 . As you see, it finally works! First of all, thanks for your help. Even if i worked a lot to deal with a mountain of problems installing this lib, i probably wouldn't have gotten there without your help.
My first mistake was to use msys2 terminal instead of mingw. Indeed, there was a problem with libcrypto and libssl (compiled as .a and .dll but source of conflicts. My second mistake was to directly clone VirusTotal yara repository to compile libyara. There are too much commit and it's the best way to deal with side effects. And last but not least, my third mistake was to compile libraries (like openssl) from source instead of leaving this task to the package manager (pacman on mingw).
I confirm that your library works on Windows, even without cross compile from a linux distribution. In few words it's necessary to:
- install msys2 (maybe i missed something with mingw64 / cygwin but i tried with these softwares and i got some errors)
- after msys2 fresh install , prefer using mingw terminal instead of msys one. (to avoid library install path conflict and missing libs)
- install dependencies
pacman -S mingw-w64-x86_64-toolchain mingw-w64-x86_64-pkg-config base-devel openssl-devel
- configure environment variables (NOT in mingw/msys terminal but in your OS) (CGO_CFLAGS / CGO_LDFLAGS / PKG_CONFIG_PATH)
- compile libyara from the latest release (and not the latest commit)
- then install your librarie (could be done with windows cmd / powershell, no need of mingw terminal)
If you want, i can make a pull request with a specific documentation for Windows installation in a seperate txt/md file
Glad we finally got things worked out. 👍