Errors building libdnet for x64 and python 2.7

Question

Errors building libdnet for x64 and python 2.7

newlog opened this issue 10 years ago · 1 comments

Hello!

I open an issue here given that you have solved my problem and I don't have another way of reaching you. I've been trying to build libdnet without success and given that you've already built it, I would really appreciate if you could point what am I doing wrong!

All my process for building it (and eventually failing) follows. I've already post this issue in another repository which is an updated fork of libdnet based on the improvements made by nmap crew to the library. I'll copy-paste the issue I already wrote there instead of only pasting the link, so people with my same problems can read them here too.

Hello,

First of all, thanks for creating this fork. I've ended here after having to fight with the "WARNING: No match between your pcap and dnet network interfaces found." error without success.

So I've tried to build your fork in my Windows 8.1 x64 machine with Python 2.7. I've had a lot of problems and I'd be really grateful if you could help me.

These are the steps I've followed:

I've cloned you repository and executed the autogen.sh script inside Cygwin, having MinGW also installed.
This gave me a lot of trouble, basically for using libtool from MinGW instead of the one from Cygwin. So I uninstalled the one from MinGW and installed the one from Cygwin. After this, the autogen.sh worked and the configure file was created.
I executed ./configure and failed when checking for Python, it seemed a MinGW32 issue. So I installed a bunch of MinGW packages of 32 bits such as headers and so. Don't know exactly how, but this solved the problem.
Nevertheless, the ./configure failed again. Now the problem was because of winpcap developer library not found. So I executed configure as:
./configure --with-wpdpack=<path_to_lib>
Then I cd python and executed /c/cygdrive/python2.7/python.exe setup.py build and got this error:

$ /cygdrive/c/Python27/python.exe setup.py build
running build
running build_ext
building 'dnet' extension
error: Unable to find vcvarsall.bat

First real unsolved problem here. So instead of using OS python.exe I've also tried to use cygwin python. I got this error:

$ /usr/bin/python2.7 setup.py build
running build
running build_ext
building 'dnet' extension
gcc -fno-strict-aliasing -ggdb -O2 -pipe -Wimplicit-function-declaration -fdebug-prefix-map=/usr/src/ports/python/python-2.7.8-1.x86_64/build=/usr/src/debug/python-2.7.8-1 -fdebug-prefix-map=/usr/src/ports/python/python-2.7.8-1.x86_64/src/Python-2.7.8=/usr/src/debug/python-2.7.8-1 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I../include -I/usr/include/python2.7 -c ./dnet.c -o build/temp.cygwin-1.7.34-x86_64-2.7/./dnet.o
./dnet.c:4:20: fatal error: Python.h: No such file or directory
 #include "Python.h"
                    ^
compilation terminated.
error: command 'gcc' failed with exit status 1

Seems that Python.h is not found, although it's in the /usr/include/Python2.7 folder. I've added it to PATH with export PATH="$PATH:/usr/include/python2.7" but it gave the same error.

Then I came back to the Unable to find vcvarsall.bat error. Still from cygwin I executed:

$ export DISTUTILS_USE_SDK=1
$ export MSSdk=1
$ /cygdrive/c/Python27/python.exe setup.py build
running build
running build_ext
building 'dnet' extension
cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I../include -I../../WPdpack/Include -IC:\Python27\include -IC:\Python27\PC /Tc./dnet.c /Fobuild\temp.win-amd64-2.7\Release\./dnet.obj
error: command 'cl.exe' failed: No such file or directory

Ok, this problem about not locating 'cl.exe' is because if you use the OS python.exe and you have not built it, the compiler that was used to build it is used to build the python extension. I've tried to add the location of cl.exe folder in path (export PATH="$PATH:/cygwin/c/Program Files (x86)/Microsoft Visual Studio 12.0/VC/bin/x86_amd64"), but it didn't work.So...

Next step was to open Visual C++ 2008 64-bits Command Prompt so cl.exe is easily found. I exported the symbols mentioned before and executed:

>C:\Python27\python.exe setup.py build
running build
running build_ext
building 'dnet' extension
C:\Users\windows8\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I../include -I../../WPdpac
k/Include -IC:\Python27\include -IC:\Python27\PC /Tc./dnet.c /Fobuild\temp.win-amd64-2.7\Release\./dnet.obj
dnet.c
../include\dnet/os.h(29) : error C2371: 'ssize_t' : redefinition; different basic types
        c:\python27\include\pyconfig.h(200) : see declaration of 'ssize_t'
../include\dnet/os.h(151) : fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory
error: command 'C:\\Users\\windows8\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\cl.exe' failed with exit status 2

This is the REAL problem. It seems that python already defines the ssize_t type in pyconfig.h. I could remove declarations or make them match. The best approach was to make them match types (modifying __int64 for long) by modifying line 199 of pyconfig.h:

#ifdef MS_WIN64
typedef long ssize_t;
#else
typedef _W64 int ssize_t;
#endif

This solved the problem of redefinitions, but still remained the fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory.

I've tried to comment this includes in os.h and intf.h, but it ended up giving me this error after a large number of warnings:

C:\Users\windows8\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Bin\amd64\cl.exe /c /nologo /Ox /MD /W3 /GS- /DNDEBUG -I../include -I../../WPdpac
k/Include -IC:\Python27\include -IC:\Python27\PC /Tc../src/addr-util.c /Fobuild\temp.win-amd64-2.7\Release\../src/addr-util.obj
addr-util.c
c:\users\windows8\appdata\local\programs\common\microsoft\visual c++ for python\9.0\vc\include\codeanalysis\sourceannotations.h(17) : warning C4114: same type qualifier
 used more than once
c:\users\windows8\appdata\local\programs\common\microsoft\visual c++ for python\9.0\vc\include\codeanalysis\sourceannotations.h(17) : error C2632: '__int64' followed by
 'int' is illegal
c:\users\windows8\appdata\local\programs\common\microsoft\visual c++ for python\9.0\vc\include\codeanalysis\sourceannotations.h(17) : warning C4091: 'typedef ' : ignore
d on left of 'unsigned __int64' when no variable is declared
C:\Users\windows8\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Include\string.h(81) : warning C4391: 'unsigned int strlen(char *)' : incorrect r
eturn type for intrinsic function, expected 'unsigned __int64'
C:\Users\windows8\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Include\string.h(254) : warning C4391: 'unsigned int wcslen(wchar_t *)' : incorre
ct return type for intrinsic function, expected 'unsigned __int64'
../src/addr-util.c(223) : warning C4244: 'function' : conversion from '__int64' to 'unsigned int', possible loss of data
error: command 'C:\\Users\\windows8\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\cl.exe' failed with exit status 2

I realised that the lack of stdint.h was because of Microsoft VC 2008 didn't supported some C99 features, so I tried with VC 2013. After removing comments from the stdint.h includes (I also could remove the modification of pyconfig.h) I executed the same command from the Developer Command Prompt for VS 2013 and got the following (complete) output:

>c:\Python27\python.exe setup.py build
running build
running build_ext
building 'dnet' extension
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /
DNDEBUG -I../include -I../../WPdpack/Include -Ic:\Python27\include -Ic:\Python27\PC /Tc./dnet.c
 /Fobuild\temp.win-amd64-2.7\Release\./dnet.obj
dnet.c
../include\dnet/os.h(29) : warning C4142: benign redefinition of type
./dnet.c(691) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(916) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(1485) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(1491) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(1810) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(1816) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(1935) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(1941) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(2044) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(2050) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(2464) : warning C4102: '__pyx_L4' : unreferenced label
./dnet.c(2462) : warning C4102: '__pyx_L3' : unreferenced label
./dnet.c(2748) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(2754) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(3118) : warning C4090: 'function' : different 'const' qualifiers
./dnet.c(3118) : warning C4028: formal parameter 1 different from declaration
./dnet.c(3211) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(3567) : warning C4018: '<' : signed/unsigned mismatch
./dnet.c(3591) : warning C4102: '__pyx_L7' : unreferenced label
./dnet.c(3589) : warning C4102: '__pyx_L6' : unreferenced label
./dnet.c(3647) : warning C4013: 'strlcpy' undefined; assuming extern returning int
./dnet.c(3655) : warning C4244: '=' : conversion from 'long' to 'unsigned short', possible loss
 of data
./dnet.c(3730) : warning C4018: '<' : signed/unsigned mismatch
./dnet.c(3745) : warning C4102: '__pyx_L9' : unreferenced label
./dnet.c(3743) : warning C4102: '__pyx_L8' : unreferenced label
./dnet.c(4135) : warning C4090: 'function' : different 'const' qualifiers
./dnet.c(4135) : warning C4028: formal parameter 1 different from declaration
./dnet.c(4225) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(4556) : warning C4090: 'function' : different 'const' qualifiers
./dnet.c(4556) : warning C4028: formal parameter 1 different from declaration
./dnet.c(4646) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(5216) : warning C4090: 'function' : different 'const' qualifiers
./dnet.c(5216) : warning C4028: formal parameter 1 different from declaration
./dnet.c(5306) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(5640) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(5730) : warning C4018: '>' : signed/unsigned mismatch
./dnet.c(5736) : warning C4102: '__pyx_L5' : unreferenced label
./dnet.c(5729) : warning C4102: '__pyx_L4' : unreferenced label
./dnet.c(5798) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(5817) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(5890) : warning C4102: '__pyx_L6' : unreferenced label
./dnet.c(5888) : warning C4102: '__pyx_L5' : unreferenced label
./dnet.c(5854) : warning C4102: '__pyx_L3' : unreferenced label
./dnet.c(6146) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(6180) : warning C4102: '__pyx_L1' : unreferenced label
./dnet.c(6421) : warning C4101: 'e' : unreferenced local variable
./dnet.c(6575) : warning C4101: 'e' : unreferenced local variable
./dnet.c(6718) : warning C4101: 'e' : unreferenced local variable
./dnet.c(6947) : warning C4101: 'e' : unreferenced local variable
./dnet.c(7098) : warning C4101: 'e' : unreferenced local variable
./dnet.c(7253) : warning C4101: 'e' : unreferenced local variable
./dnet.c(7409) : warning C4101: 'e' : unreferenced local variable
./dnet.c(7564) : warning C4101: 'e' : unreferenced local variable
./dnet.c(7718) : warning C4101: 'e' : unreferenced local variable
./dnet.c(7867) : warning C4101: 'e' : unreferenced local variable
./dnet.c(8018) : warning C4101: 'e' : unreferenced local variable
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /
DNDEBUG -I../include -I../../WPdpack/Include -Ic:\Python27\include -Ic:\Python27\PC /Tc../src/a
ddr-util.c /Fobuild\temp.win-amd64-2.7\Release\../src/addr-util.obj
addr-util.c
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\crtdefs.h(496) : warning C4114:
same type qualifier used more than once
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\crtdefs.h(496) : error C2632: 'i
nt' followed by 'int' is illegal
C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\INCLUDE\crtdefs.h(496) : warning C4091:
'typedef ' : ignored on left of '__w64 unsigned int' when no variable is declared
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\VC\\BIN\\cl.exe' failed
with exit status 2

Seems that the problem is that size_t is already defined as int so the following code in crtdefs.h fails:

#ifndef _SIZE_T_DEFINED
#ifdef _WIN64
typedef unsigned __int64    size_t;
#else  /* _WIN64 */
typedef _W64 unsigned int   size_t;      <--
#endif  /* _WIN64 */
#define _SIZE_T_DEFINED
#endif  /* _SIZE_T_DEFINED */

It seems that there's again a redefinition. I've found it in libdnet source at include/config.h in line 270:

/* Define to `unsigned int' if <sys/types.h> does not define. */
//#define size_t unsigned int

After commenting it, the build process has continued with a lot of warnings until this error:

C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /
DNDEBUG -I../include -I../../WPdpack/Include -Ic:\Python27\include -Ic:\Python27\PC /Tc../src/e
th-win32.c /Fobuild\temp.win-amd64-2.7\Release\../src/eth-win32.obj
eth-win32.c
../src/eth-win32.c(20) : fatal error C1083: Cannot open include file: 'pcap.h': No such file or
 directory
error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio 12.0\\VC\\BIN\\cl.exe' failed
with exit status 2

Given that the libdnet doesn't have any pcap.h file, I guess that this file must refer to the winpcap development library. So I try to add the correct directory to the INCLUDE environment variable with set INCLUDE=%INCLUDE%;<path_to_winpcap_dev>\WpdPack_4_1_2\WpdPack\Include

There are two pcap.h files. One in the Include directory and another in the Include\pcap directory. I've tried to add both of them in the INCLUDE environment, but neither worked. I finally got a long list of new errors:

C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\cl.exe /c /nologo /Ox /MD /W3 /GS- /
DNDEBUG -I../include -I../../WPdpack/Include -Ic:\Python27\include -Ic:\Python27\PC /Tc../src/f
w-pktfilter.c /Fobuild\temp.win-amd64-2.7\Release\../src/fw-pktfilter.obj
fw-pktfilter.c
C:\Program Files (x86)\Windows Kits\8.1\include\shared\ipifcons.h(246) : error C2061: syntax er
ror : identifier 'IFTYPE'
C:\Program Files (x86)\Windows Kits\8.1\include\shared\ipifcons.h(246) : error C2059: syntax er
ror : ';'
C:\Program Files (x86)\Windows Kits\8.1\include\shared\iprtrmib.h(145) : error C2016: C require
s that a struct or union has at least one member
C:\Program Files (x86)\Windows Kits\8.1\include\shared\iprtrmib.h(145) : error C2061: syntax er
ror : identifier 'DWORD'
C:\Program Files (x86)\Windows Kits\8.1\include\shared\iprtrmib.h(146) : error C2061: syntax er
ror : identifier 'rgdwVarIndex'
(...)

So I'm stuck here and I don't know what else to do. This is a nightmare, guys. How could you build this? I'm defeated....

Thank you very much for reading until here.

Answer 1 · 2016-02-10T13:19:48.000Z

Hey,

sorry for the kinda late response, I was planning to release a fixed version of libdnet, but then ran out of time etc.. I can share the fixed source if you're interested. I built this using Visual Studio 2008 (other versions don't build Python modules correctly). If you're still interested, I can try to repeat the compilation and write some guide for it. Send me a reminder to konrad(at)brodzik(dot)it if I don't do it in reasonable time. As for the interface matching, it's tricky. I didn't really need it (I only needed to generate traffic without receiving it) but I needed to have a mapping between interfaces in gwmi win32_networkinterfaces and the stuff libdnet returns, so I basically added a field returning the GUID there.