Add `shared` option to recipe

Question

Add `shared` option to recipe

Closed this issue 3 years ago · 7 comments

Hi, I'm currently creating my own library that builds on top of gRPC and I'm using conan to manage all of my dependencies. My goal is to build a shared library that other client applications can link to. These applications also use some parts of the gRPC and protobuf libraries. However, I'm kind of stuck now because I cannot manage to "pass through" the symbols of gRPC and protobuf to my client applications. I presume this has something to do with mixing static libraries (produced by the conan recipe) and my shared library.

I noticed that you commented out the line where the shared option is defined. Is there any particular reason for that? Because all of gRPC's dependencies at least provide this option, even if it is set to false by default. It would really make my life easier having a shared library of gRPC.

Thanks in advance for your help!

Answer 1 · 2020-01-15T18:52:12.000Z

If I recall this correctly, than grpc had (or has?) upstream issues with building as a shared library

I can try again soon

Answer 2 · 2020-01-16T07:05:39.000Z

That sounds great!

I saw that there have been some issues with building gRPC as a shared library but I think it should be possible. I tried it myself yesterday and almost got it working:

I added the shared option and set it to true
Also, all of gRPC's dependencies were built as shared libraries
In the root CMakeLists.txt file of gRPC I had to change

if(WIN32 and MSVC)
  set(_gRPC_BASELIB_LIBRARIES wsock32 ws2_32)
endif()

to

if(WIN32)
  set(_gRPC_BASELIB_LIBRARIES wsock32 ws2_32)
endif()

because despite I'm building on Windows I am using MinGW's gcc as my compiler and without that change, I would get linker errors (this issue discusses that problem but the related PR only fixes it for MSVC on Windows).

With those changes and adjustments, I was able to get almost through the build. It seems like all gRPC shared libraries are built and linked properly. However, I'm running into a problem with protobuf's protoc:

--grpc_out: protoc-gen-grpc: Plugin failed with status code 3221225781.

Click to expand log

...
[ 98%] Linking CXX shared library ..\bin\libgrpc++.dll
cd /d D:\.conan\d04c65\1\build_subfolder\source_subfolder && "C:\Program Files\CMake\bin\cmake.exe" -E cmake_link_script CMakeFiles\grpc++.dir\link.txt --verbose=1
"C:\Program Files\CMake\bin\cmake.exe" -E remove -f CMakeFiles\grpc++.dir/objects.a
D:\.conan\a0f254\1\bin\ar.exe cr CMakeFiles\grpc++.dir/objects.a @CMakeFiles\grpc++.dir\objects1.rsp
D:\.conan\a0f254\1\bin\g++.exe  -D_WIN32_WINNT=0x600  -m64  -std=c++11 -g    -m64   -shared -o ..\bin\libgrpc++.dll -Wl,--out-implib,..\lib\libgrpc++.dll.a -Wl,--major-image-version,0,--minor-image-version,0 -Wl,--whole-archive CMakeFiles\grpc++.dir/objects.a -Wl,--no-whole-archive @CMakeFiles\grpc++.dir\linklibs.rsp
mingw32-make.exe[2]: Leaving directory 'D:/.conan/d04c65/1/build_subfolder'
[ 98%] Built target grpc++
D:/.conan/a0f254/1/bin/mingw32-make.exe -f source_subfolder\CMakeFiles\grpc++_error_details.dir\build.make source_subfolder/CMakeFiles/grpc++_error_details.dir/depend
D:/.conan/a0f254/1/bin/mingw32-make.exe -f source_subfolder\CMakeFiles\grpc++_reflection.dir\build.make source_subfolder/CMakeFiles/grpc++_reflection.dir/depend
D:/.conan/a0f254/1/bin/mingw32-make.exe -f source_subfolder\CMakeFiles\grpcpp_channelz.dir\build.make source_subfolder/CMakeFiles/grpcpp_channelz.dir/depend
mingw32-make.exe[2]: Entering directory 'D:/.conan/d04c65/1/build_subfolder'
mingw32-make.exe[2]: Entering directory 'D:/.conan/d04c65/1/build_subfolder'
mingw32-make.exe[2]: Entering directory 'D:/.conan/d04c65/1/build_subfolder'
[ 98%] Running gRPC C++ protocol buffer compiler on src/proto/grpc/reflection/v1alpha/reflection.proto
[ 98%] Running gRPC C++ protocol buffer compiler on src/proto/grpc/status/status.proto
[ 98%] Running gRPC C++ protocol buffer compiler on src/proto/grpc/channelz/channelz.proto
cd /d D:\.conan\d04c65\1\source_subfolder && D:\.conan\e929b0\1\bin\protoc.exe --grpc_out=generate_mock_code=true:D:/.conan/d04c65/1/build_subfolder/gens --cpp_out=D:/.conan/d04c65/1/build_subfolder/gens --plugin=protoc-gen-grpc=D:/.conan/d04c65/1/build_subfolder/bin/grpc_cpp_plugin.exe -I . -I D:/.conan/073407/1/include src/proto/grpc/reflection/v1alpha/reflection.proto
cd /d D:\.conan\d04c65\1\source_subfolder && D:\.conan\e929b0\1\bin\protoc.exe --grpc_out=generate_mock_code=true:D:/.conan/d04c65/1/build_subfolder/gens --cpp_out=D:/.conan/d04c65/1/build_subfolder/gens --plugin=protoc-gen-grpc=D:/.conan/d04c65/1/build_subfolder/bin/grpc_cpp_plugin.exe -I . -I D:/.conan/073407/1/include src/proto/grpc/status/status.proto
cd /d D:\.conan\d04c65\1\source_subfolder && D:\.conan\e929b0\1\bin\protoc.exe --grpc_out=generate_mock_code=true:D:/.conan/d04c65/1/build_subfolder/gens --cpp_out=D:/.conan/d04c65/1/build_subfolder/gens --plugin=protoc-gen-grpc=D:/.conan/d04c65/1/build_subfolder/bin/grpc_cpp_plugin.exe -I . -I D:/.conan/073407/1/include src/proto/grpc/channelz/channelz.proto
--grpc_out: protoc-gen-grpc: Plugin failed with status code 3221225781.
--grpc_out: protoc-gen-grpc: Plugin failed with status code 3221225781.
--grpc_out: protoc-gen-grpc: Plugin failed with status code 3221225781.
mingw32-make.exe[2]: *** [source_subfolder\CMakeFiles\grpc++_reflection.dir\build.make:66: gens/src/proto/grpc/reflection/v1alpha/reflection.grpc.pb.cc] Error 1
mingw32-make.exe[2]: Leaving directory 'D:/.conan/d04c65/1/build_subfolder'
mingw32-make.exe[2]: *** [source_subfolder\CMakeFiles\grpcpp_channelz.dir\build.make:66: gens/src/proto/grpc/channelz/channelz.grpc.pb.cc] Error 1
mingw32-make.exe[2]: Leaving directory 'D:/.conan/d04c65/1/build_subfolder'
mingw32-make.exe[1]: *** [CMakeFiles\Makefile2:583: source_subfolder/CMakeFiles/grpcpp_channelz.dir/all] Error 2
mingw32-make.exe[1]: *** Waiting for unfinished jobs....
mingw32-make.exe[2]: *** [source_subfolder\CMakeFiles\grpc++_error_details.dir\build.make:66: gens/src/proto/grpc/status/status.grpc.pb.cc] Error 1
mingw32-make.exe[2]: Leaving directory 'D:/.conan/d04c65/1/build_subfolder'
mingw32-make.exe[1]: *** [CMakeFiles\Makefile2:525: source_subfolder/CMakeFiles/grpc++_reflection.dir/all] Error 2
mingw32-make.exe[1]: *** [CMakeFiles\Makefile2:494: source_subfolder/CMakeFiles/grpc++_error_details.dir/all] Error 2
mingw32-make.exe[1]: Leaving directory 'D:/.conan/d04c65/1/build_subfolder'
mingw32-make.exe: *** [Makefile:132: all] Error 2
grpc/1.25.0@inexorgame/stable:
grpc/1.25.0@inexorgame/stable: ERROR: Package '9067382fae465efcd90e567ccc49e91ed96a97ea' build failed
grpc/1.25.0@inexorgame/stable: WARN: Build folder D:\.conan\d04c65\1
ERROR: grpc/1.25.0@inexorgame/stable: Error in build() method, line 121
        cmake.build()
        ConanException: Error 2 while executing cmake --build D:\.conan\d04c65\1\build_subfolder -- -j4

Maybe you can find something that I missed or did wrong here. Anyways, thanks for looking into this!

Oh, and if I might point you to MSYS2's gRPC package (here). They seem to it working somehow.

Answer 3 · 2020-01-16T18:19:08.000Z

At least it isn't working out of the box

137e40b
https://ci.appveyor.com/project/inexorgame/conan-grpc-xr8rf/builds/30152254
https://travis-ci.org/inexorgame/conan-grpc/builds/637844218

Answer 4 · 2020-01-17T07:02:53.000Z

Would have been too good to be true if it had worked right out of the box.

I took a quick look at the logs you linked.
Since I'm not a Visual Studio expert I can't really tell why it's failing. It has to do something with the linker (but I guess you already figured that out yourself).
The gcc and clang builds look not too bad, though. The only thing that fails is when trying to build the test package it cannot find the shared libraries (see here). I think you might be able to solve that by either setting the LD_LIBRARY_PATH environment variable to point to the directory containing the shared object files or you could simply copy the libraries into the bin folder.

Hope this helps.

Answer 5 · 2020-01-20T08:26:08.000Z

Hi, we also have the same problem here, building shared grpc libraries with Conan for MinGW. Because I'm an absolut Conan beginner, I did not try to fix the Conan build, but I tried to build shared grpc libraries manually to gather as much information as possible that could help to solve the problem.

Building grpc with MinGW as shared library

Building OpenSSL

To match the Conan controlled build I used the OpenSSL library as a package instead of the BoringSSL from third_party folder. To build OpenSSL with MinGW I followed the instructions from here. I download the source from GitHub. On the release page there are 1.0 and 1.1 releases. Because of this problem I had to use the latest 1.0 version of OpenSSL. FindOpenSSL.cmake searches for the 1.0 library names libeay32 and sleay32. Following the instructions from here I could succesfully build OpenSSL.

Building grpc with modules

In the first step I decided, to build all grpc dependencies as moduls except for the SSL library. For the SSL library I configured OpenSSL as package and used the just builded OpenSSL installation. The first problem was, that the FindOpenSSL.cmake script did not detect the right OpenSSL libraries. It found libeay32 and sleay32 DLLs in my ProgrammFiles folder from another application. Therefore I needed to adjust the two cmake settings LIB_EAY and SSL_EAY to point to my just created OpenSSL DLLs.

To build grpc as shared library I set the globale cmake flag BUILD_SHARED_LIBS=ON.

During build, two errors occured. To fix these build errors I had to apply the following patch:

 CMakeLists.txt       | 7 ++++++-
 third_party/protobuf | 0
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 3e9105b49e..552deac371 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -161,6 +161,11 @@ if(MSVC)
   # TODO(jtattermusch): needed to build boringssl with VS2017, revisit later
   set(_gRPC_C_CXX_FLAGS "${_gRPC_C_CXX_FLAGS} /wd4987 /wd4774 /wd4819 /wd4996 /wd4619")
 endif()
+
+if(MINGW)
+  add_definitions(-D_WIN32_WINNT=0x600)
+endif()
+
 set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${_gRPC_C_CXX_FLAGS}")
 set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${_gRPC_C_CXX_FLAGS}")
 
@@ -208,7 +213,7 @@ elseif(UNIX)
   set(_gRPC_ALLTARGETS_LIBRARIES ${CMAKE_DL_LIBS} rt m pthread)
 endif()
 
-if(WIN32 AND MSVC)
+if(WIN32)
   set(_gRPC_BASELIB_LIBRARIES wsock32 ws2_32)
 endif()
 
diff --git a/third_party/protobuf b/third_party/protobuf
--- a/third_party/protobuf
+++ b/third_party/protobuf
@@ -1 +1 @@
-Subproject commit 09745575a923640154bcf307fba8aedff47f240a
+Subproject commit 09745575a923640154bcf307fba8aedff47f240a-dirty

With theses settings I could successfully build grpc as shared library with MinGW.

Building grpc with packages

To match the Conan build more closely, I decided to switch the other grpc dependencies from modules to packages in cmake. That means I first manually builded and installed the following libraries from the third_party folder:

abseil
cares
protobuf
zlib

Afterwards I switched grpc cmake configuration for these libraries from modules to packages. Building went smoothly until 98%. Then the build stopped, because running grpc_verify_jwt.exe failed, because the exe required some DLLs from the other libraries, that where not in the folder of the grpc_verify_jwt.exe file. I could manually fix this problem, by adding the installation directory of the other libraries to the PATH variable:

set PATH=C:\grpc\install;%PATH%

After this change, the build completed successfully. This build error is not exactly the same like the build error in the log provided by Florian in the post above, but it is quite similar and it also happens hat 98%. It looks like some DLLs cannot be found that are required to rund the built exe files. This problem does not occur for static build, because it doe not need to search for DLLs.

Next steps

I will create a pull request for the grpc CMakeLists.txt file and maybe some Conan expert can fix the problem with the DLLs that are not in the search path.

Hope this helps and thank you for all the work you have put into the grpc Conan recipe.

Answer 6 · 2020-02-01T17:02:25.000Z

After a a lot of testing and a lot of searching I finally managed to build shared grpc libraries with MinGW. I just created a pull request (#30) to add support for shared library build with MinGW.

Answer 7 · 2021-05-01T15:33:13.000Z

This recipe is now in the Conan Center Index.

Please migrate to the recipe version there.
If this is still an issue, please create a new issue in the Conan Center Index issue tracker.

Thanks!

https://conan.io/center/grpc