Tradias/asio-grpc

Need some basic help!

Closed this issue · 9 comments

vtharmalingam commented

Hi,

Firstly, I want to express my gratitude to the team for their outstanding work on this project. Over the past three years, I've had the pleasure of successfully utilizing Boost::Beast, which has given me a deep appreciation for the power of ASIO. Additionally, I've been intrigued by the potential of integrating gRPC into my projects, but I've found it challenging due to my lack of confidence in the build process. To address this, I took the initiative to learn tools such as CMake, Conan, and a bit of Bazel.

Your work has particularly caught my attention because it combines the strengths of both ASIO and gRPC. I've spent several days attempting to integrate it into my projects using both CMake and Conan, but unfortunately, I haven't had much success. Despite this setback, I'm determined not to give up.

I'm seeking your advice on how frameworks like this can be more easily integrated. Perhaps there's a secret sauce that I'm missing, and I'm struggling to identify where I can learn more about it.

For instance, my goal is to develop a C++ server using asio-grpc to serve clients written in PHP and Python, seamlessly exchanging protobuf messages. I'm also interested in creating a trimmed version of my server, which currently relies on Boost::Beast.

Given these use cases, the codebase appears somewhat daunting to me. Could you please provide guidance on the steps involved in building this library as a standalone component, enabling me to integrate it into my project effortlessly?

1. Find the package using 'find_package'
2. What paths to be added to "target_include_directories"
3. What libs are to be added to "target_link_libraries"

As said, I tried with Conan and did "install" using the below CMakeLIsts.txt, but I ran into several LINK errors:

CMakeLists.txt

cmake_minimum_required(VERSION 3.15)

project(asio_example CXX)

# Minimum C++ standard
set(CMAKE_CXX_STANDARD 20)

find_package(ASIO-GRPC REQUIRED)

add_executable(${PROJECT_NAME} src/file-transfer-server.cpp)

target_link_libraries(${PROJECT_NAME}  PRIVATE asio-grpc::asio-grpc )

Install command

conan install . --output-folder=build --build=missing -s compiler.cppstd=20

LINK Errors

libprotobuf-lite.lib(libprotobuf-lite.dll) : error LNK2005: "public: virtual __cdecl google::protobuf::io::StringOutputStream::~StringOutputStream(void)" (??1Str       
ingOutputStream@io@protobuf@google@@UEAA@XZ) already defined in libprotobuf.lib(message_differencer.obj) [C:\Users\thavi\.conan2\p\b\grpc67bcbddea9216\b\bu       
ild\grpc_ruby_plugin.vcxproj]
libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: bool __cdecl google::protobuf::ServiceDescriptor::GetSourceLocation(struct google::protobuf::SourceLoc       
ation *)const " (?GetSourceLocation@ServiceDescriptor@protobuf@google@@QEBA_NPEAUSourceLocation@23@@Z) already defined in libprotobuf.lib(descriptor.obj) [C:\Use       
rs\thavi\.conan2\p\b\grpc67bcbddea9216\b\build\grpc_ruby_plugin.vcxproj]
libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: class google::protobuf::Descriptor const * __cdecl google::protobuf::MethodDescriptor::input_type(void       
)const " (?input_type@MethodDescriptor@protobuf@google@@QEBAPEBVDescriptor@23@XZ) already defined in libprotobuf.lib(descriptor.obj) [C:\Users\thavi\.conan       
2\p\b\grpc67bcbddea9216\b\build\grpc_ruby_plugin.vcxproj]
libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: class google::protobuf::Descriptor const * __cdecl google::protobuf::MethodDescriptor::output_type(voi       
d)const " (?output_type@MethodDescriptor@protobuf@google@@QEBAPEBVDescriptor@23@XZ) already defined in libprotobuf.lib(descriptor.obj) [C:\Users\thavi\.con       
an2\p\b\grpc67bcbddea9216\b\build\grpc_ruby_plugin.vcxproj]
libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: bool __cdecl google::protobuf::MethodDescriptor::GetSourceLocation(struct google::protobuf::SourceLoca       
tion *)const " (?GetSourceLocation@MethodDescriptor@protobuf@google@@QEBA_NPEAUSourceLocation@23@@Z) already defined in libprotobuf.lib(descriptor.obj) [C:\Users       
\thavi\.conan2\p\b\grpc67bcbddea9216\b\build\grpc_ruby_plugin.vcxproj]
libprotobuf.lib(libprotobuf.dll) : error LNK2005: "public: bool __cdecl google::protobuf::FileDescriptor::GetSourceLocation(class std::vector<int,class std::allo

Apologies for any inconvenience caused, but I'm genuinely eager to learn and gain knowledge about the build process, particularly because I'm enthusiastic about utilizing this library. So whatever guidance and pointers you could give, i will be really grateful to you.

Thanks,
Tharma

Tradias commented

Hi, thanks for your detailed report. I personally do not use conan but I have a little project where I smoketest asio-grpc with conan. Using default setting with MSVC it builds static libraries for me, for you it seems like you have selected shared libraries (at least for protobuf) and now tries to link with both libprotobuf-lite and regular libprotobuf. I tried to reproduce that without luck.

Here is the conanfile.txt that I used for the reproduction:

[requires]
asio-grpc/2.9.2

[generators]
CMakeDeps
CMakeToolchain

[layout]
cmake_layout

[options]
protobuf/*:shared=True

After running the conan install command, I do

cmake --build --preset conan-release
vtharmalingam commented

Thank you so much @Tradias for your quick response!
Further, I moved to CMake instead, and still face a last bit of problems as stated below in getting my feet wet on this package:

I followed the below instructions of the README.md, and it was successful in installing the library.
Though I do not understand the significance of "lib" folder inside the install directory is filled with many cmake files. Maybe that got anything to with my problem, not sure...

image

In stage 2, where I wanted to integrate the library to my target. Here is the snippet of CMakeLists.txt that builds my target linking asio-grpc:

set (CMAKE_PREFIX_PATH C:/Users/Downloads/asio-grpc/install)
find_package(asio-grpc)

## ...

add_executable(WT    hello-world-server.cpp ${HEADERS}  ${SOURCES} )

##...


# Specify include directories if your project has header files in multiple directories.
target_include_directories(WT 
    PRIVATE
     #asio-grpc
    ${CMAKE_PREFIX_PATH}/include
    ${CMAKE_SOURCE_DIR}/helper
  
    #... 
)

target_link_libraries(WT 
        PUBLIC  
       
        asio-grpc::asio-grpc
        Boost::headers

        #..
)

After configuring my CMakeLists.txt, when I build it, I get the below error:

C:\thavi\CPP_WS\asio-grpc-experiment\helper\awaitable_server_rpc.hpp(24,41): error C2039: 'use_awaitable_t': is not a member of 'boost::asio' [C:\TH
ARMA\CPP_WS\asio-grpc-experiment\build64\WT.vcxproj]
C:/thavi/vcpkg/installed/x64-windows/include\boost/asio/ssl/stream.hpp(39,11): message : see declaration of 'boost::asio' [C:\thavi\CPP_WS\mt5\manager\mt5_ib_to 
olstandalone\build64\WT.vcxproj]
C:\thavi\CPP_WS\asio-grpc-experiment\helper\awaitable_server_rpc.hpp(24): error C2061: syntax error: identifier 'use_awaitable_t' [C:\thavi\CPP_WS\ 
asio-grpc-experiment\build64\WT.vcxproj]
C:\thavi\CPP_WS\asio-grpc-experiment\helper\awaitable_server_rpc.hpp(24,60): error C2039: 'as_default_on_t': is not a member of '`global namespace'' 
 [C:\thavi\CPP_WS\asio-grpc-experiment\build64\WT.vcxproj]
C:\thavi\CPP_WS\asio-grpc-experiment\hello-world-server.cpp(16,1): fatal  error C1083: Cannot open include file: 'helloworld/helloworld.grpc.pb.h':  
No such file or directory [C:\thavi\CPP_WS\asio-grpc-experiment\build64\WT.vcxproj]

Primarily the issue is about missing "use_awaitable_t"... not clear if my verion of Boost::asio (1.75) is missing it perhaps?
Maybe you can take a look at my environment (snapshot below):

image

Other general question:
Since it is a header-only library, why do we need to link "asio-grpc::asio-grpc". Sorry for the basic question.

Thanks for your help in advance!

Thanks,
Tharma

Tradias commented

In the long run I highly recommend using a package manager. At work and personally I use vcpkg.

I assume in your CMake you forgot to add:

target_compile_features(WT PRIVATE cxx_std_20)

Specifying a target in target_link_libraries is CMake's magic way of propagating any kind of build requirements to your target. This includes: libraries to link, directories to include, pre-processor definitions, compile features/flags, .... In the case of asio-grpc that would be linking with gRPC::grpc++, enabling compile feature cxx_std_17 and adding pre-processor definition AGRPC_BOOST_ASIO. In general it is a good idea to specify all direct dependencies in a target_link_libraries call, regardless of whether they are header-only or compiled.

Also at some point you will have to generate source code from .proto files. You can do so using asio-grpc's helper function:

asio_grpc_protobuf_generate(
    GENERATE_GRPC
    TARGET WT
    OUT_DIR "${CMAKE_CURRENT_BINARY_DIR}/generated"
    IMPORT_DIRS "${CMAKE_CURRENT_SOURCE_DIR}/proto"
    PROTOS "${CMAKE_CURRENT_SOURCE_DIR}/proto/helloworld/helloworld.proto")
vtharmalingam commented

Thank you so much! I had a valuable learning experience with this cmake function, target_compile_features. Additionally, I proceeded with "vcpkg" as you suggested, and I must say, I found it to be incredibly user-friendly.

Regarding the linking errors, you were absolutely correct! When I replaced cxx_std_17 with cxx_std_20 in the code below, the issues were resolved:

target_compile_features(WT PRIVATE cxx_std_20)

With that, I could "configure" (cmake . -B ./build) without any error.
But when I "build" ( cmake --build ./build --config Release or cmake --build ./build --config Debug ),
I face just these two errors (details below):

C:/Users/thavi/Downloads/asio-grpc/install/include\agrpc/detail/register_awaitable_rpc_handler.hpp(58,15): error C2672: 'co_spawn': 
no matching overloaded function found

C:\thavi\build\generated\helloworld\helloworld.pb.cc(43,80): error C2127: 'helloworld::_HelloReply 
_default_instance_': illegal initialization of 'constinit' entity with a non-constant expression

I was wondering if you could provide guidance on this as well? Thanks for your help, in advance.

Thanks,
Tharma

Tradias commented

Here is a complete minimal gRPC helloworld example: https://github.com/Tradias/example-vcpkg-grpc/tree/asio-grpc-97

The register_awaitable_rpc_handler.hpp error seems to be a Boost 1.75 specific thing where asio forgot to add type traits for executor_with_default of asio::use_awaitable_t and misdetects MSVC's decltype capabilities. Nothing I can do on my side but if you use the example above that won't be an issue (because it will use Boost 1.83).

The protobuf error is this one protocolbuffers/protobuf#8688 and was fixed in a later protobuf release. Again, none issue if you use my example.

vtharmalingam commented

@Tradias :

The minimal gRPC "helloworld" example worked seamlessly! Interestingly, while examining that example, I gained insights into certain concepts related to both CMake and VCPKG as well :) I truly appreciate your guidance throughout this process. BTW, I've successfully upgraded my Boost version to the latest release (1.84.0).

However, upon attempting to integrate the success into my production application, I encountered the following linking error:

It has been quite perplexing, and I've spent a significant amount of time troubleshooting since yesterday. I understand that I've already taken your valuable time, so if you have a chance to provide any insight or guidance into this error, I'd be grateful. Otherwise, no worries; I sincerely appreciate all the assistance you've provided thus far. You are the best!

grpc_unsecure.lib(bdp_estimator.cc.obj) : error LNK2001: unresolved external symbol __imp_rand [C:\thavi\experiment\asio-grpc-prod\build64\src\WT...]

Here is the complete list of .obj files that have linking problem due to missing the exact sample symbol:

grpc_unsecure.lib(bdp_estimator.cc.obj) 
grpc_unsecure.lib(stream_map.cc.obj) 
grpc_unsecure.lib(server.cc.obj 
grpc_unsecure.lib(dns_resolver_ares.cc.obj) 
grpc_unsecure.lib(channel_idle_filter.cc.obj) 
grpc_unsecure.lib(round_robin.cc.obj) 
grpc_unsecure.lib(flow_control.cc.obj)

Thanks,
Tharma

vtharmalingam commented

@Tradias : Sorry to bother you. This issue has become a roadblock for me. I tried on another project too, but I ran into exactly the same issue. I looked at your [Documentation] (https://tradias.github.io/asio-grpc/md_doc_v3_migration.html) and I tried like this, but with no luck:

set(ASIO_GRPC_DISABLE_AUTOLINK on)
find_package(asio-grpc)
find_package(gRPC)
 
target_link_libraries(your_app PUBLIC asio-grpc::asio-grpc gRPC::grpc++_unsecure)

I am unable to proceed and got stuck for almost 2 days! Any quick help will be really appreciated.

Thanks,
Tharma

Tradias commented

It very much sounds like a gRPC issue, I don't think I am the right person to help here. I suspect it has something to do with the flags you used when compiling gRPC, maybe a mix of /MD and /MT. You could try the following, but I doubt it will help:

set(ASIO_GRPC_DISABLE_AUTOLINK on)
find_package(asio-grpc)
find_package(gRPC)
 
target_link_libraries(your_app PUBLIC asio-grpc::asio-grpc gRPC::grpc++)
vtharmalingam commented

Sure, thank you, @Tradias. I will dig further into this.

Thanks,
Tharma