external symbol g_allocator in header file breaks DelayLoadDLLs on windows
RuurdBeerstra opened this issue · 6 comments
Describe the bug
I use the AWS SDK dll's on Windows in a binary that can also run outside the cloud on arbitrary windows systems.
Support for AWS was added a few years ago and we used the DelayLoadDLLs directive when building our binaries so the DLL's are not required in a non-AWS environment (and not used/loaded even in AWS until it actually uses AWS functionality).
After an upgrade to the latest AWS SDK this was broken: The linker told me I could not use the DelayLoadDLLs for the AWS dll's because of the "g_allocator" symbol.
4>LINK : fatal error LNK1194: cannot delay-load 'aws-crt-cpp.dll' due to import of data symbol '"__declspec(dllimport) struct aws_allocator * Aws::Crt::g_allocator" (_imp?g_allocator@Crt@Aws@@3PEAUaws_allocator@@ea)'; link without /DELAYLOAD:aws-crt-cpp.dll
Digging through the AWS source I found the culprit: aws/crt/StlAllocator.h has the line:
extern AWS_CRT_CPP_API Allocator *g_allocator;
Which implies that any component that includes AWS header files ends up with an extern DATA reference to the g_allocator.
The DelayLoadDLLs can handle references to functions, but because the value of g_allocator must be resolved at start-time of the binary, this one single pointer caused our product to have a very undesirable hard runtime dependency with AWS.
I did a crude hack to get around this. I replaced the "extern" line with this:
extern AWS_CRT_CPP_API Allocator *Get_g_allocator();
#define g_allocator Get_g_allocator()
So everywhere the code uses "g_allocator" it now calls a getter function for the current value.
In crt/aws-crt-cpp/source/Api.cpp (where g_allocator is defined) I added (after the #includes):
/* Undo hack in StlAllocator.h */
#ifdef g_allocator
#undef g_allocator
#else
#error StlAllocator must define g_allocator as a call to Get_g_allocator()
#endif
The #error is just to make sure my includes and defines and up as intended.
The getter function is trivial:
AWS_CRT_CPP_API Allocator *Get_g_allocator();
Allocator *Get_g_allocator() { return g_allocator; }
Building with these modified header files yields a working configuration that no longer suffers from the hard dependency.
The hack is, of course, ugly as sin.
But because there are some 53 uses of "g_allocator" in a some 20 different files, I did it the quick & dirty way.
Expected Behavior
That we could have a single binary that optionally uses AWS dll's (like was possible in an older version of the AWS SDK).
Current Behavior
Linking a Windows binary that uses DelayLoadDLLs on aws-sdk-core.dll yields this error:
4>LINK : fatal error LNK1194: cannot delay-load 'aws-crt-cpp.dll' due to import of data symbol '"__declspec(dllimport) struct aws_allocator * Aws::Crt::g_allocator" (_imp?g_allocator@Crt@Aws@@3PEAUaws_allocator@@ea)'; link without /DELAYLOAD:aws-crt-cpp.dll
Reproduction Steps
The code is not the problem: Try linking a binary that has a DelayLoadDLLs in the vcxproj files likes this:
aws-cpp-sdk-core.dll
And get the error about the DelayLoad not being possible because of the g_allocator symbol.
Possible Solution
Change all occurrences of g_allocator in the header files to a function call that returns the current value.
Additional Information/Context
The (crude) patch provides a workaround for us, so it is currently not an urgent issue.
But I'd like to see a fix for this in a future version.
AWS CPP SDK version used
git clone from April 25, 2022.
Compiler and Version used
Visual Studio 2019
Operating System and version
Windows 2016
Thanks for bringing this up to us. I will see if this is a change that we want to make to this sdk.
I'm not able to reproduce the linker error.
I don't have access to Windows Server 2016, only Windows 10. I took the following steps:
- Installed Visual Studio 2019
- Compiled the SDK like so (64 bit, Release), and installed to
D:\dev\install-sdk-cppon my machine:cmake -B build -S . -G "Visual Studio 16 2019" -A x64 -DBUILD_ONLY=s3 -DCMAKE_PREFIX_PATH=D:/dev/install-sdk-cpp -DCMAKE_INSTALL_PREFIX=D:/dev/install-sdk-cpp -DENABLE_TESTING=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ONcmake --build build --target install --config Release --parallel
- Created a Visual Studio 2019 project, set up like so (I'll attach file to ticket so you can inspect it):
- the main cpp file slept for two seconds before initializing the SDK and making an S3 call.
- I actually tweaked the SDK before compiling so that it would printf something during DLL load. I did this to sanity-check that it was in fact being loaded after a delay.
- I did need to add #define USE_IMPORT_EXPORT to the top the single cpp file, or I got other linker errors
- C/C++ -> General -> Additional Include Directories:
D:\dev\install-sdk-cpp\include;%(AdditionalIncludeDirectories) - C/C++ -> Linker -> General -> Additional Library Directories:
D:\dev\install-sdk-cpp\bin;%(AdditionalLibraryDirectories) - C/C++ -> Linker -> Input -> Additional Dependencies:
aws-cpp-sdk-core.lib;aws-cpp-sdk-s3.lib;%(AdditionalDependencies) - C/C++ -> Linker -> Input -> Delay Loaded Dlls:
aws-cpp-sdk-core.dll;aws-cpp-sdk-s3.dll - Use a Post-Build event to copy the SDK DLLs to be alongside the EXE:
- Build Events -> Post-Build Event -> Command Line:
xcopy /y /d D:\dev\install-sdk-cpp\bin "$(OutDir)"
- Build Events -> Post-Build Event -> Command Line:
- the main cpp file slept for two seconds before initializing the SDK and making an S3 call.
I also tried using Visual Studio 2015, and that worked as well.
If you tweak this project, does it work on your machine? (You'll need to modify that D:\dev\install-sdk-cpp path)
Removing the Aws::Crt::g_allocator extern is a breaking change, so I'm very hesitant to do that unless it's absolutely necessary.
Here's the test project. I included the compiled EXE and DLLs too
MrDelayLoad3.zip
I've been able to reproduce this, and I'm working on a fix that won't break the API...