How to include msvcr100.dll in Windows native image
remkop opened this issue · 28 comments
Related to #1407: I'm trying to create an instruction manual for creating native image command line applications using the picocli library.
On Windows 10, the generated native image cannot be executed on a clean machine; it fails with an error like the following:
Note that this problem does not manifest on the machine where the native image was built. The DLL will be present on machines that have the toolchain set up to build the native image (Microsoft Windows SDK for Windows 7 and .NET Framework 4 and the C compilers from KB2519277). The problem only manifests when you try to execute the generated native image on a "clean" Windows 10 machine. (I used a VirtualBox VM with a 64-bit Windows 10 guest OS to reproduce the issue.)
It turns out that msvcr100.dll from VS C++ Redistributable 2010 must be present or the application will fail to start. I would like to include msvcr100.dll in the native image so my application can be distributed as a single executable.
Will this DLL be included automatically in the native image in future releases? If not, is there a way for me ensure it is included?
Steps to reproduce:
C:\Temp\issue1762>echo public class HelloWorld { public static void main(String[] args) { System.out.println("Hello World"); } } > HelloWorld.java
C:\Temp\issue1762>javac HelloWorld.java
C:\Temp\issue1762>C:\apps\graalvm-ce-19.2.1\bin\native-image -cp . HelloWorld
C:\Temp\issue1762>dir
...
22/10/2019 01:16 421 HelloWorld.class
22/10/2019 01:20 8,235,520 helloworld.exe
22/10/2019 01:20 263,425 helloworld.exp
22/10/2019 01:16 108 HelloWorld.java
22/10/2019 01:20 439,264 helloworld.lib
22/10/2019 01:20 1,338,368 helloworld.pdb
C:\Temp\issue1762>helloworld.exe
Hello World
Test on clean machine:
- install Oracle VM VirtualBox
- create new virtual machine with guest OS 64-bit Windows 10
- copy
helloworld.exeto this VM (see shared folders) - open DOS console and execute
helloworld.exe - this will show the above
System Error - The program can't start because MSVCR100.dll is missing from your computer...error dialog
GraalVM must use /MT compiler flag to static CRT link. But it use /MD flag.
@webfolderio Thanks for the link!
I understand now that a module compiled with MT will have the runtime "inside it", while a module compiled with MD will link with a DLL at the moment of execution. When compiled with MD the DLL must exist on the target machine, which is the problem because MSVCR100.dll is often missing.
The comment in the source says:
Must use /MD in order to link with JDK native libraries built that way
I found that MD compiled DLLs should not be mixed with MT compiled DLLs. This is probably what the comment is referring to.
So, in order to fix this, we need to get MT compiled versions of some JDK native libraries? Does anyone know which JDK native libraries the comment refers to?
As most of jdk native libraries are provided as static lib and already included in compiled exe, I think we can use /MT mode at most time. But if --enable-all-security-service is enabled and so sunec.dll must be present, /MT maybe cause crash.Maybe you can use some third party crypto libraries instead of jce in this case.
@bobvandette Would it be possible to use /MT by default, and only switch to /MD when necessary, e.g., when we need to link with a JDK native library built that way? (Which JDK native libraries are built with MD?)
The JDK static libraries for Windows are all built with /MD. I would have to experiment with building them with /MT to see if this could work. I would expect your generated native images to increase in size if we did this.
That makes sense. The msvcr100.dll file is ~810 KB but I suppose other libraries may be pulled in too. It would be good to better understand the trade-offs.
For many CLI applications, the ability to distribute apps as a single executable is a big part of the value of native images. It's a pity that we cannot rely on msvcr100.dll always being present.
Thank you for spending time on this!
@bobvandette Have you had a chance to try this? Is it feasible to remove the dependency on an external msvcr100.dll?
Any chance of getting this in GraalVM 19.3?
I did look into this and came to the conclusion that this wasn't something we should do since you cannot mix executables and libraries that were built with /MD and /MT. There are use cases where we generate DLL's that are used with the JDK and other cases where .DLL's from the JDK are used to augment a generated native-image.
The only way around this would be to build your own JDK and native-image binaries with some minor modifications to the sources. Sorry.
@bobvandette I am aware that /MD and /MT cannot be mixed, and I understand there are use cases where /MD is unavoidable.
What I was hoping was that the native-image generator tool could be enhanced to offer an option on Windows to build with /MT, where the tool would exit abnormally when it detects that /MT cannot be used because of the use cases you mention (required dependency on other .DLL's from the JDK, for example).
@shelajev I finally had a chance to try this with GraalVM 19.3 for Java 11. The issue remains with a different error (because native-image with Java 11 uses a different toolchain). The Japanese error message reads:
System Error
The program can't start because VCRUNTIME140.dll is missing from your computer. You may be able to resolve this by reinstalling the program.
@bobvandette raises a good point that executables and libraries that were built with /MD and /MT cannot be mixed, so this needs to be done with care.
However, I expect a similar request will be raised to allow statically linking libstdc++ on the linux/maxos platforms, because since 19.3 the sunec lib gets statically linked into the image when needed, which means that on Linux and macOS the image will depend on libstdc++.
I propose adding a new option to native-image, say, -H:+StaticallyLinkAll. Using this option would result in all dependencies being statically linked (with /MT on windows). There may be restrictions on the resulting images (for example, generate .exe only, no .dll's; cannot be used with dynamically linked libs, etc.), but this option would allow developers to distribute their application as a single executable file.
Minor point -- I think the message actually refers to VCRUNTIME140.dll.
Minor point -- I think the message actually refers to VCRUNTIME140.dll.
@sogaiu Thank you! I fixed the typo in my previous comment.
allow developers to distribute their application as a single executable file
This sounds nice if doable!
The suggestion of implementing -H:+StaticallyLinkAll sounds useful but has a few challenges associated with its implementation.
GraalVM 19.3 switched to using JDK native code instead of the manual substitutions that were used for earlier versions of GraalVM. The JDK native code is delivered in GraalVM in the form of static libraries. These static libraries (zip.a, net.a, java.a, nio.a, etc) are built on Windows using /MD. The reason for this is that it allows the use of other JDK DLLs or third party DLL with native-image generated .exe's. If -H+StaticallyLinkAll were implemented, we would have to provide another set of these static libraries that are compiled using /MT and find a way of prohibiting the use of other DLLs or static libraries that were not properly compiled.
The mac/linux example described above is not the same problem since Linux and Mac do not have a problem mixing shared libraries and static libraries in a single executable. The solution to the libstdc++ problem is to optionally add the static version of that library on the link line in order to eliminate the runtime dependency on the shared library. This of course assumes that this static version is available in the toolchain. I don't see this library in my Xcode installation.
@bobvandette Is this related? RFR: 8239563 - Reduce public exports in dynamic libraries built from JDK static libraries
No, sorry. That fix is about avoiding adding export symbols from the JDK static libraries to native-image generated DLLs.
Is there any plans to solve this issue?
It prevents providing an autocontained binary for windows.
Workaround:
Use pefrmdllembed. Syntax:
pefrmdllembed -impinj VCRUNTIME140.DLL your-native-image.exe your-native-image-fixed.exe
Worked for me.
Workaround:
Use pefrmdllembed. Syntax:
pefrmdllembed -impinj VCRUNTIME140.DLL your-native-image.exe your-native-image-fixed.exe
Worked for me.
@mmellon Awesome, thank you so much! I will include your workaround in the picocli documentation. 👍
Caveat for the above workaround:
If you embed MSVCR100.DLL or VCRUNTIME140.DLL in your exe rather than create an installer that installs a dependency, you will probably lose the ability of Windows to apply updates for vulnerabilities in the shared runtime that might impact the security of your application. Windows installer has magic to keep you from downgrading the DLL in %System32%, and, once it is installed, I believe Windows Update keeps it up-to-date.
You don't have to statically link to the CRT to get rid of the "VCruntime140.dll" error. Windows has had "universal CRT" deployed as part of Windows for a decade now. Code should simply link with that dynamically and it will work with anything Windows 10 and later. Prior versions of Windows can install the Universal CRT separately.
See https://learn.microsoft.com/en-us/cpp/windows/universal-crt-deployment
@kasajian Thank you for pointing that out, I was not aware of that!
Windows has had "universal CRT" deployed as part of Windows (...) Code should simply link with that dynamically (...)
Is that something that application authors (using the native-image tool and MS toolchain) can control, or is this something that the GraalVM project needs to do?
@remkop In my opinion, the Graal code that creates the Windows executable is currently configured to some settings (looks like very old settings) to like the C-Runtime in a certain way that requires additional distribution by the end-user (the person creating the install. It's unnecessary. Today, and for a long time, when you create an EXE on Windows, it will built in such as way that it will link to the version of CRT that ships with Windows, and kept up-to-date by Microsoft.
More info from Microsoft: https://learn.microsoft.com/en-us/cpp/windows/universal-crt-deployment?view=msvc-170
Is this absolutely needed? no. This is a feature request, not a bug. But it would be good if Graal attempted build the Windows image the way Windows images are typically built these days.
bumping this up - would be super helpful to remove this dependency, if all that is required is to change the link process to rely on UCRT
For the record, when targeting Windows this issue makes GraalVM Native Image much less attractive compared to a jlink image/jpackage installer.
Is this absolutely needed? no. This is a feature request, not a bug. But it would be good if Graal attempted build the Windows image the way Windows images are typically built these days.
@kasajian this sound like a useful feature. Please create a new Feature request with a proper title so that we can close this issue here. Btw, we also accept pull requests ;-)
cc @pejovica