Unreal 4 and Sol2

[shohnwal]

I'm trying to use Unreal 4 with sol2 in VS2015, using the sol2 single-header file.
The sol2 documentation says if a program is not configured to use C++14 then supposedly there will be tons of errors regarding the 'auto' keyword.

I'm a bit at a loss now whether i made a mistake when configuring UE4 correctly for C++14 or whether sol2 or UE4 is having some other kind of problem... i mean, it doesn't throw any errors for the 'auto' keywords, which the documention says it should if it was not configured properly for C++14. On the other hand, it might be the case that i made a mistake when i tried to configure UE4 for C++14.

When trying to compile my UE4 project there are no errors regarding the 'auto' keyword, instead there are tons of errors regarding the various bool check(...) templates aswell as the struct checker templates (both for the definitions aswell as when they're called in the sol2 code)
All of them have a similar pattern that consists of the same errors, i'll just list one of each error because each error gets listed numerous times
1>..\sol2\Includes\sol.hpp(4083): error C4002: too many actual parameters for macro 'check'

1>..\sol2\Includes\sol.hpp(4083): error C2988: unrecognizable template declaration/definition

1>..\sol2\Includes\sol.hpp(4083): error C2059: syntax error: '<end Parse>'

1>..\sol2\Includes\sol.hpp(4083): error C2143: syntax error: missing ';' before '{'

1>..\sol2\Includes\sol.hpp(4083): error C2447: '{': missing function header (old-style formal list?)

1>..\sol2\Includes\sol.hpp(4088): error C4002: too many actual parameters for macro 'check'

1>..\sol2\Includes\sol.hpp(4098): error C2988: unrecognizable template declaration/definition

1>..\sol2\Includes\sol.hpp(4158): error C2334: unexpected token(s) preceding '{'; skipping apparent function body

aswell as some other errors
1>..\sol2\Includes\sol.hpp(4821): error C4346: 'sol::is_unique_usertype<T>::value': dependent name is not a type

and some warnings with
1>..\sol2\Includes\sol.hpp(1060): warning C4628: digraphs not supported with -Ze. Character sequence '<:' not interpreted as alternate token for '['
That seems to be because sol2 has some lines where <::std::...> i used, like struct tie_size<::sol::tie_t<Tn...>> : ::std::tuple_size<::std::tuple<Tn...>> { };.
Removing the "::" left of std:: and sol:: seems to remove these warnings, and considering a few lines below that std:: and sol:: are used without a "::" prefix i think removing these prefixes shouldn't result in any problem.

Is there any way to tell whether i didn't configure UE4 properly (i tried numerous times with fresh source files changing any entry in the UnrealBuildTool files from "-std=c++11" to "-std=c++14" or -std=c++1z before compiling) or if Sol2 and UE4 are straight up refusing to work with eachother for some reason? I'm quite desperate, i really need Lua for this project and after having used Sol2 for my last 2 projects i refuse to use any other Lua wrapper because it's so great. <3 :)

[ThePhD]

I have a rant prepared if this works.

Try putting #undef check just before you include Sol, see what explodes.

[Rapptz]

Unreal Engine seems to define macros regarding assertions: https://docs.unrealengine.com/latest/INT/Programming/Assertions/

[ThePhD]

Insert long rant about how Unreal's a piece of shit and it shouldn't have macros without a namespace seriously that's an addage as old as C namespace your functions and macros you're not the standard library Unreal JFC.

Rant aside, I can do nothing to help you. check is part of the customization points API, which means to rename it would be a public-facing breaking change. I honestly didn't think a framework as big as Unreal would be this dumb. :I

[shohnwal]

Thank you so much for your answers. While it really frustrates me because i'd rally hate to give up on Sol2, i've tried your first suggestion...
I looked through the UE4 source and looked where they #define check(ed so i can redefine it after the inclusion of sol2.
I found three different #define check(s in UE4:

1. in \Engine\Source\Runtime\Core\Public\Misc\AssertionMacros.h(30)

#define check(expr) { if(UNLIKELY(!(expr))) { FDebug::LogAssertFailedMessage( #expr, __FILE__, __LINE__ ); _DebugBreakAndPromptForRemote(); FDebug::AssertFailed( #expr, __FILE__, __LINE__ ); CA_ASSUME(false); } }

2. \Engine\Source\Runtime\Core\Public\Misc\AssertionMacros.h(70)
   #define check(expr) { CA_ASSUME(expr); }
   however that one was greyed out because it took 1) instead
   ( 1) and 2) were mutually exclusive defined in an #ifdef.... 1) #else ... 2) )

3. \Engine\Source\Runtime\Core\Public\HTML5\HTML5AssertionMacros.h(42)

#define check(expr) { if (!(expr)) { \
        emscripten_log(255, "Expression '" #expr "' failed in " __FILE__ ":" PREPROCESSOR_TO_STRING(__LINE__) "!\n"); \
        html5_break_msg(#expr, __FILE__, __LINE__); \
    } \

I assume that the most logical thing would be that 1) is needed because 2) is greyed out and it probably is not 3) since i didn't do anthing that does HTML5. And even if something goes wrong it won't be hard to play around and find out which one has to be redefined after i included sol2.

So what i did is #define check again after sol.hpp

#undef check
#include "sol.hpp"
#define check(expr) { if(UNLIKELY(!(expr))) { FDebug::LogAssertFailedMessage( #expr, __FILE__, __LINE__ ); _DebugBreakAndPromptForRemote(); FDebug::AssertFailed( #expr, __FILE__, __LINE__ ); CA_ASSUME(false); } }}

And that seems to get rid of the errors i listed above.

There are now other errors, but far less than before, and of different types, only 2 lines are causing an error now, one is a destructor :

2>\sol2\Includes\sol.hpp(1028):` error C4583: 'sol::storage_t<T>::value_': destructor is not implicitly called
2>          with
2>          [
2>              T=sol::basic_object<sol::reference>
2>          ]
2> \sol2\Includes\sol.hpp(1023): note: while compiling class template member function 'sol::storage_t<T>::storage_t(sol::trivial_init_t) noexcept'
2>          with
2>          [
2>              T=sol::basic_object<sol::reference>
2>          ]
2> \sol2\Includes\sol.hpp(1051): note: see reference to function template instantiation 'sol::storage_t<T>::storage_t(sol::trivial_init_t) noexcept' being compiled
2>          with
2>          [
2>              T=sol::basic_object<sol::reference>
2>          ]

and one is a constructor:

2>\sol2\Includes\sol.hpp(1023): error C4582: 'sol::storage_t<T>::value_': constructor is not implicitly called
2>          with
2>          [
2>              T=sol::basic_object<sol::reference>
2>          ]
2>  \sol2\Includes\sol.hpp(1023): note: while compiling class template member function 'sol::storage_t<T>::storage_t(sol::trivial_init_t) noexcept'
2>          with
2>          [
2>              T=sol::basic_object<sol::reference>
2>          ]
2>  \sol2\Includes\sol.hpp(1051): note: see reference to function template instantiation 'sol::storage_t<T>::storage_t(sol::trivial_init_t) noexcept' being compiled
2>          with
2>          [
2>              T=sol::basic_object<sol::reference>
2>          ]

So undefining and redefining check before/after #include "sol.hpp" seems to have worked and it seems that with a few workarounds sol2 might work with UE4 afterall. :)

If you're still willing to help me solving these new issues i'd greatly appreciate it.

If everything fails i'll probably just try to write a C-style Sol2-wrapper/translator and compile it + the Sol2 hpp as a dll and then just try to link it up with UE4 so it's "Sol2 C++14 -> C interface -> UE4 C++11".

[ThePhD]

I've never seen this error before: 2>\sol2\Includes\sol.hpp(1028): error C4583: 'sol::storage_t<T>::value_': destructor is not implicitly called. Do you know what causes it specifically? I mean, it LOOKS like it's just from optional, in which case... that's a very bizarre error to come from Unreal. If it's compiling under C++14 (the default for Visual Studio), that should just... work.

[shohnwal]

Sorry, i don't know what specifically causes that.
When i double click the error message it takes me to this template, I've marked/commented the lines that cause the errors.

template <class T>
union storage_t
{
  unsigned char dummy_;
  T value_;

  constexpr storage_t( trivial_init_t ) noexcept : dummy_() {}; //<<< causes error C4582

  template <class... Args>
  constexpr storage_t( Args&&... args ) : value_(constexpr_forward<Args>(args)...) {}

  ~storage_t() { } // <<< causes error error C4583
};

The thing is, according to the UE4 coding standard
https://docs.unrealengine.com/latest/INT/Programming/Development/CodingStandard/index.html
they seem to be fixated on staying with C++11 to guarantee absolute compatibility with all common compilers, so i'm not sure if the Visual C++ compiler, despite being C++14 by default, can even compile it with C++14 if they somehow force it to be C++11. To quote them :

However, certain language features we may opt to avoid entirely until we are confident we will not be surprised by a new platform appearing that cannot digest the syntax.

If i understand that correctly, it might be the case that they disable certain C++11 features. Do i understand that correctly? Could it be that they deactivated features like optional?

Anyway, i've looked up what these compiler warnings actually mean
https://msdn.microsoft.com/en-us/library/mt694070.aspx
but for these two compiler warnings there is no elaboration, no explanation, yay -_-
I've opened a ticket at the microsoft Visual studio community forums and wait for someone to explain what these compiler warnings actually mean.

[shohnwal]

OK, some interesting news:
I managed to modify the single header sol-hpp so it actually compiles with UE4.

Lets take a look at this template in the sol.hpp:

template <class T>
union storage_t
{
  unsigned char dummy_;
  T value_;

  constexpr storage_t( trivial_init_t ) noexcept : dummy_() {};

  template <class... Args>
  constexpr storage_t( Args&&... args ) : value_(constexpr_forward<Args>(args)...) {}

  ~storage_t(){}
};

The reason why the compiler complains about implicit constructors/destructors is that it does not know how to initialize and delete T value_, the constructor provides a default initialization for dummy_but not for T value_.
God knows why this only occurs in combinationwith UE4, it works perfectly fine with any other software project i've done so far.

Now, i've fiddled around with some random alterations (REALLY random alterations, i've been on this for the entire night without sleep so i've no idea what i was doing XD) a bit and it seems like changing it like this makes the VS2015 compiler stop complaining:

template <class T>
struct storage_t
{
  T value_;

  constexpr storage_t( trivial_init_t ) noexcept : value_(unsigned char(0)) {};

  template <class... Args>
  constexpr storage_t( Args&&... args ) : value_(constexpr_forward<Args>(args)...) {}

  ~storage_t() { this->value_.T::~T(); }
};

in short what i did:
-changed from union to struct
-removed the _dummy member variable, it wasn't called anywhere in the first place
-add : value_(unsigned char(0)) to the constructor initializer list
-add this->value_.T::~T(); to the destructor

for the following reasons:

-if i didn't change it to a struct the compiler would complain (but with only one data member, i don't think it would need to be a union anyway)
-if i didn't give unsigned char(0)) as default parameter in the trivial constructor but left the brackets empty, the compiler would complain ("can't find viable default constructor for T value_")
-if i didn't put the destructor stuff in there, the compiler would complain about the destructor being called implicitly

With these changes it even compiles with the unmodified default UE4 C++11!!!
Not sure why, considering your manual says Sol2 needs C++14 and UE4 supposedly only accepts C++11 but ohwell, i'm not complaining... XD

Now the thing is : i haven't tested all Sol2 features yet.
I have no idea what consequences these changes i did actually have.
I don't know if giving value_ an unsigned char(0)as initialize value has any negative consequences, i just assumed that because dummy_ was an unsigned char that got default-initialized by the constructor that i could maybe instead initialize value_ with an ùnsigned char(0) and thus have the same starting value for the "union" (which i changed to a struct since i threw the seemingly useless _dummy out).

Then i don't know if adding this->value_.T::~T(); to the destructor has any negative consequences, considering the

template <class T>
struct optional_base
{

a few lines further down has a destructor calling
~optional_base() { if (init_) storage_.value_.T::~T(); },
which possibly could result in the storage_t's value_ destructor getting called twice (once by optional_base and once by the storage_t modification that i've made to make the compiler shut up), but i'm not sure.

However, i was able to compile UE4+LuaJit+Sol2, call the Sol2 lua instance ingame with some simple lines like

    this->lua.set("number", 24);
    this->lua["number2"] = 25;
    this->lua.script("abc = { test = 100 }");
    int32 x = (int32)lua["abc"]["test"] + (int32)this->lua["number"];
    std::string value = this->lua.script("return number + number2");
    GEngine->AddOnScreenDebugMessage(-1, 15.0f, FColor::Red, FString(value.c_str()));
    GEngine->AddOnScreenDebugMessage(-1, 15.0f, FColor::Red, FString::FromInt(x));

which allowed me to test table access and return ints and strings without any problems, forward it to UE4s blueprint scripting system and print it out as a UE4 debug message on screen, so at least it seems to be working in these simple cases. ^___^

Haven't tested more complicated stuff, like creating objects, though....

However, as mentioned above, i have no idea what consequences the changes i've made actually have since i've no idea what the storage_t and optional_base are used for and if the simple test cases i've listed above even used the storage_t and optional_base, maybe you can judge if these changes are acceptable/safe or, incase i broke something but didn't notice it yet, if it gives you an idea how it could be made more compatible.

If you want to test it or, provided everything works and i didn't break anything by accident, want to apply it to your Sol2, here is a short summary of all the changes i've made so far:

1. removing the "::" in front of ::std:: and ::sol:: so they only say std:: and sol::
   -solves the "unsupported digraphs, '<:' won't be interpreted as ']' " error

2. undef/redef UE4s check macro around the #include "sol.hpp"

#undef check
#include "sol.hpp"
#define check(expr) { if(UNLIKELY(!(expr))) { FDebug::LogAssertFailedMessage( #expr, __FILE__, __LINE__ ); _DebugBreakAndPromptForRemote(); FDebug::AssertFailed( #expr, __FILE__, __LINE__ ); CA_ASSUME(false); } }}

-solves the conflict with UE4s "error with macro 'check' " error

3. changes to the

template <class T>
union storage_t

which i have listed in this post, which solves the "no implicit constructor/destructor" error

[ThePhD]

I put in the fixes for things like ::std:: and ::sol::, since they're no longer needed.

I don't even know what to tell you about the storage_t for optional, though. The changes you made honestly look fundamentally broken from a C++ point of view. For example, you're deleting the member of an object in a struct manually: that's borderline disastrous (and potentially up to 3 deletes total instead of just the 1 necessary one).

I'll... look into a way of programming it and ship you a header specifically: if it works, then I'll make sure it's correct and then proceed.

As far as the check thing is concerned, it's completely out of my hands. There should be a bug report to Unreal for them to fix that absolutely broken Macro.

[ThePhD]

You should be all set: https://github.com/ThePhD/sol2/releases/tag/v2.14.8

Good luck.

[ThePhD]

As a side note, I attempted to look at the Unreal 4 source and build this check natively into Sol2, so you don't have to: https://github.com/ThePhD/sol2/releases/tag/v2.14.9