Parsed file contains 0 entities?
DethRaid opened this issue · 14 comments
I'm trying to use cppast to automatically generate code for classes with a specific C++ custom attribute. I have my codegen tool set up to parse all the header files in a specific folder. I can parse them just fine, but when I look through the parse results I only have cpp_file entities - no class entities, not in any file
Here is the code I use to iterate over the entities in a file. When I put a breakpoint here and inspect each file, there's no entities in the file. A screenshot of the VS variable inspector. As you can see, the file's entity container is empty. This is the source file I'm looking at the parse result of. As you can see, this file has a number of entities
Is there anything obviously wrong that I'm doing which would cause cppast to think that my file contains no entities?
I think there was an issue with pragma once, what happens when you run the tool on the header file?
I ran the tool with the command
.\cppast.exe --database_dir src --std c++20 -I"E:\Documents\SanityEngine\SanityEngine\extern\rex\include" -I"E:\Documents\SanityEngine\SanityEngine\extern\physx\include" -I"E:\Documents\SanityEngine\SanityEngine\extern\physx\include\physx" -I"E:\Documents\SanityEngine\SanityEngine\extern\rex\include" -I"E:\Documents\SanityEngine\SanityEngine\extern\tracy" -I"E:\Documents\SanityEngine\SanityEngine\extern\json5\include" -I"E:\Documents\SanityEngine\SanityEngine\extern\dotnet\include" -I"E:\Documents\SanityEngine\SanityEngine\extern\D3D12MemoryAllocator" -I"E:\Documents\SanityEngine\SanityEngine\extern\bve\include" -I"E:\Documents\SanityEngine\SanityEngine\extern\pix\include" -I"E:\Documents\SanityEngine\SanityEngine\extern" -I"E:\Documents\SanityEngine\SanityEngine\src" -I"E:\Documents\SanityEngine\vcpkg_installed\x64-windows\include" -I"E:\Documents\SanityEngine\SanityEngine\src" -DWIN32 -D_WINDOWS -DTRACY_ENABLE -DRX_DEBUG -DGLM_ENABLE_EXPERIMENTAL -D_CRT_SECURE_NO_WARNINGS -DGLM_FORCE_LEFT_HANDED -DNOMINMAX -DWIN32_LEAN_AND_MEAN -DGLFW_DLL -DCMAKE_INTDIR=Debug
with E:\Documents\SanityEngine\SanityEngine as the working directory
I got no output in my terminal
Ok, I'm guessing if you remove the #pragma once it works.
I'll see what I can do.
I tried converting the #pragma once in my header file with a #ifndef include guard, but still had the same problem
I looked through the files included by that file to see if any used #pragma once. A few do. I can edit my local copies of my dependencies - except that MSVC's C++ standard library also uses #pragma once. I do not feel comfortable editing my local copy of the C++ standard library
Okay, running ./cppast SanityEngine/src/adapters/rex/rex_wrapper.hpp -I SanityEngine/extern/rex/include/ in a clone of your repo works for me as intended. Could you post your compile_commands.json?
compile_commands.json (had to rename the file cause GitHub doesn't like .json)
Here is the code I'm using to generate the compilation commands file
OK, so c++20 for --std is not valid, the best you can do is c++1z at the moment. I'm getting an error as expected when trying to run it. You're not getting anything?
I do not get an error with that compilation commands file
My program passes that file to the cppast C++ interface. I don't get any warnings or errors about invalid command line parameters when I run my program
I tried specifying my compilation commands file on the cppast command-line interface. I didn't get any errors about C++ standards, but I did get a different error
PS E:\Documents\SanityEngine\SanityEngine> .\cppast.exe --database_dir .\src -I.\extern\rex\include .\src\adapters\rex\rex_wrapper.hpp
AST for '.\src\adapters\rex\rex_wrapper.hpp':
PS E:\Documents\SanityEngine\SanityEngine> .\cppast.exe --database_file .\src\compile_commands.json -I.\extern\rex\include .\src\adapters\rex\rex_wrapper.hpp
[preprocessor] [critical] .\src\adapters\rex\rex_wrapper.hpp:4: 'core/Prelude.hpp' file not found
[fatal parsing error] preprocessor: command 'C:/Program Files/LLVM/bin/clang.exe -x c++ -E -dD -CC -Xclang -dI -fno-caret-diagnostics -fno-show-column -fdiagnostics-format=msvc -Wno-macro-redefined -Wno-pragma-once-outside-header -Wno-pragma-system-header-outside-header -Wno-include-next-outside-header "-IC:\Program^ Files\LLVM\lib\clang\10.0.0\include" "-D__cppast__=libclang" "-D__cppast_version_major__=0" "-D__cppast_version_minor__=0" "-I.\extern\rex\include" "-std=c++14" ".\src\adapters\rex\rex_wrapper.hpp"' exited with non-zero exit code (1)
I expected the error for the second command. I didn't specify all my include directories
I don't see a way to tell clang itself to run the compilation commands file
I changed the language standard to c++1z in my compilation commands file, then ran my program again. The behavior of my program didn't change. It generated the compilation commands file, then told cppast to parse my files. That took about two minutes, and completed without any errors, but there's still no children entities in any of my cpp_file entities
Just to make sure we're on the same page, here's my clang version:
PS E:\Documents\SanityEngine\SanityEngine> clang --version
clang version 10.0.0
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
I think I encountered a similar issue. For some reason, the cppast tool does not seem to like backslashes in a path to an input file and the resulting AST is empty. If the backslashes in the path are replaced by slashes, the output is generated as expected.
I face similar issue, as @gmueckl suggest, and try to fix this: firodj@35d5155
Ahh, that makes a lot of sense!
Thank you for the fix. I don't have a dev setup on Windows, so if you confirm that it works, please make a PR!
Request to reopen this bug: I'm still getting this problem even with the latest release. My compile_commands.json is generated on a window machine, and so literally every path in there uses \\ as a directory separator. I did a find-replace-all of \\ to / in compile_commands.json, and suddenly the diagnostic logger is printing all sorts of info about the file to my logs! I can't do that find-replace as part of my build process -- cmake just generates that file out of the blue -- so this is kind of impossible to work around at the moment...
Without replacing every backslash with a slash in that file, literally nothing seems to happen (and the debug *.pp files are all blank). Nothing seems to happen, but it sure is taking a lot of time to do the nothing... so it's clearly doing a whole lot of something then discarding it all or something.
Edit: I was able to work around this by reimplementing the parse_database() function to do a std::replace of \\ to / before passing it to parser.parse.
I'm debugging this locally and it seems like the issue is that parse_linemarker (preprocessor.cpp) is parsing a line like
# 1 "C:\\foo\\bar\\baz.h" 2
and replacing each backslash with two backslashes, resulting in
C:\\\\foo\\\\bar\\\\baz.h
which then of course fails to compare equal to C:\foo\bar\baz.h upon returning to detail::preprocess. I changed the contents of the loop to do the opposite, and that fixes the issue for me:
if ( starts_with( p, "\\\\" ) )
{
file_name += "\\";
p.skip();
}
else
file_name += *p.ptr();
Great, can you do a PR?