standardese/cppast

Failure to parse some complicated preprocessor stuff

GabrielRavier opened this issue · 2 comments

GabrielRavier commented
  • cppast version: latest
  • parser: libclang_parser
  • clang version: 10.0.0

cppast seems to be choking on some weird preprocessing stuff related to boost. The boost code that is failing is as such :

#include <boost/mpl/aux_/config/compiler.hpp>
#include <boost/mpl/aux_/config/preprocessor.hpp>
#include <boost/mpl/aux_/config/workaround.hpp>
#include <boost/preprocessor/cat.hpp>
#include <boost/preprocessor/stringize.hpp>

#if !defined(BOOST_NEEDS_TOKEN_PASTING_OP_FOR_TOKENS_JUXTAPOSING)
#   define AUX778076_PREPROCESSED_HEADER \
    BOOST_MPL_CFG_COMPILER_DIR/BOOST_MPL_PREPROCESSED_HEADER \
/**/
#else
#   define AUX778076_PREPROCESSED_HEADER \
    BOOST_PP_CAT(BOOST_MPL_CFG_COMPILER_DIR,/)##BOOST_MPL_PREPROCESSED_HEADER \
/**/
#endif

#if BOOST_WORKAROUND(__IBMCPP__, BOOST_TESTED_AT(700))
#   define AUX778076_INCLUDE_STRING BOOST_PP_STRINGIZE(boost/mpl/aux_/preprocessed/AUX778076_PREPROCESSED_HEADER)
#   include AUX778076_INCLUDE_STRING
#   undef AUX778076_INCLUDE_STRING
#else
#   include BOOST_PP_STRINGIZE(boost/mpl/aux_/preprocessed/AUX778076_PREPROCESSED_HEADER) // This line errors
#endif

#   undef AUX778076_PREPROCESSED_HEADER

#undef BOOST_MPL_PREPROCESSED_HEADER

The code I used for parsing is as such :

#include <cppast/libclang_parser.hpp>
#include <iostream>

int main()
{
        const cppast::cpp_entity_index index{};
        cppast::libclang_compilation_database compilation_database{"."};
        cppast::simple_file_parser<cppast::libclang_parser> parser{type_safe::ref(index)};
        try
        {
                cppast::parse_database(parser, compilation_database);
        }
        catch (cppast::libclang_error& libclang_error)
        {
                std::cout << "libclang: " << libclang_error.what() << '\n';
        }
}

with the parsed code being :

#include <boost/regex.hpp>

The compile command (from compile_commands.json) is /usr/bin/c++ -std=c++17 -c test.cpp

Output of the test program :

[simple file parser] [info] parsing file '/tmp/test/./test.cpp'
[preprocessor] [error] In file included from /tmp/test/./test.cpp: 1:
[preprocessor] [error] In file included from /usr/include/boost/regex.hpp: 31:
[preprocessor] [error] In file included from /usr/include/boost/regex/v4/regex.hpp: 79:
[preprocessor] [error] In file included from /usr/include/boost/regex/v4/regex_format.hpp: 32:
[preprocessor] [error] In file included from /usr/include/boost/mpl/and.hpp: 42:
[preprocessor] [error] /usr/include/boost/mpl/aux_/include_preprocessed.hpp:37: 'boost/mpl/aux_/preprocessed/gcc/and.hpp /**/' file not found, did you mean 'boost/mpl/aux_/preprocessed/gcc/and.hpp'?
[preprocessor] [error] /usr/include/boost/preprocessor/stringize.hpp:28: note: expanded from macro 'BOOST_PP_STRINGIZE'
[preprocessor] [error] /usr/include/boost/preprocessor/stringize.hpp:32: note: expanded from macro 'BOOST_PP_STRINGIZE_I'
[preprocessor] [error] <scratch space>:122: note: expanded from here
libclang: preprocessor: command '/usr/bin/clang -x c++ -E -dD -CC -Xclang -dI -fno-caret-diagnostics -fno-show-column -fdiagnostics-format=msvc -Wno-macro-redefined -Wno-pragma-once-outside-header -Wno-pragma-system-header-outside-header -Wno-include-next-outside-header "-I/usr/bin/../lib/gcc/x86_64-redhat-linux/10/../../../../include/c++/10" "-I/usr/bin/../lib/gcc/x86_64-redhat-linux/10/../../../../include/c++/10/x86_64-redhat-linux" "-I/usr/bin/../lib/gcc/x86_64-redhat-linux/10/../../../../include/c++/10/backward" "-I/usr/local/include" "-I/usr/lib64/clang/10.0.0/include" "-I/usr/include" "-D__cppast__=libclang" "-D__cppast_version_major__=0" "-D__cppast_version_minor__=0" "-std=c++17" "/tmp/test/./test.cpp"' exited with non-zero exit code (1)

I have attached a minimal CMake project to reproduce this easily here

deadlocklogic commented

Old reply, but I encountered the same situation with boost libs.
Solution:

  1. create a patch file name <absolute path>/patch/boost/mpl/aux_/include_preprocessed.hpp:
  2. Clean the offending preprocessor like this and paste them to the created file:
// NO INCLUDE GUARDS, THE HEADER IS INTENDED FOR MULTIPLE INCLUSION

// Copyright Aleksey Gurtovoy 2000-2006
//
// Distributed under the Boost Software License, Version 1.0. 
// (See accompanying file LICENSE_1_0.txt or copy at 
// http://www.boost.org/LICENSE_1_0.txt)
//
// See http://www.boost.org/libs/mpl for documentation.

// $Id$
// $Date$
// $Revision$

#include <boost/mpl/aux_/config/compiler.hpp>
#include <boost/mpl/aux_/config/preprocessor.hpp>
#include <boost/mpl/aux_/config/workaround.hpp>
#include <boost/preprocessor/cat.hpp>
#include <boost/preprocessor/stringize.hpp>

#if !defined(BOOST_NEEDS_TOKEN_PASTING_OP_FOR_TOKENS_JUXTAPOSING)
#   define AUX778076_PREPROCESSED_HEADER \
    BOOST_MPL_CFG_COMPILER_DIR/BOOST_MPL_PREPROCESSED_HEADER
#else
#   define AUX778076_PREPROCESSED_HEADER \
    BOOST_PP_CAT(BOOST_MPL_CFG_COMPILER_DIR,/)##BOOST_MPL_PREPROCESSED_HEADER
#endif

#if BOOST_WORKAROUND(__IBMCPP__, BOOST_TESTED_AT(700))
#   define AUX778076_INCLUDE_STRING BOOST_PP_STRINGIZE(boost/mpl/aux_/preprocessed/AUX778076_PREPROCESSED_HEADER)
#   include AUX778076_INCLUDE_STRING
#   undef AUX778076_INCLUDE_STRING
#else
#   include BOOST_PP_STRINGIZE(boost/mpl/aux_/preprocessed/AUX778076_PREPROCESSED_HEADER)
#endif

#   undef AUX778076_PREPROCESSED_HEADER

#undef BOOST_MPL_PREPROCESSED_HEADER
  1. parse with a config adding compile_config::add_include_dir("<absolute path>/patch/")
foonathan commented

This is a duplicate of #22. Setting remove_comments_in_macro() should also fix it: https://github.com/foonathan/cppast/blob/e558e2d58f519e3a83af770d460672b1d4ba2886/include/cppast/libclang_parser.hpp#L148

  • 👍1