bazelbuild/bazel

C++ code fails to build with clang 4.0

ulfjack opened this issue · 47 comments

ulfjack commented

On a Ubuntu 17.04 machine, with clang-4.0 installed, I get this error from Bazel for some straightforward C++ code:

this rule is missing dependency declarations for the following files included by 'C-Source/something/other.c':
  '/usr/lib/llvm-4.0/lib/clang/4.0.0/include/stdbool.h'
  '/usr/lib/llvm-4.0/lib/clang/4.0.0/include/stdint.h'
  '/usr/lib/llvm-4.0/lib/clang/4.0.0/include/stddef.h'
ulfjack commented

Failing command line:

$ CC=clang-4.0 bazel run -c opt --copt=-O2 :main

It works with clang-3.9:

$ CC=clang-3.9 bazel run -c opt --copt=-O2 :main
ulfjack commented

The -c and --copt parameters are not relevant. Same behavior without them.

benjaminp commented

I think this is due to a Debian/Ubuntu patch that results in the clang installation system include directory being realpathed in the output of -v but not -MD.

davido commented

FYI: Bazel cannot be built on GCC 7 too: #3903.

nicolasnoble commented

Is there any bug entered in the Debian bug system about that if we believe this is on them?

ulfjack commented

Good point. I don't think we have.

nicolasnoble commented

Can I provide help in any way? This is currently a fairly big issue for us grpc folks. The link to the Debian patch provided by @benjaminp seems to be a 404 for me. If you an point me at the exact source of the problem in terms of actual patches links, I can start bugging Debian maintainers about it.

ulfjack commented

Note that I haven't double-checked @benjaminp's statement, but he's usually right about these things.

hlopko commented

Hmm cannot repro.

ulfjack commented

I still get errors with that patch

this rule is missing dependency declarations for the following files included by 'third_party/mersenne/random.c':
  '/usr/lib/clang/4.0.1/include/stdint.h'
ulfjack commented

Here's a snippet from the CROSSTOOL file:

  cxx_builtin_include_directory: "/usr/include/c++/7.2.0"
  cxx_builtin_include_directory: "/usr/include/x86_64-linux-gnu/c++/7.2.0"
  cxx_builtin_include_directory: "/usr/include/c++/7.2.0/backward"
  cxx_builtin_include_directory: "/usr/include/clang/4.0.1/include"
  cxx_builtin_include_directory: "/usr/local/include"
  cxx_builtin_include_directory: "/usr/include/x86_64-linux-gnu"
  cxx_builtin_include_directory: "/usr/include"
ulfjack commented 
$ ls -l /usr/include/clang/4.0.1/include
lrwxrwxrwx 1 root root 45 Okt  1 15:24 /usr/include/clang/4.0.1/include -> ../../../lib/llvm-4.0/lib/clang/4.0.1/include
ulfjack commented 
$ clang-4.0 -E -xc++ - -v
...
 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0
 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0
 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward
 /usr/include/clang/4.0.1/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
...

When I add -no-canonical-prefixes, I get an error.

clang-4.0: error: unable to execute command: Executable "clang-4.0" doesn't exist!

It looks like clang is broken.

ulfjack commented

Ok, the problem isn't that we're not specifying -no-canonical-prefixes. The problem is that the include paths for C and C++ are significantly different.

For C:

 /usr/local/include
 /usr/lib/llvm-4.0/bin/../lib/clang/4.0.1/include
 /usr/include/x86_64-linux-gnu
 /usr/include

For C++:

 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0
 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/x86_64-linux-gnu/c++/7.2.0
 /usr/bin/../lib/gcc/x86_64-linux-gnu/7.2.0/../../../../include/c++/7.2.0/backward
 /usr/include/clang/4.0.1/include
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include

If I rename the file to .cc, then it works. We'll need to add c_builtin_include_directory, populate that, and then use the correct path in Bazel.

Previously this worked because the .c include path was a subset of the .cc include path. It still is, except that we can't tell because /usr/include/clang/4.0.1/include is a symlink to /usr/lib/llvm-4.0/lib/clang/4.0.1/include. As an alternative to tracking both C and C++ include paths, we could do a readlink -f on each entry and also add those to the list.

Option 2 seems a bit more hacky and might break again if we ever have the case where the .cc path is not a superset of the .c path. Option 1 seems more principled. We might also need to do this for objectivec.

Also, we might want to add a mechanism that allows a user to override the list of include paths on the command line, so that we at least have a workaround next time we run into this problem.

jmillikin-stripe commented

See also envoyproxy/envoy#2631 -- include paths are the same for C and C++, but different if -xc++ is set.

$ echo '' | clang-5.0 -E - -v 2>&1 | grep -A 5 'search starts here'
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/llvm-5.0/lib/clang/5.0.1/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

$ echo '' | clang++-5.0 -E - -v 2>&1 | grep -A 5 'search starts here'
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/include
 /usr/lib/llvm-5.0/lib/clang/5.0.1/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

$ echo '' | clang-5.0 -E -xc++ - -v 2>&1 | grep -A 5 'search starts here'
#include "..." search starts here:
#include <...> search starts here:
 /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0
 /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/x86_64-linux-gnu/c++/5.4.0
 /usr/bin/../lib/gcc/x86_64-linux-gnu/5.4.0/../../../../include/c++/5.4.0/backward
 /usr/include/clang/5.0.1/include
 /usr/local/include
ulfjack commented

Alternatively, we could add an option to disable the include check - if we're running the action in sandbox, it should be safe.

hlopko commented

Thanks all for investigating!

  • Having c_builtin_include_paths SGTM
  • Bazel option for controlling builtin include paths paths SGTM
  • Option to disable .d check SGTM

And I'll throw the need for layering check in bazel in here as a bonus.

kamalmarhubi commented

Is there a near-term workaround?

hlopko commented

Modulo writing your own crosstool, there is not. We could apply a fix and run both CC -E -xc++ - -v and CC -E -xc - -v when detecting C++ toolchain, if this is pressing the world too much, but I'd rather do the principled solution, even though it's not high on our priority list. I'd probably accept a cl with the temporary fix :)

ulfjack commented

Erm, why did you set this to P3? C++ compilation not working at all with no workaround seems rather like a P0 to me.

hlopko commented

@ulfjack Write your own crosstool has been the answer for similar issues since I can remember, therefore I considered is as an issue with workaround, and by P2-P3 I expressed how likely I'm to work on it in near future. But if you think this is more important than other work I'm doing, I'll gladly consider it as P0, it's not hard to fix.

lberki commented

This should really be fixed soon because it's ugly, but is anyone blocked by it at the moment? My understanding is that it's not, so it's not a P0. @jmillikin-stripe -- is this something you currently paper over with your wrapper script? If so, that makes it more pressing.

jmillikin-stripe commented

Yes, the wrapper script works around this this bug in two ways:

  • It backports https://bazel-review.googlesource.com/c/bazel/+/39951 by checking for "clang" in argv[0] and argv[1:] == ["-E", "-xc++", "-", "-v"], then injecting the -no-canonical-prefixes flag.
  • It runs clang++ instead of clang -xc++, so that the include paths are consistent between C and C++.
lberki commented

@jmillikin-stripe : I hear "P2" then -- that wrapper script should go away very soon, but there is no other urgency.

davejrichardson commented

This affects my team's project as well.

g-easy commented

This is still happening with bazel 0.12.0, e.g.:

cc_library(
    name = "main",
    srcs = ["main.c"],
)

Where main.c is just #include <stdint.h>

I get:

$ env CC=clang-4.0 bazel build :main
[...]
this rule is missing dependency declarations for the following files included by 'main.c':
  '/usr/lib/clang/4.0.1/include/stdint.h'
Target //:main failed to build
nicolasnoble commented
ulfjack commented

Marking P1 based on feedback from grpc team. This is a really bad user experience for anyone on standard Ubuntu, which we claim to support.

hlopko commented

c122e47 fixed the issue for me, could you verify that it also fixes you cases? You'll have to build bazel@HEAD for this.

ulfjack commented

Works for me.

g-easy commented

#3977 (comment) is still happening with bazel 0.13.0, but not if it's a *.cc file. Maybe it's specific to C now? Please fix this for C sources also, e.g. grpc depends on zlib. :)

ulfjack commented

C and C++ is treated identically, so it's surprising if it works for one but not the other. Can you give us more context? What platform are you on? You may need to regenerate the C++ config, e.g., by setting CC=gcc.

g-easy commented

What platform are you on?

Linux.

You may need to regenerate the C++ config, e.g., by setting CC=gcc.

$ bazel clean --expunge
$ env CC=gcc bazel build :main
[succeeds]
$ env CC=clang bazel build :main
[...]
this rule is missing dependency declarations for the following files included by 'main.c':
  '/usr/lib/clang/4.0.1/include/stdint.h'
Target //:main failed to build
ulfjack commented

Unable to reproduce. What does your main.c file look like?

tommadams commented

Discovered this issue because I'm unable to build grpc with clang & bazel. I can repro g-easy's build errors on gLinux. This is blocking development of tensorflow/minigo.

Here's how I repro'd g-easy's example:

$ which clang
/usr/bin/clang

$ clang --version
clang version 4.0.1-10 (tags/RELEASE_401/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ which bazel
/usr/local/bin/bazel

$ bazel version
Build label: 0.13.0
Build target: bazel-out/k8-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Mon Oct 18 21:33:40 +50297 (1525078013620)
Build timestamp: 1525078013620
Build timestamp as int: 1525078013620

$ cat cc/bazel.c
#include <stdlib.h>
int main() { return 0; }

$ cat cc/BUILD
cc_binary(
    name = "bazel",
    srcs = ["bazel.c"],
)

# builds when not setting CC
$ bazel build cc:bazel
[succeeds]

# builds with CC=gcc
$ CC=gcc bazel build cc:bazel
[succeeds]

# does not build with CC=clang
$ CC=clang bazel build cc:bazel
INFO: Analysed target //cc:bazel (1 packages loaded).
INFO: Found 1 target...
ERROR: cc/BUILD:304:1: undeclared inclusion(s) in rule '//cc:bazel':
this rule is missing dependency declarations for the following files included by 'cc/bazel.c':
  '/usr/lib/clang/4.0.1/include/stddef.h'
Target //cc:bazel failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.434s, Critical Path: 0.13s
INFO: 0 processes.
FAILED: Build did NOT complete successfully

For completeness, here are the errors I get trying to build grpc's examples:greeter_client:

$ bazel run examples:greeter_client
WARNING: /usr/local/google/home/tmadams/grpc/BUILD:1973:1: in srcs attribute of cc_library rule //:grpc_nanopb: please do not import '//third_party/nanopb:pb_common.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /usr/local/google/home/tmadams/grpc/bazel/grpc_build_system.bzl:172:12
WARNING: /usr/local/google/home/tmadams/grpc/BUILD:1973:1: in srcs attribute of cc_library rule //:grpc_nanopb: please do not import '//third_party/nanopb:pb_decode.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /usr/local/google/home/tmadams/grpc/bazel/grpc_build_system.bzl:172:12
WARNING: /usr/local/google/home/tmadams/grpc/BUILD:1973:1: in srcs attribute of cc_library rule //:grpc_nanopb: please do not import '//third_party/nanopb:pb_encode.c' directly. You should either move the file to this package or depend on an appropriate rule there. Since this rule was created by the macro 'grpc_generate_one_off_targets', the error might have been caused by the macro implementation in /usr/local/google/home/tmadams/grpc/bazel/grpc_build_system.bzl:172:12
INFO: Analysed target //examples:greeter_client (16 packages loaded).
INFO: Found 1 target...
ERROR: /usr/local/google/home/tmadams/.cache/bazel/_bazel_tmadams/4c83ceb29628e8749e4b91b77f6b4cd3/external/com_github_madler_zlib/BUILD.bazel:1:1: undeclared inclusion(s) in rule '@com_github_madler_zlib//:z':
this rule is missing dependency declarations for the following files included by 'external/com_github_madler_zlib/adler32.c':
  '/usr/lib/clang/4.0.1/include/stddef.h'
  '/usr/lib/clang/4.0.1/include/__stddef_max_align_t.h'
  '/usr/lib/clang/4.0.1/include/limits.h'
  '/usr/lib/clang/4.0.1/include/stdarg.h'
Target //examples:greeter_client failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 12.545s, Critical Path: 0.30s
INFO: 3 processes: 2 linux-sandbox, 1 local.
FAILED: Build did NOT complete successfully
ERROR: Build failed. Not running target
g-easy commented

What does your main.c file look like?

A single line:

#include <stdint.h>
ulfjack commented

The fix unfortunately didn't make it into 0.13.0, but will be in 0.14.0 (you can see this in the GitHub UI when you click on the commit id c122e47 - it shows that it's in master, but it doesn't have any release tags). I tried @g-easy 's example with Bazel from HEAD, where it works for me.

tommadams commented

Ah, ok. Thanks!

soonho-tri commented

@ulfjack , is there a chance that c122e47 is included in the upcoming 0.13.1?

ulfjack commented

I believe we only cherrypick commits for regressions, not for bug fixes.

ulfjack commented

(That's a no.)

  • 👍1
nicolasnoble commented

When is 0.14.0 scheduled then ?

g-easy commented

I can confirm my problem goes away with https://releases.bazel.build/0.14.0/rc2/index.html
Thanks!

lberki commented

0.14 is slated for early next week, according to @lfpino , the current release manager.

nicolasnoble commented

I believe that since 0.14 has been released, this issue can be closed now, right ?