google/distbench

Trying to compile with gRPC 1.41.0

obedmr opened this issue · 4 comments

obedmr commented

Hi,
Not sure what's the best method to reach to you folks, but let me go here. I have an specific requirement to build distbench with gRPC 1.41.0. When I try to build it with the bazel build :all command I'm getting the following error:

Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Analyzing: 23 targets (1 packages loaded)
Analyzing: 23 targets (1 packages loaded, 0 targets configured)
Analyzing: 23 targets (34 packages loaded, 36 targets configured)
ERROR: Traceback (most recent call last):
        File "/root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/build_bazel_rules_apple/apple/internal/testing/ios_rules.bzl", line 62, column 61, in <toplevel>
                ios_ui_test_bundle = rule_factory.create_apple_bundling_rule(
        File "/root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/build_bazel_rules_apple/apple/internal/rule_factory.bzl", line 904, column 55, in _create_apple_bundling_rule
                rule_attrs.append(_common_binary_linking_attrs(rule_descriptor))
        File "/root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/build_bazel_rules_apple/apple/internal/rule_factory.bzl", line 218, column 21, in _common_binary_linking_attrs
                apple_common.objc_proto_aspect,
Error: 'apple_common' value has no field or method 'objc_proto_aspect'
INFO: Repository go_sdk instantiated at:
  /distbench/WORKSPACE.bazel:37:16: in <toplevel>
  /root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/com_github_grpc_grpc/bazel/grpc_extra_deps.bzl:50:27: in grpc_extra_deps
  /root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/io_bazel_rules_go/go/private/sdk.bzl:268:28: in go_register_toolchains
  /root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/io_bazel_rules_go/go/private/sdk.bzl:92:21: in go_download_sdk
Repository rule _go_download_sdk defined at:
  /root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/io_bazel_rules_go/go/private/sdk.bzl:79:35: in <toplevel>
ERROR: /distbench/BUILD:191:16: error loading package '@com_github_grpc_grpc//src/compiler': at /root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/com_github_grpc_grpc/bazel/grpc_build_system.bzl:29:6: at /root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/build_bazel_rules_apple/apple/ios.bzl:22:5: initialization of module 'apple/internal/testing/ios_rules.bzl' failed and referenced by '//:_distbench_cc_grpc_proto_grpc_codegen'
ERROR: Analysis of target '//:_distbench_cc_grpc_proto_grpc_codegen' failed; build aborted: Analysis failed
INFO: Elapsed time: 26.284s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (34 packages loaded, 36 targets configured)
FAILED: Build did NOT complete successfully (34 packages loaded, 36 targets configured)

I modified the WORKSPACE.bazel in order to point to my custom gRPC. Below my change:

diff --git a/WORKSPACE.bazel b/WORKSPACE.bazel
index d35ccd8..3d919e5 100644
--- a/WORKSPACE.bazel
+++ b/WORKSPACE.bazel
@@ -16,13 +16,20 @@ rules_proto_toolchains()

 ##### gRPC Rules for Bazel
 http_archive(
-    name = "com_github_grpc_grpc",
+    name = "com_github_grpc_grpc2",
     sha256 = "bba65891b8c8eaea9f10b0ccccb4b6173590aee36937f5b52e8665720bcaa4c1",
     urls = [
         "https://github.com/grpc/grpc/archive/b0f37a22bbae12a4b225a88be6d18d5e41dce881.tar.gz",  # Release 1.43
     ],
     strip_prefix = "grpc-b0f37a22bbae12a4b225a88be6d18d5e41dce881",
 )
+
+new_local_repository(
+    name = "com_github_grpc_grpc",
+    path = "/custom-grpc",
+    build_file = "BUILD",
+)
+
 load("@com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
 grpc_deps()

It seems to me like an Apple specific build, since I'm only interested in a Linux-only build, is there a way to overcome this?

In case that you're interested in how I'm building it, below the Dockerfile I'm playing with:

FROM archlinux

RUN pacman -Syu --noconfirm bazel git gcc which python
RUN git clone https://github.com/google/distbench.git
WORKDIR distbench
# Build
COPY custom-grpc /custom-grpc
COPY workspace.diff /

# Apply patches
RUN git apply /workspace.diff

RUN bazel build :all

Thanks in advance for your help.

oserres commented

Hi,

Going back to 1.41.1, with

 http_archive(
     name = "com_github_grpc_grpc",
-    sha256 = "bba65891b8c8eaea9f10b0ccccb4b6173590aee36937f5b52e8665720bcaa4c1",
+    sha256 = "bd2ec83a5ed255950daa00e5c8a557a97b8ae20e530757c6c07ea4d78c846caa",
     urls = [
-        "https://github.com/grpc/grpc/archive/b0f37a22bbae12a4b225a88be6d18d5e41dce881.tar.gz",  # Release 1.43
+        "https://github.com/grpc/grpc/archive/635693ce624f3b3a89e5a764f0664958ef08b2b9.tar.gz",  # Release 1.41.1
     ],
-    strip_prefix = "grpc-b0f37a22bbae12a4b225a88be6d18d5e41dce881",
+    strip_prefix = "grpc-635693ce624f3b3a89e5a764f0664958ef08b2b9",
 )

Make me get the exact same error as you in my environment.

I am pretty sure that if you updated your custom grpc to a newer version it would solve the problem. (also 1.41.1 was working before for me, so something was updated which broke grpc and then was fixed in a newer version).

I hope this helps.

obedmr commented

Hi @oserres, thanks for the help, I was able to run in 1.41.0 release (sorry I put the wrong version before). One of the issue is that I was putting the custom gRPC dependency as new_local_repository instead of local_repository. I also modified the apple dependency to the one used in 1.43.0. And it works. But, when running the benchmark I was able to only run:

bazel run :distbench -- test_sequencer &
bazel run :distbench -- node_manager --test_sequencer=localhost:10000 --port=9999 &

But, when I run the simple_test.sh it fails on this; I've tried with a newer version of boringssl but still the same.

ERROR: /root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/external/boringssl/BUILD:161:11: Compiling src/crypto/x509v3/v3_utl.c failed: (Exit 1): process-wrapper failed: error executing command
  (cd /root/.cache/bazel/_bazel_root/41bc40a3bd5fcb088ad6274fd8e222db/sandbox/processwrapper-sandbox/15725/execroot/__main__ && \
  exec env - \
    BAZEL_LINKLIBS=-l%:libstdc++.a:-lm \
    BAZEL_LINKOPTS=-static-libgcc \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
    TMPDIR=/tmp \
  /root/.cache/bazel/_bazel_root/install/814b1f4390b05dc026f88e181a6b474d/process-wrapper '--timeout=0' '--kill_delay=15' /usr/sbin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections -MD -MF bazel-out/k8-opt/bin/external/boringssl/_objs/crypto/v3_utl.d '-frandom-seed=bazel-out/k8-opt/bin/external/boringssl/_objs/crypto/v3_utl.o' -iquote external/boringssl -iquote bazel-out/k8-opt/bin/external/boringssl -isystem external/boringssl/src/include -isystem bazel-out/k8-opt/bin/external/boringssl/src/include -Wa,--noexecstack '-D_XOPEN_SOURCE=700' -Wall -Werror '-Wformat=2' -Wsign-compare -Wmissing-field-initializers -Wwrite-strings -Wshadow -fno-common '-std=c11' -Wmissing-prototypes -Wold-style-definition -Wstrict-prototypes -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/boringssl/src/crypto/x509v3/v3_utl.c -o bazel-out/k8-opt/bin/external/boringssl/_objs/crypto/v3_utl.o)
In file included from /usr/include/string.h:535,
                 from external/boringssl/src/crypto/x509v3/v3_utl.c:63:
In function 'memset',
    inlined from 'OPENSSL_memset' at external/boringssl/src/crypto/x509v3/../internal.h:840:10,
    inlined from 'ipv6_from_asc' at external/boringssl/src/crypto/x509v3/v3_utl.c:1328:9:
/usr/include/bits/string_fortified.h:59:10: error: '__builtin_memset' forming offset 16 is out of the bounds [0, 16] [-Werror=array-bounds]
   59 |   return __builtin___memset_chk (__dest, __ch, __len,
      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   60 |                                  __glibc_objsize0 (__dest));
      |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Target //:distbench failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.296s, Critical Path: 0.98s
INFO: 55 processes: 55 internal.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
obedmr commented

Wow, It's solved. There was a compiler version related issue. I was doing it with Archlinux gcc (12.1.0) and it was failing, then I moved to an Ubuntu's based docker image (with gcc 11.2.0) and it worked. Later, it would be worth to see why this new gcc compiler version is complaining.

➜  docker-distbench git:(master) ✗ docker run --rm distbench:ubuntu gcc --version
gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

➜  docker-distbench git:(master) ✗ docker run --rm distbench:archlinux gcc --version
gcc (GCC) 12.1.0
Copyright (C) 2022 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Thanks a lot for the help folks, I'm closing the issue since it's already solved in my side.

oserres commented

Thanks for the update ! Good to know.