wspr-ncsu/visiblev8

Build errors

Closed this issue · 8 comments

lionelhorn commented

Hello and thanks for this very interesting project 👍

I tried to build it with the following system:
Ubuntu 20.04
Python 3.8.5
Docker version 20.10.1, build 831ebea

I followed the Readme

root@re-UX330UAK:/home/re/custom-chromium/source/src/v8# $VV8/builder/tool.py -d $WD build @std
INFO:root:Using build image 'chrome-builder:5afa96dadfe803e8a058d6ede0c9c3987405b8d8' to build '5afa96dadfe803e8a058d6ede0c9c3987405b8d8' inside /home/re/custom-chromium/source
INFO:root:Using '/home/re/custom-chromium/source/src/out/Builder' as build/output directory
INFO:root:Configuration placed in /home/re/custom-chromium/source/src/out/Builder/args.gn
INFO:root:Build targets: ['chrome', 'chrome/installer/linux:stable_deb', 'v8_shell', 'v8/test/unittests']
usermod: no changes
Configuring build in '/work/src/out/Builder'...

And got the error message below.

ERROR at //third_party/dawn/generator/dawn_generator.gni:95:3: Unable to create directory.
  write_file(expected_outputs_file, invoker.outputs)
  ^---------
I was using "/work/src/out/Builder/gen/third_party/dawn".
See //third_party/dawn/BUILD.gn:36:1: whence it was called.
dawn_generator("libdawn_native_utils_gen") {
^-------------------------------------------
See //BUILD.gn:145:7: which caused the file to be included.
      "//third_party/dawn:dawn_end2end_tests_temp_group",
      ^-------------------------------------------------
Traceback (most recent call last):
  File "/home/re/custom-chromium/visiblev8/builder/tool.py", line 461, in <module>
    main(sys.argv)
  File "/home/re/custom-chromium/visiblev8/builder/tool.py", line 455, in main
    args.handler(args)
  File "/home/re/custom-chromium/visiblev8/builder/tool.py", line 312, in do_build
    docker_run_builder(cname, builder_image, 
  File "/home/re/custom-chromium/visiblev8/builder/tool.py", line 125, in docker_run_builder
    subprocess.check_call(docker_args)
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--name', 'builder-tool-build-186130', '-v', '/home/re/custom-chromium/source:/work', '-e', 'WORKSPACE=/work', '-v', '/home/re/custom-chromium/visiblev8/builder/resources/build:/setup:ro', '-e', 'SETUP=/setup', '-w', '/setup', 'chrome-builder:5afa96dadfe803e8a058d6ede0c9c3987405b8d8', './entry.sh', '/work/src/out/Builder', 'chrome', 'chrome/installer/linux:stable_deb', 'v8_shell', 'v8/test/unittests']' returned non-zero exit status 1.
kapravel commented

Hey @lionelhorn,

are you sure you're running this on the correct version of V8? dawn is not supposed to be in the third_party directory for the V8 commit hash that you're using: https://chromium.googlesource.com/chromium/src/+/5afa96dadfe803e8a058d6ede0c9c3987405b8d8/third_party/

You need to checkout the correct version of V8:
$VV8/builder/tool.py -d $WD checkout 5afa96dadfe803e8a058d6ede0c9c3987405b8d8
If you want a more recent version take a look into the patches directory.

I hope this helps!

lionelhorn commented

Thanks @kapravel for the reply and tip.

My first try was with a more recent version. Using the faf34a95a93ff46eb36000dab45cf66e58048b0f commit and patches.
Got a different error related.

It's only after failing to build that one that I tried with the one mentionned in the readme.

Will keep you updated.

lionelhorn commented

Ok so I used the following

#/bin/bash
clear

docker --version
python3 --version

WR=/home/re/custom-chromium/
WD=$WR/chromium/
VV8=$WR/visiblev8/
COMMIT_HASH=5afa96dadfe803e8a058d6ede0c9c3987405b8d8

mkdir $WD
mkdir $VV8

cd $WR
git clone https://github.com/wspr-ncsu/visiblev8

sudo docker rmi $(docker images -q)
sudo docker rm $(docker ps -aq)
sudo docker system prune

sudo $VV8/builder/tool.py -d $WD checkout $COMMIT_HASH
cd $WD/src/V8
patch -p1 <$VV8/patches/5afa96dadfe803e8a058/trace-apis.diff
sudo $VV8/builder/tool.py -d $WD build @std

And got the following error for the build step

re@re:~/custom-chromium/chromium/src/v8$ sudo $VV8/builder/tool.py -d $WD build @std
[sudo] password for re: 
INFO:root:Using build image 'chrome-builder:5afa96dadfe803e8a058d6ede0c9c3987405b8d8' to build '5afa96dadfe803e8a058d6ede0c9c3987405b8d8' inside /home/re/custom-chromium/chromium
INFO:root:Using '/home/re/custom-chromium/chromium/src/out/Builder' as build/output directory
INFO:root:Configuration placed in /home/re/custom-chromium/chromium/src/out/Builder/args.gn
INFO:root:Build targets: ['chrome', 'chrome/installer/linux:stable_deb', 'v8_shell', 'v8/test/unittests']
usermod: no changes
Configuring build in '/work/src/out/Builder'...
ERROR at //tools/grit/grit_rule.gni:312:3: Unable to create directory.
  write_file(asserted_list_file,
  ^---------
I was using "/work/src/out/Builder/obj/net".
See //net/BUILD.gn:2337:1: whence it was called.
grit("net_resources") {
^----------------------
See //BUILD.gn:71:5: which caused the file to be included.
    "//net:net_unittests",
    ^--------------------
Traceback (most recent call last):
  File "/home/re/custom-chromium//visiblev8//builder/tool.py", line 461, in <module>
    main(sys.argv)
  File "/home/re/custom-chromium//visiblev8//builder/tool.py", line 455, in main
    args.handler(args)
  File "/home/re/custom-chromium//visiblev8//builder/tool.py", line 312, in do_build
    docker_run_builder(cname, builder_image, 
  File "/home/re/custom-chromium//visiblev8//builder/tool.py", line 125, in docker_run_builder
    subprocess.check_call(docker_args)
  File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'run', '--name', 'builder-tool-build-127309', '-v', '/home/re/custom-chromium/chromium:/work', '-e', 'WORKSPACE=/work', '-v', '/home/re/custom-chromium/visiblev8/builder/resources/build:/setup:ro', '-e', 'SETUP=/setup', '-w', '/setup', 'chrome-builder:5afa96dadfe803e8a058d6ede0c9c3987405b8d8', './entry.sh', '/work/src/out/Builder', 'chrome', 'chrome/installer/linux:stable_deb', 'v8_shell', 'v8/test/unittests']' returned non-zero exit status 1.
kapravel commented

My next guess would be filesystem-related errors. Do you have enough disk space (~50GB) and are the user permissions correct? The build fails at a point that is not affected by VisibleV8 patches and on an assertion that verifies that some directory was created.

See the code where it fails here:
https://chromium.googlesource.com/chromium/src/+/5afa96dadfe803e8a058d6ede0c9c3987405b8d8/tools/grit/grit_rule.gni#312

jueckstock commented

Don't run the builder tool using sudo. That will
result in parts of the workspace being owned by root and will prevent the processes running inside the builder container from making directories and files inside the workspace (we do the Right Thing™️ and run the build inside the container as a non root user).

lionelhorn commented

I got enough disk space so I think you may be right about it being a permissions issue.

Considering I had

Got permission denied while trying to connect to the Docker daemon socket

I too quickly resorted to sudoing the builder script.

Thanks again for the help.
I'll install docker using the rootless https://docs.docker.com/engine/security/rootless/ and try again.

jueckstock commented

You should be able to just add your user to the docker group to be able to run it without sudo...