depot_tools executes the . /cipd script and cannot find the cipd_client_version.digests file.
chenggwang opened this issue · 14 comments
Describe your problem:
Subprocess failed with return code 1.
./cipd: line 137: ./depot_tools/cipd_client_version.digests: No such file or directory
Platform linux-amd64 is not supported by the CIPD client bootstrap: there's no pinned SHA256 hash for it in the *.digests file.
The cipd file is in the same directory as cipd_client_version.digests, so the file path should not be ". /depot_tools/cipd_client_version.digests", but should be ". /cipd_client_version.digests".
Operating system is ubuntu 20.04.6 I looked at the depot_tools directory and cipd and cipd_client_version.digests are in the same directory and the error thrown is . /cipd: line 137: . /depot_tools/cipd_client_version.digests: No such file or directory.Here is the cipd script source code line 137：function expected_sha256() { local line while read -r line; do if [[ "${line}" =~ ^([0-9a-z\-]+)[[:blank:]]+sha256[[:blank:]]+([0-9a-f]+)$ ]] ; then local plat="${BASH_REMATCH[1]}" local hash="${BASH_REMATCH[2]}" if [ "${plat}" == "$1" ]; then echo "${hash}" return 0 fi fi done < ""${VERSION_FILE}.digests" VERSION_FILE="${MYPATH}/cipd_client_version" Is there a problem with the MYPATH of depot_tools, or is our build script affecting the MYPATH of depot_tools?
I have managed to reproduce the bug described here in v8.0.1.31 tag in Ubuntu 16.04 (actually it's inside of the Docker).
Here there is how I built this as a root user:
cd /root
git clone \
--depth=1 \
--recursive \
--branch v8.0.1.31 \
https://github.com/ONLYOFFICE/build_tools.git \
build_tools
# Ignore detached head warning
cd build_tools
mkdir out
docker build --tag onlyoffice-document-editors-builder .
docker run -e PRODUCT_VERSION=8.0.1 -e BUILD_NUMBER=31 -e NODE_ENV='production' -v $(pwd)/out:/build_tools/out onlyoffice-document-editors-builder /bin/bash -c 'cd tools/linux && python3 ./automate.py --branch=tags/v8.0.1.31'
Error excerpt 1:
Cloning into 'depot_tools'...
WARNING: Your metrics.cfg file was invalid or nonexistent. A new one will be created.
error: unknown option `type'
usage: git config [<options>]
Config file location
--global use global config file
Error excerpt 2:
[0:03:44] Still working on: [33/97738]
[0:03:44] v8
________ running 'python3 third_party/depot_tools/update_depot_tools_toggle.py --disable' in '/core/Common/3dParty/v8_89/v8'
________ running 'python3 build/landmines.py --landmine-scripts tools/get_landmines.py' in '/core/Common/3dParty/v8_89/v8'
________ running 'python3 third_party/depot_tools/download_from_google_storage.py --no_resume --no_auth --bucket chromium-clang-format -s buildto
ols/linux64/clang-format.sha1' in '/core/Common/3dParty/v8_89/v8'
File "third_party/depot_tools/download_from_google_storage.py", line 51
return f'.{gcs_file_name}{MIGRATION_TOGGLE_FILE_SUFFIX}'
^
SyntaxError: invalid syntax
Error: Command 'python3 third_party/depot_tools/download_from_google_storage.py --no_resume --no_auth --bucket chromium-clang-format -s buildtool
s/linux64/clang-format.sha1' returned non-zero exit status 1 in /core/Common/3dParty/v8_89/v8
File "third_party/depot_tools/download_from_google_storage.py", line 51
return f'.{gcs_file_name}{MIGRATION_TOGGLE_FILE_SUFFIX}'
^
SyntaxError: invalid syntax
Running: gclient root
Running: gclient config --spec 'solutions = [
{
"name": "v8",
"url": "https://chromium.googlesource.com/v8/v8.git",
"deps_file": "DEPS",
"managed": False,
"custom_deps": {},
},
]
'
Running: gclient sync --with_branch_heads
Subprocess failed with return code 2.
./cipd: line 137: ./depot_tools/cipd_client_version.digests: No such file or directory
Platform linux-amd64 is not supported by the CIPD client bootstrap: there's no pinned SHA256 hash for it in the *.digests file.
error: unknown option `type'
usage: git config [<options>]
Config file location
Error excerpt 3:
[fetch & build]: icu
[fetch & build]: openssl
delete warning [file not exist]: ./openssl.data
gn gen out.gn/linux_64 --args="v8_static_library=true is_component_build=false v8_monolithic=true v8_use_external_startup_data=false use_custom_libcxx=false treat_warnings_as_errors=false target_cpu=\"x64\" v8_target_cpu=\"x64\" is_debug=false is_clang=true use_sysroot=false"
Error (ninja): 1
install dependencies...
Node.js version cannot be less 14
Reinstall
install qt...
---------------------------------------------
build branch: tags/v8.0.1.31
---------------------------------------------
---------------------------------------------
build modules: desktop builder server
---------------------------------------------
Error (./make.py): 1
My suspicions about what this bug might be about are centered around:
File "third_party/depot_tools/download_from_google_storage.py", line 51
return f'.{gcs_file_name}{MIGRATION_TOGGLE_FILE_SUFFIX}'
^
SyntaxError: invalid syntax
which might be caused by a python version changed made on 02426e4 commit.
This is only a suspicion and, well, I'm not very confident on it. I am just throwing it here just in case it helps.
v8.0.0.99 attempt to build seems to have the same problem.
cd /root
git clone \
--depth=1 \
--recursive \
--branch v8.0.0.99 \
https://github.com/ONLYOFFICE/build_tools.git \
build_tools
# Ignore detached head warning
cd build_tools
mkdir out
docker build --tag onlyoffice-document-editors-builder .
docker run -e PRODUCT_VERSION=8.0.0 -e BUILD_NUMBER=99 -e NODE_ENV='production' -v $(pwd)/out:/build_tools/out onlyoffice-document-editors-builder /bin/bash -c 'cd tools/linux && python3 ./automate.py --branch=tags/v8.0.0.99'
Exceprt 1:
WARNING: 'v8/third_party/abseil-cpp' is no longer part of this client.
It is recommended that you manually remove it or use 'gclient sync -D' next time.
________ running 'python third_party/depot_tools/update_depot_tools_toggle.py --disable' in '/core/Common/3dParty/v8_89/v8'
________ running 'python build/landmines.py --landmine-scripts tools/get_landmines.py' in '/core/Common/3dParty/v8_89/v8'
________ running 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s buildtools/linux64/clang
-format.sha1' in '/core/Common/3dParty/v8_89/v8'
File "/build_tools/scripts/../../core/Common/3dParty/v8_89/depot_tools/download_from_google_storage.py", line 51
return f'.{gcs_file_name}{MIGRATION_TOGGLE_FILE_SUFFIX}'
^
SyntaxError: invalid syntax
Error: Command 'download_from_google_storage --no_resume --platform=linux* --no_auth --bucket chromium-clang-format -s buildtools/linux64/clang-f
ormat.sha1' returned non-zero exit status 1 in /core/Common/3dParty/v8_89/v8
File "/build_tools/scripts/../../core/Common/3dParty/v8_89/depot_tools/download_from_google_storage.py", line 51
return f'.{gcs_file_name}{MIGRATION_TOGGLE_FILE_SUFFIX}'
^
SyntaxError: invalid syntax
Done. Made 165 targets from 92 files in 301ms
Exceprt 2:
[1/2933] CXX obj/cppgc_base/compaction-worklists.o
FAILED: obj/cppgc_base/compaction-worklists.o
../../third_party/llvm-build/Release+Asserts/bin/clang++ -MMD -MF obj/cppgc_base/compaction-worklists.o.d -DUSE_UDEV -DUSE_AURA=1 -DUSE_GLIB=1 -D
USE_NSS_CERTS=1 -DUSE_OZONE=1 -DUSE_X11=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -DCR_CLANG_REVISION=\"ll
vmorg-12-init-16296-g5e476061-1\" -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D_FORTIFY_SOURCE=2 -DNDEBUG -DNVALGRIND -DDYNAMIC_ANNOTATIONS_
ENABLED=0 -DV8_TYPED_ARRAY_MAX_SIZE_IN_HEAP=64 -DENABLE_GDB_JIT_INTERFACE -DENABLE_MINOR_MC -DV8_INTL_SUPPORT -DV8_ATOMIC_OBJECT_FIELD_WRITES -DV
8_ATOMIC_MARKING_STATE -DV8_ENABLE_LAZY_SOURCE_POSITIONS -DV8_WIN64_UNWINDING_INFO -DV8_ENABLE_REGEXP_INTERPRETER_THREADED_DISPATCH -DV8_SNAPSHOT
_COMPRESSION -DV8_COMPRESS_POINTERS -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_DEPRECATION_WARNINGS -DV8_IMMINENT_DEPRECATION_WARNINGS -DV8_NO_ARGUMENTS_
ADAPTOR -DCPPGC_CAGED_HEAP -DV8_TARGET_ARCH_X64 -DV8_HAVE_TARGET_OS -DV8_TARGET_OS_LINUX -DDISABLE_UNTRUSTED_CODE_MITIGATIONS -DV8_COMPRESS_POINT
ERS -DV8_31BIT_SMIS_ON_64BIT_ARCH -DV8_DEPRECATION_WARNINGS -DV8_IMMINENT_DEPRECATION_WARNINGS -DV8_NO_ARGUMENTS_ADAPTOR -DCPPGC_CAGED_HEAP -I../
.. -Igen -I../.. -I../../include -Igen -I../../include -fno-delete-null-pointer-checks -fno-strict-aliasing --param=ssp-buffer-size=4 -fstack-pro
tector -funwind-tables -fPIC -pthread -fcolor-diagnostics -fmerge-all-constants -fcrash-diagnostics-dir=../../tools/clang/crashreports -mllvm -in
stcombine-lower-dbg-declare=0 -m64 -march=x86-64 -msse3 -Wno-builtin-macro-redefined -D__DATE__= -D__TIME__= -D__TIMESTAMP__= -Xclang -fdebug-com
pilation-dir -Xclang . -no-canonical-prefixes -Wall -Wextra -Wimplicit-fallthrough -Wunreachable-code -Wthread-safety -Wextra-semi -Wno-missing-f
ield-initializers -Wno-unused-parameter -Wno-c++11-narrowing -Wno-unneeded-internal-declaration -Wno-undefined-var-template -Wno-psabi -Wno-ignor
ed-pragma-optimize -Wno-implicit-int-float-conversion -Wno-final-dtor-non-final-class -Wno-builtin-assume-aligned-alignment -Wno-deprecated-copy -Wno-non-c-typedef-for-linkage -Wmax-tokens -fno-omit-frame-pointer -g0 -Wheader-hygiene -Wstring-conversion -Wtautological-overlap-compare -Wmis
sing-field-initializers -Wunreachable-code -Wshorten-64-to-32 -O3 -fno-ident -fdata-sections -ffunction-sections -fvisibility=default -Wexit-time
-destructors -std=c++14 -fno-trigraphs -Wno-trigraphs -fno-exceptions -fno-rtti -c ../../src/heap/cppgc/compaction-worklists.cc -o obj/cppgc_base
/compaction-worklists.o
/bin/sh: 1: ../../third_party/llvm-build/Release+Asserts/bin/clang++: not found
Exceprt 3:
[fetch & build]: openssl delete warning [file not exist]: ./openssl.data
gn gen out.gn/linux_64 --args="v8_static_library=true is_component_build=false v8_monolithic=true v8_use_external_startup_data=false use_custom_l
ibcxx=false treat_warnings_as_errors=false target_cpu=\"x64\" v8_target_cpu=\"x64\" is_debug=false is_clang=true use_sysroot=false"
Error (ninja): 1
install dependencies...
Node.js version cannot be less 14
Reinstall
install qt...
---------------------------------------------
build branch: tags/v8.0.0.99
---------------------------------------------
---------------------------------------------
build modules: desktop builder server
---------------------------------------------
Error (./make.py): 1
I think that the problem relies on depot_tools needing a minimum of Python 3.8 while Ubuntu 16.04 only has Python 3.5.2.
The error is around an f-string which was introduced to Python in Python 3.6.x.
Apparently this was detected on the windows xp build and it was fixed there thanks to 02426e4 . I'm not sure about the consequences of using such bootstrap python version and what would be the best way of fixing it.
My workaround will be checking out an older v8 version which does not have that commit that introduces the f-string.
Changing distro from ubuntu:16.04 to ubuntu:20.04 in Dockerfile enables you to build this without any of the problems described here.
Changing distro from ubuntu:16.04 to ubuntu:20.04 in Dockerfile enables you to build this without any of the problems described here.
Could you copy the complete script that you managed to build correctly, for tag v8.0.1.31 together with ubuntu 20.04?
Changing distro from ubuntu:16.04 to ubuntu:20.04 in Dockerfile enables you to build this without any of the problems described here.
Could you copy the complete script that you managed to build correctly, for tag v8.0.1.31 together with ubuntu 20.04?
Well, it should be the same thing as: #802 (comment) :
cd /root
git clone \
--depth=1 \
--recursive \
--branch v8.0.1.31 \
https://github.com/ONLYOFFICE/build_tools.git \
build_tools
# Ignore detached head warning
cd build_tools
mkdir out
docker build --tag onlyoffice-document-editors-builder .
docker run -e PRODUCT_VERSION=8.0.1 -e BUILD_NUMBER=31 -e NODE_ENV='production' -v $(pwd)/out:/build_tools/out onlyoffice-document-editors-builder /bin/bash -c 'cd tools/linux && python3 ./automate.py --branch=tags/v8.0.1.31'
Anyways if you want to reproduce exactly what I built you can check: https://github.com/btactic-oo/unlimited-onlyoffice-package-builder .
I am still having this problem
I am still having this problem
Ok I solved by: https://stackoverflow.com/a/77476594
just don't use relative path.
I am still having this problem
Ok I solved by: stackoverflow.com/a/77476594 just don't use relative path.
Great!
Thanks to @adriangibanelbtactic for all your help! We already have build 8.0.1.31 running with ubuntu 2004