Failed to build for `aarch64-linux-android`

Question

Failed to build for `aarch64-linux-android`

secext2022 opened this issue a year ago · 13 comments

secext2022 commented a year ago

Build command:

V8_FROM_SOURCE=1 cargo build -vv --release --target aarch64-linux-android

got error:

[v8 0.74.1] The current directory is /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.74.1
[v8 0.74.1] gn gen --root=/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.74.1 /home/runner/work/v8-src/v8-src/target/aarch64-linux-android/release/gn_out
[v8 0.74.1] android/icudtl.dat
[v8 0.74.1] ERROR at //build/android/BUILD.gn:93:17: Could not read file.
[v8 0.74.1]   pydeps_file = "pylib/results/presentation/test_results_presentation.pydeps"
[v8 0.74.1]                 ^------------------------------------------------------------
[v8 0.74.1] I resolved this to "/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.74.1/build/android/pylib/results/presentation/test_results_presentation.pydeps".
[v8 0.74.1] See //build/android/BUILD.gn:92:1: whence it was called.
[v8 0.74.1] python_library("test_result_presentations_py") {
[v8 0.74.1] ^-----------------------------------------------
[v8 0.74.1] See //v8/tools/BUILD.gn:39:19: which caused the file to be included.
[v8 0.74.1]     data_deps = [ "//build/android:test_runner_py" ]
[v8 0.74.1]                   ^-------------------------------
[v8 0.74.1] thread 'main' panicked at 'assertion failed: Command::new(gn()).arg(format!(\"--root={}\",\n                                                dirs.root.display())).arg(format!(\"--script-executable={}\",\n                                            python())).arg(\"gen\").arg(&gn_out_dir).arg(\"--args=\".to_owned()\n                                +\n                                &args).stdout(Stdio::inherit()).stderr(Stdio::inherit()).envs(env::vars()).status().expect(\"Coud not run `gn`\").success()', /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.74.1/build.rs:717:5
[v8 0.74.1] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The following warnings were emitted during compilation:

warning: Not using sccache or ccache

error: failed to run custom build command for `v8 v0.74.1`

Caused by:
  process didn't exit successfully: `/home/runner/work/v8-src/v8-src/target/release/build/v8-c037bc55a0f012fb/build-script-build` (exit status: 101)

It seems that some files is missing at build/android/pylib

Answer 1 · 2023-07-08T13:09:53.000Z

related issues: #480, #970

Answer 2 · 2023-07-08T16:57:08.000Z

To build it, we have to checkout v8 source and copy missing files.
I use this github workflow file to build:

name: CI

on:
  push:
    branches: [ "main" ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v3

    - run: cargo fetch
    # symlink `v8_src` to rusty_v8
    - run: cargo metadata | node get_v8_path.js

    # FIXME: we have to checkout v8 source, because there is missing files in rusty_v8
    #### start checkout v8 source
    - run: git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git --single-branch --depth=1
    - run: export PATH=$(pwd)/depot_tools:$PATH && echo $PATH && mkdir v8

    # https://v8.dev/docs/source-code
    - run: export PATH=$(pwd)/depot_tools:$PATH && cd v8 && gclient
    - run: export PATH=$(pwd)/depot_tools:$PATH && cd v8 && fetch v8

    # https://v8.dev/docs/cross-compile-arm
    - run: echo "target_os = ['android']" >> v8/.gclient
    - run: cat v8/.gclient

    - run: export PATH=$(pwd)/depot_tools:$PATH && cd v8 && gclient sync
    #### done checkout v8 source

    # copy missing files
    - run: cp -rn v8/v8/build/android v8_src/build || true

    # build
    - run: cd v8_src && rustup target add aarch64-linux-android
    - run: cd v8_src && V8_FROM_SOURCE=1 cargo build -vv --release --target aarch64-linux-android

    - uses: actions/upload-artifact@v3
      with:
        name: librusty_v8_release_aarch64-linux-android.a
        path: v8_src/target/aarch64-linux-android/release/gn_out/obj/librusty_v8.a

Answer 3 · 2023-07-10T02:05:32.000Z

Build command:

V8_FROM_SOURCE=1 cargo build -vv --release --target aarch64-linux-android

got error:

[v8 0.74.1] The current directory is /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.74.1
[v8 0.74.1] gn gen --root=/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.74.1 /home/runner/work/v8-src/v8-src/target/aarch64-linux-android/release/gn_out
[v8 0.74.1] android/icudtl.dat
[v8 0.74.1] ERROR at //build/android/BUILD.gn:93:17: Could not read file.
[v8 0.74.1]   pydeps_file = "pylib/results/presentation/test_results_presentation.pydeps"
[v8 0.74.1]                 ^------------------------------------------------------------
[v8 0.74.1] I resolved this to "/home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.74.1/build/android/pylib/results/presentation/test_results_presentation.pydeps".
[v8 0.74.1] See //build/android/BUILD.gn:92:1: whence it was called.
[v8 0.74.1] python_library("test_result_presentations_py") {
[v8 0.74.1] ^-----------------------------------------------
[v8 0.74.1] See //v8/tools/BUILD.gn:39:19: which caused the file to be included.
[v8 0.74.1]     data_deps = [ "//build/android:test_runner_py" ]
[v8 0.74.1]                   ^-------------------------------
[v8 0.74.1] thread 'main' panicked at 'assertion failed: Command::new(gn()).arg(format!(\"--root={}\",\n                                                dirs.root.display())).arg(format!(\"--script-executable={}\",\n                                            python())).arg(\"gen\").arg(&gn_out_dir).arg(\"--args=\".to_owned()\n                                +\n                                &args).stdout(Stdio::inherit()).stderr(Stdio::inherit()).envs(env::vars()).status().expect(\"Coud not run `gn`\").success()', /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.74.1/build.rs:717:5
[v8 0.74.1] note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The following warnings were emitted during compilation:

warning: Not using sccache or ccache

error: failed to run custom build command for `v8 v0.74.1`

Caused by:
  process didn't exit successfully: `/home/runner/work/v8-src/v8-src/target/release/build/v8-c037bc55a0f012fb/build-script-build` (exit status: 101)

It seems that some files is missing at build/android/pylib

Detail build log: https://github.com/fm-elpac/v8-src/actions/runs/5494146045/jobs/10012760135

Answer 4 · 2023-07-10T04:33:16.000Z

@afalon44
Sorry, I don't understand what you say.

Answer 5 · 2024-03-06T18:43:52.000Z

我尝试编译 v0.83.2 成功了，但是我在使用时遇到了一些问题，使用 deno_core 运行例子时会报空指针错误，请问你成功运行了吗

Answer 6 · 2024-03-06T18:54:19.000Z

我尝试编译 v0.83.2 成功了，但是我在使用时遇到了一些问题，使用 deno_core 运行例子时会报空指针错误，请问你成功运行了吗

以前的版本是可以的, 最新的版本有点问题, 还没成功.

Answer 7 · 2024-03-06T19:09:08.000Z

我尝试编译 v0.83.2 成功了，但是我在使用时遇到了一些问题，使用 deno_core 运行例子时会报空指针错误，请问你成功运行了吗

以前的版本是可以的, 最新的版本有点问题, 还没成功.

我还以为是我编译v8出现问题了呢，我现在在尝试重新编译了🤦‍♂️

Answer 8 · 2024-03-06T19:13:17.000Z

我尝试编译 v0.83.2 成功了，但是我在使用时遇到了一些问题，使用 deno_core 运行例子时会报空指针错误，请问你成功运行了吗

以前的版本是可以的, 最新的版本有点问题, 还没成功.

以前的版本确实是可以的我也一直在用，我之前用的就是 v0.64.0，只是最近我想升级就挑了 v0.83.2，没想到就遇到一些麻烦了

Answer 9 · 2024-03-06T19:45:20.000Z

我刚测试分别运行 v8 和 deno_core 的 hello_world 例子
结果是 v8 的例子可以运行但 deno_core 的例子会崩溃，这是不是说明问题就出在 deno_core，似乎是在 JsRuntime 初始化过程中崩溃的

v8 运行结果

    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
./src/: 1 file pushed, 0 skipped. 0.5 MB/s (2615 bytes in 0.005s)
./target/aarch64-linux-android/debug/test_build_v8: 1 file pushed, 0 skipped. 75.1 MB/s (47282496 bytes in 0.600s)
Hello World!
3 + 4 = 7

deno_core 主机运行结果(预期结果)

    Finished dev [unoptimized + debuginfo] target(s) in 0.07s
     Running `target/debug/test_build_deno_core`
The sum of
1,2,3
is
6
Exception:
TypeError: serde_v8 error: invalid type; expected: array, got: Number

deno_core 运行结果(实际结果)

    Finished dev [unoptimized + debuginfo] target(s) in 0.03s
./src/: 1 file pushed, 0 skipped. 0.5 MB/s (1582 bytes in 0.003s)
./target/aarch64-linux-android/debug/test_build_deno_core:...e pushed, 0 skipped. 96.0 MB/s (105099880 bytes in 1.045s)

crash log

********** Crash dump: **********                                                                                                                  [26/1976]Build fingerprint: 'Xiaomi/cepheus/cepheus:13/TQ3A.230901.001/1700048355:user/release-keys'
#00 0x00000000014d84e2 /data/local/tmp/deno_snapshot_builder/test_build_deno_core (offset 0x2586000)
                                                                                   v8__Isolate__PerformMicrotaskCheckpoint
                                                                                   ./../../../../src/binding.cc:222:12
#01 0x000000000151b780 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
v8::String::NewExternalOneByte(v8::Isolate*, v8::String::ExternalOneByteStringResource*)
./../../../../v8/src/api/api.cc:7578:17
#02 0x0000000001306c7c /data/local/tmp/deno_snapshot_builder/test_build_deno_core
v8::string::_$LT$impl$u20$v8..data..String$GT$::new_from_onebyte_const::_$u7b$$u7b$closure$u7d$$u7d$::hc504783f0c9d33b9
/home/autsing/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.83.2/src/string.rs:461:9
#03 0x0000000001157120 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
v8::scope::HandleScope$LT$$LP$$RP$$GT$::cast_local::h1c5e242b54bec948
/home/autsing/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.83.2/src/scope.rs:239:21
v8::string::_$LT$impl$u20$v8..data..String$GT$::new_from_onebyte_const::h92a7fb1d6c9d0d8f
/home/autsing/.cargo/registry/src/index.crates.io-6f17d22bba15001f/v8-0.83.2/src/string.rs:460:7
deno_core::runtime::bindings::v8_static_strings::new::h1a380defbc8a9ceb
/home/autsing/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_core-0.262.0/runtime/bindings.rs:139:5
#04 0x00000000010eb448 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
deno_core::runtime::bindings::initialize_deno_core_namespace::hd0800f7cd14c4254
/home/autsing/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_core-0.262.0/runtime/bindings.rs:199:18
#05 0x00000000011327e8 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
deno_core::runtime::jsruntime::JsRuntime::new_inner::h83c4a100d15d3fd3
/home/autsing/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_core-0.262.0/runtime/jsruntime.rs:754:7
#06 0x0000000001130b98 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
deno_core::runtime::jsruntime::JsRuntime::new::h76d74b1ab1866c2a
/home/autsing/.cargo/registry/src/index.crates.io-6f17d22bba15001f/deno_core-0.262.0/runtime/jsruntime.rs:568:11
#07 0x00000000010b6ad8 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
test_build_deno_core::main::he56560a423cd102f
/home/autsing/Gits/test_build_deno_core/src/main.rs:26:23
#08 0x00000000010bd024 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
core::ops::function::FnOnce::call_once::h6e4d1f2aabaaef94
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:250:5
#09 0x00000000010b63dc /data/local/tmp/deno_snapshot_builder/test_build_deno_core
std::sys_common::backtrace::__rust_begin_short_backtrace::h98fe2dc7d443d20b
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:155:18
#10 0x00000000010b6480 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::hf8933a5d837627f6
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:166:18
#11 0x000000000255b9f0 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
core::ops::function::impls::_$LT$impl$u20$core..ops..function..FnOnce$LT$A$GT$$u20$for$u20$$RF$F$GT$::call_once::h9983aae1f0c22e8a
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13
std::panicking::try::do_call::h066238e64a26bede
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
std::panicking::try::h33c132dbcd21b2c1
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
std::panic::catch_unwind::h105dc9d93356acbb
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
std::rt::lang_start_internal::_$u7b$$u7b$closure$u7d$$u7d$::h541039ddd50eb0d5
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48
std::panicking::try::do_call::h6d067e65329ff03d
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40
std::panicking::try::h68073e7dd5111ae0
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19
std::panic::catch_unwind::h6d258ee41ea7e92f
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14
std::rt::lang_start_internal::h71c6fdbbf3980bb9
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20
#12 0x00000000010b6450 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
std::rt::lang_start::ha150faaacc9fba53
/rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:165:17
#13 0x00000000010bacd8 /data/local/tmp/deno_snapshot_builder/test_build_deno_core
main
??:0:0
#14 0x000000000004a320 /apex/com.android.runtime/lib64/bionic/libc.so (__libc_init+96) (BuildId: feb08ab3b0daed1a79b081a68ee9fbfd)
Crash dump is completed

个人对v8研究不是特别深入，看字面意思似乎是字符串相关问题导致的？看来只能换个版本了

Answer 10 · 2024-03-07T02:21:59.000Z

最新版 deno (aarch64-linux-gnu) 使用 proot 是可以在 Android 正常运行的, 所以这确实是个奇怪的 BUG.

Answer 11 · 2024-03-07T06:00:03.000Z

最新版 deno (aarch64-linux-gnu) 使用 proot 是可以在 Android 正常运行的, 所以这确实是个奇怪的 BUG.

根据堆栈信息我翻看了一下 deno_core 源码

bindings.rs

pub(crate) fn initialize_deno_core_namespace<'s>(
  scope: &mut v8::HandleScope<'s>,
  context: v8::Local<'s, v8::Context>,
  init_mode: InitMode,
) {
  let global = context.global(scope);
  let deno_str = v8_static_strings::new(scope, &v8_static_strings::DENO);

  let maybe_deno_obj_val = global.get(scope, deno_str.into());

  // If `Deno.core` is already set up, let's exit early.
  if let Some(deno_obj_val) = maybe_deno_obj_val {
    if !deno_obj_val.is_undefined() {
      return;
    }
  }
...

&v8_static_strings::DENO 似乎是这个导致的？

bindings.rs

pub mod v8_static_strings {
  pub fn new<'s>(
    scope: &mut v8::HandleScope<'s>,
    str_: &'static v8::OneByteConst,
  ) -> v8::Local<'s, v8::String> {
    v8::String::new_from_onebyte_const(scope, str_).unwrap()
  }

  pub fn new_from_static_str<'s>(
    scope: &mut v8::HandleScope<'s>,
    str_: &'static [u8],
  ) -> v8::Local<'s, v8::String> {
    v8::String::new_external_onebyte_static(scope, str_).unwrap()
  }

  macro_rules! onebyte_const {
    ($ident:ident, $str_:literal) => {
      pub static $ident: v8::OneByteConst =
        v8::String::create_external_onebyte_const($str_.as_bytes());
    };
  }

  onebyte_const!(DENO, "Deno");
  onebyte_const!(CORE, "core");
  onebyte_const!(OPS, "ops");
...

onebyte_const!(DENO, "Deno"); 会生成一个 OneByteConst

v8/string.rs

  // Creates a v8::String from a `&'static [u8]`,
  // must be Latin-1 or ASCII, not UTF-8 !
  #[inline(always)]
  pub fn new_external_onebyte_static<'s>(
    scope: &mut HandleScope<'s, ()>,
    buffer: &'static [u8],
  ) -> Option<Local<'s, String>> {
    let buffer_len = buffer.len().try_into().ok()?;
    unsafe {
      scope.cast_local(|sd| {
        v8__String__NewExternalOneByteStatic(
          sd.get_isolate_ptr(),
          buffer.as_ptr() as *const char,
          buffer_len,
        )
      })
    }
  }

然后在 v8 会使用这个 v8__String__NewExternalOneByteStatic 在往下就是 v8 的 c源码了
估计是这里引用了空指针了吧，似乎是因为没对齐导致的？(没有那么了解)

Answer 12 · 2024-03-10T10:30:39.000Z

Sorry for bumbling into this thread but the timing seemed serendipitous, since I was also trying to get some Android Deno builds off the ground.

I ran exactly into the:

[v8 0.74.1] ERROR at //build/android/BUILD.gn:93:17: Could not read file.
[v8 0.74.1]   pydeps_file = "pylib/results/presentation/test_results_presentation.pydeps"
[v8 0.74.1]                 ^------------------------------------------------------------

I'm not sure if you folks already have a plan of action but wouldn't it be possible to tweak the "gn gen" incantation to exclude all the testonly targets? Presumably we don't care about test targets for rusty_v8 or Deno. Ideally, this wouldn't only solve the issue at hand but also slim builds?