ArweaveTeam/arweave

Start local testnet crash

yangcancai opened this issue · 8 comments

yangcancai commented

The rebar.config

{erl_opts, [debug_info,
{i,"apps"},
      {d,'TESTNET',true},
      {d,'REWARD_HISTORY_BLOCKS',720},
      {d,'NONCE_LIMITER_MAX_CHECKPOINTS_COUNT',1320},
      {d,'NONCE_LIMITER_RESET_FREQUENCY',3000},
      {d,'PRICE_2_6_TRANSITION_BLOCKS',324000},
      {d,'NETWORK_NAME',"arweave.2.6.testnet"}]}.

Start the testnet will fail

Cannot do a trusted join - there are not enough checkpoints to apply quickly; step number: 8243300, previous step number: 8241060.[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
[os_mon] memory supervisor port (memsup): Erlang has closed

How to Fix This ? Thx

yangcancai commented

When I Changed rebar.config to

{d,'NONCE_LIMITER_MAX_CHECKPOINTS_COUNT',2240},

Other error show

Downloaded the block trail successfully.
[error] GenServer :ar_nonce_limiter terminating
** (MatchError) no match of right hand side value: 1320
    (arweave 2.6.7.1) /root/arweave_ex/arweave/apps/arweave/src/ar_nonce_limiter.erl:751: :ar_nonce_limiter.apply_chain/2
    (arweave 2.6.7.1) /root/arweave_ex/arweave/apps/arweave/src/ar_nonce_limiter.erl:472: :ar_nonce_limiter.handle_cast/2
    (stdlib 4.3) gen_server.erl:1123: :gen_server.try_dispatch/4
    (stdlib 4.3) gen_server.erl:1200: :gen_server.handle_msg/6
    (stdlib 4.3) proc_lib.erl:240: :proc_lib.init_p_do_apply/3
Last message: {:"$gen_cast", {:initialize, [{:block, 205, <<79, 80, 72, 200, 169, 47, 75, 156, 52, 13, 218, 62, 115, 244, 89, 90, 217, 72, 105, 61, 131, 205, 95, 134, 12, 200, 243, 48, 81, 85, 127, 4, 118, 187, 178, 223, 216, 83, 30,
vird commented
  1. I'm using testnet and I had this error, but it's from time to time. All code should work without patches
  2. What is your launch script?
    This should work
./_build/testnet/rel/arweave/bin/start data_dir $DATA_DIR \
  peer testnet-1.arweave.net peer testnet-3.arweave.net peer testnet-5.arweave.net

Currently testnet-5.arweave.net doesn't work, but testnet-6.arweave.net works

yangcancai commented

This the launch script

 ./_build/testnet/rel/arweave/bin/start peer testnet-1.arweave.net peer testnet-2.arweave.net peer testnet-3.arweave.net

Currently testnet-2.arweave.net doesn't work.
I will try the startup script you suggested.

yangcancai commented

I used the script you suggested to start, but I still encountered the same error. Here is the complete log:

 arweave git:(master) ✗ bin/start data_dir ./data  peer testnet-1.arweave.net peer testnet-6.arweave.net peer testnet-3.arweave.net
Launching Erlang Virtual Machine...
Exec: /root/arweave_ex/arweave/_build/testnet/rel/arweave/erts-13.2/bin/erlexec -noinput +Bd -boot /root/arweave_ex/arweave/_build/testnet/rel/arweave/releases/2.6.9/start -mode embedded -boot_var SYSTEM_LIB_DIR /root/arweave_ex/arweave/_build/testnet/rel/arweave/lib -config /root/arweave_ex/arweave/_build/testnet/rel/arweave/releases/2.6.9/sys.config -args_file /root/arweave_ex/arweave/_build/testnet/rel/arweave/releases/2.6.9/vm.args -- foreground +MBas aobf +MBlmbcs 512 +Ktrue +A20 +SDio20 +sbwtvery_long +sbwtdcpuvery_long +sbwtdiovery_long +swtvery_low +swtdcpuvery_low +swtdiovery_low +Bi -run ar main data_dir ./data peer testnet-1.arweave.net peer testnet-6.arweave.net peer testnet-3.arweave.net
Root: /root/arweave_ex/arweave/_build/testnet/rel/arweave
/root/arweave_ex/arweave/_build/testnet/rel/arweave

****************************************
Launching with config:
init: false
port: 1984
mine: false
peers: ["95.216.19.227:1984","65.21.73.167:1984","148.251.135.52:1984"]
block_gossip_peers: []
local_peers: []
sync_from_local_peers_only: false
data_dir: "./data"
log_dir: "logs"
metrics_dir: "metrics"
polling: 2
block_pollers: 10
auto_join: true
join_workers: 10
diff: 6
mining_addr: not_set
max_miners: 0
io_threads: 10
stage_one_hashing_threads: 1
stage_two_hashing_threads: 1
hashing_threads: 1
mining_server_chunk_cache_size_limit: undefined
packing_cache_size_limit: undefined
data_cache_size_limit: undefined
tx_validators: undefined
post_tx_timeout: 20
max_emitters: 16
tx_propagation_parallelization: undefined
sync_jobs: 100
header_sync_jobs: 1
disk_pool_jobs: 20
load_key: not_set
disk_space: undefined
disk_space_check_frequency: 30000
storage_modules: []
start_from_block_index: false
internal_api_secret: not_set
enable: []
disable: []
transaction_blacklist_files: []
transaction_blacklist_urls: []
transaction_whitelist_files: []
transaction_whitelist_urls: []
gateway_domain: not_set
gateway_custom_domains: []
requests_per_minute_limit: 900
requests_per_minute_limit_by_ip: #{}
max_propagation_peers: 16
max_block_propagation_peers: 1000
ipfs_pin: false
webhooks: []
max_connections: 1024
max_gateway_connections: 128
max_poa_option_depth: 500
disk_pool_data_root_expiration_time: 1800
max_disk_pool_buffer_mb: 100000
max_disk_pool_data_root_buffer_mb: 10000
randomx_bulk_hashing_iterations: 8
semaphores: #{arql => 10,gateway_arql => infinity,get_and_pack_chunk => 1,
              get_block_index => 1,get_chunk => 100,get_sync_record => 10,
              get_tx_data => 1,get_wallet_list => 1,post_chunk => 100,
              post_tx => 20}
disk_cache_size: 5120
packing_rate: undefined
max_nonce_limiter_validation_thread_count: 1
max_nonce_limiter_last_step_validation_thread_count: 1
nonce_limiter_server_trusted_peers: []
nonce_limiter_client_peers: []
debug: false
repair_rocksdb: []
run_defragmentation: false
defragmentation_trigger_threshold: 1500000000
defragmentation_modules: []
block_throttle_by_ip_interval: 1000
block_throttle_by_solution_interval: 2000
p3: {p3_config,#{},#{}}
****************************************
=WARNING REPORT==== 8-Jun-2023::02:13:54.267121 ===
WARNING: Running only one CPU core / Erlang scheduler may cause issues.


VDF step computed in 1.53 seconds.

WARNING: your VDF computation speed is low - consider fetching VDF outputs from an external source (see vdf_server_trusted_peer and vdf_client_peer command line parameters).


Setting the mining address to ut6KPvYjIEBIX4HwLp7Vei4zBVPbKVvXga5j_AvfXpg.

Initialising RandomX dataset for fast packing. Key: <<"ZGVmYXVsdCBhcndlYXZlIDIuNSBwYWNrIGtleQ">>. The process may take several minutes.
RandomX dataset initialisation complete.

The node is configured to pack around 10 chunks per second. To increase the packing rate, start with `packing_rate [number]`. Estimated maximum rate: 8.93 chunks/s.

Setting the packing chunk cache size limit to 1300 chunks.

Sync request latency target is: 10000ms.

Setting the data chunk cache size limit to 1100 chunks.
Joining the Arweave network...

The node has stopped syncing headers. Add more disk space if you wish to store more block and transaction headers. The node will keep recording account tree updates and transaction confirmations - they do not take up a lot of space but you need to make sure the remaining disk space stays available for the node.

The mining performance is not affected.

The node has stopped syncing data into the storage module default due to the insufficient disk space.
Downloaded the block index successfully.
Downloading joining block 3mxYZiw25chOkAkd1cqA1ZDwGxITzoWwBS03qi9aKeXPS9fRkGWSEu1IpU-t98m_.
Downloading the block trail.
Downloaded the block trail successfully.
Downloading the wallet tree, chunk 1.
Downloading the wallet tree, chunk 2.
Downloading the wallet tree, chunk 3.
Downloading the wallet tree, chunk 4.
Cannot do a trusted join - there are not enough checkpoints to apply quickly; step number: 8243300, previous step number: 8241060.
Downloading the wallet tree, chunk 5.
Downloading the wallet tree, chunk 6.
Downloading the wallet tree, chunk 7.
Downloading the wallet tree, chunk 8.
Downloading the wallet tree, chunk 9.
Downloading the wallet tree, chunk 10.
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
[os_mon] memory supervisor port (memsup): Erlang has closed
Arweave Heartbeat: Server terminated safely.

In addition, my Erlang version is：

Erlang/OTP 25 [erts-13.2] [source] [64-bit] [smp:1:1] [ds:1:1:10] [async-threads:1] [jit:ns]

Eshell V13.2  (abort with ^G)
  • 👍1
MeerKatDev commented

I get this instead:

Initialising RandomX dataset for fast packing. Key: <<"ZGVmYXVsdCBhcndlYXZlIDIuNSBwYWNrIGtleQ">>. The process may take several minutes.
RandomX dataset initialisation complete.

The node is configured to pack around 10 chunks per second. To increase the packing rate, start with `packing_rate [number]`. Estimated maximum rate: 8.48 chunks/s.

Setting the packing chunk cache size limit to 600 chunks.

Setting the data chunk cache size limit to 600 chunks.

        Peer 95.216.19.227:1984 does not belong to the network arweave.N.1.


        Peer 5.9.18.85:1984 does not belong to the network arweave.N.1.


        Peer 148.251.135.52:1984 does not belong to the network arweave.N.1.

The specified trusted peers are not valid.
[os_mon] memory supervisor port (memsup): Erlang has closed
[os_mon] cpu supervisor port (cpu_sup): Erlang has closed
Arweave Heartbeat: Server terminated safely.
vird commented

does not belong to the network arweave.N.1.
You specified mainnet peers
try testnet-1.arweave.net (1 - 6)

MeerKatDev commented

I used this command:

./bin/start peer testnet-1.arweave.net peer testnet-2.arweave.net peer testnet-3.arweave.net

as specified in the README

vird commented

Arweave needs to be compiled with testnet.
See readme ./rebar3 as testnet tar
Mainnet binary checks all peers to be matched with mainnet network name arweave.N.1

bin/start will be inside _build/testnet/rel/arweave

  • 👍1