ETL process started but no data is inserted

Question

ETL process started but no data is inserted

nikoes opened this issue 3 years ago · 5 comments

Hello,

I've followed all the steps and the ETL process is up and running.
However, no data has been inserted for the last 20 hours.

Here are some logs:

error.log

2021-08-20 05:14:30.353 [error] <0.9847.3> Supervisor {<0.9847.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,open_stre am,undefined} at <0.9851.3> exit with reason noproc in context shutdown_error
2021-08-20 05:19:52.679 [error] <0.10477.3> Supervisor {<0.10477.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive _stream,undefined} at <0.10481.3> exit with reason noproc in context shutdown_error
2021-08-20 05:48:52.440 [error] <0.14191.3> Supervisor {<0.14191.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive _stream,undefined} at <0.14195.3> exit with reason noproc in context shutdown_error
2021-08-20 06:03:54.502 [error] <0.12749.3> Supervisor {<0.12749.3>,libp2p_simple_sup} had child 27 started with {libp2p_yamux_stream,open_s tream,undefined} at <0.16226.3> exit with reason noproc in context shutdown_error
2021-08-20 06:28:41.869 [error] <0.19475.3> Supervisor {<0.19475.3>,libp2p_simple_sup} had child 2 started with {libp2p_yamux_stream,open_st ream,undefined} at <0.19478.3> exit with reason noproc in context shutdown_error
2021-08-20 06:49:58.278 [error] <0.22507.3> Supervisor {<0.22507.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive _stream,undefined} at <0.22511.3> exit with reason noproc in context shutdown_error
2021-08-20 06:59:18.397 [error] <0.23806.3> Supervisor {<0.23806.3>,libp2p_simple_sup} had child 3 started with {libp2p_yamux_stream,receive _stream,undefined} at <0.23812.3> exit with reason noproc in context shutdown_error
2021-08-20 06:59:18.397 [error] <0.23806.3> Supervisor {<0.23806.3>,libp2p_simple_sup} had child 1 started with {libp2p_yamux_stream,receive _stream,undefined} at <0.23810.3> exit with reason noproc in context shutdown_error
2021-08-20 07:00:05.639 [error] <0.23959.3> Supervisor {<0.23959.3>,libp2p_simple_sup} had child 3 started with {libp2p_yamux_stream,receive _stream,undefined} at <0.23979.3> exit with reason noproc in context shutdown_error
2021-08-20 07:20:17.066 [error] <0.1249.0> Supervisor {<0.1249.0>,libp2p_group_worker_sup} had child #Ref<0.3970633756.1779957763.133930> st arted with {libp2p_group_worker,start_link,undefined} at <0.26311.3> exit with reason killed in context shutdown_error

console.log

2021-08-20 07:27:16.983 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:27:22.660 [info] <0.1275.0>@blockchain_worker:handle_info:605 block sync down: normal
2021-08-20 07:27:22.662 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27440.3>, Ref: #Ref<0.3970633756.1779957768.128850>, Peer: "/p2p/112bpa8Y4KqXfHYhxG1rtpNaPmDJcianyP9ekmaYYfW2THMJydbx"
2021-08-20 07:27:26.984 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:27:36.904 [info] <0.1275.0>@blockchain_worker:handle_info:605 block sync down: normal
2021-08-20 07:27:36.906 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27456.3>, Ref: #Ref<0.3970633756.1779957761.151365>, Peer: "/p2p/11vyHF7YcEVu4xgXxxGAGsZCoQE9bXsV7qLmM8QFdh2q9G31mch"
2021-08-20 07:27:36.985 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:27:46.986 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:27:47.366 [info] <0.1275.0>@blockchain_worker:handle_info:605 block sync down: normal
2021-08-20 07:27:47.367 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27471.3>, Ref: #Ref<0.3970633756.1779957768.129121>, Peer: "/p2p/11LbRh3ztxfKcbiW2JD7sCBDrXLSFsJi2KAgVvWVeyz48weGti2"
2021-08-20 07:27:52.396 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27476.3>, Ref: #Ref<0.3970633756.1779957767.132306>, Peer: "/p2p/11ckxdQsReXpqwCrbbREZj6urEuNEGf2Zk5d4UnsuPMsJDSihwy"
2021-08-20 07:27:56.987 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:28:03.413 [info] <0.1275.0>@blockchain_worker:handle_info:605 block sync down: normal
2021-08-20 07:28:03.415 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27507.3>, Ref: #Ref<0.3970633756.1779957767.132478>, Peer: "/p2p/112jHZvWftRgdTgf1HXkSTfFS3r1ssrHP3Y7JMC92tENAhXMC27d"
2021-08-20 07:28:06.988 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:28:14.587 [info] <0.1275.0>@blockchain_worker:handle_info:605 block sync down: normal
2021-08-20 07:28:14.588 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27522.3>, Ref: #Ref<0.3970633756.1779957766.153635>, Peer: "/p2p/11ZvtNjXiFJv2C19MzvTcdNznTZoVEEH4iZHTtpWEE5TwJ1jKBn"
2021-08-20 07:28:16.989 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:28:26.243 [info] <0.1275.0>@blockchain_worker:handle_info:605 block sync down: normal
2021-08-20 07:28:26.244 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27538.3>, Ref: #Ref<0.3970633756.1779957766.153720>, Peer: "/p2p/112shbCtf29VMsqdZHXEmfnQV2g2iuck5LWCwg7JWVctm9eYLJpZ"
2021-08-20 07:28:26.990 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:28:36.991 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:28:37.577 [info] <0.1275.0>@blockchain_worker:handle_info:605 block sync down: normal
2021-08-20 07:28:37.579 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27553.3>, Ref: #Ref<0.3970633756.1779957762.169849>, Peer: "/p2p/11ehDCG8hE9k58BNs6vVxbU2fpZ1gTt2yKAefxXzhewA7mp3E8C"
2021-08-20 07:28:42.610 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27554.3>, Ref: #Ref<0.3970633756.1779957767.132675>, Peer: "/p2p/112A1ewfWuPHchWy2peSZ8GLGxDUPDCm4ttgfFVSwYk1zgpSi6xa"
2021-08-20 07:28:46.992 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:28:48.039 [notice] <0.27567.3>@libp2p_stream_identify:handle_info:64 Identify timed out
2021-08-20 07:28:48.039 [warning] <0.1245.0>@libp2p_swarm_server:handle_info:66 ignoring session after failed identify "/ip4/84.15.127.163/tcp/44158": timeout
2021-08-20 07:28:48.039 [notice] <0.1246.0>@libp2p_transport_tcp:handle_info:473 session identification failed for "/ip4/84.15.127.163/tcp/44158": timeout
2021-08-20 07:28:52.989 [info] <0.1275.0>@blockchain_worker:handle_info:605 block sync down: normal
2021-08-20 07:28:52.990 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27570.3>, Ref: #Ref<0.3970633756.1779957761.151913>, Peer: "/p2p/112Bifv2RjoYkyAJUEUUuH8CTpxjfgMGBVoDYkrkV3qN86MMwKTY"
2021-08-20 07:28:56.993 [info] <0.1415.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 07:28:58.022 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.27571.3>, Ref: #Ref<0.3970633756.1779957763.137550>, Peer: "/p2p/112X9uFX99jgJT7LMpsq651cRSAAZmmV8iDGM6sSktwmZ7gdSMzA"

crash.log

Context: shutdown_error
Reason: noproc
Offender: [{pid,<0.23979.3>},{id,3},{mfargs,{libp2p_yamux_stream,receive_stream,undefined}},{restart_type,temporary},{shutdown,1000},{child_type,worker}]

2021-08-20 07:20:17 =SUPERVISOR REPORT====
Supervisor: {<0.1249.0>,libp2p_group_worker_sup}
Context: shutdown_error
Reason: killed
Offender: [{pid,<0.26311.3>},{id,#Ref<0.3970633756.1779957763.133930>},{mfargs,{libp2p_group_worker,start_link,undefined}},{restart_type,temporary},{shutdown,5000},{child_type,worker}]

run_erl.log

run_erl [35737] Thu Aug 19 14:46:48 2021
Args before exec of shell:
run_erl [35737] Thu Aug 19 14:46:48 2021
argv[0] = sh
run_erl [35737] Thu Aug 19 14:46:48 2021
argv[1] = -c
run_erl [35737] Thu Aug 19 14:46:48 2021
argv[2] = exec "/home/helium/helium-etl/blockchain-etl/_build/dev/rel/blockchain_etl/bin/blockchain_etl" "console" '' --relx-disable-hooks

.env file

I haven't added Google API key.

All tables are empty and there is no data inserted in the database.
Please help.

Thank you for this great tool!

Answer 1 · 2021-08-20T09:54:01.000Z

I also found some invalid_binary in the log.

2021-08-20 09:46:46.039 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.6432.0>, Ref: #Ref<0.2698514698.3832807428.102051>, Peer: "/p2p/1Z9DrDd2MKxwip5qFBs7zm1bjKkUuzp4CbKchcYk1TKAsnV6fVn"
2021-08-20 09:46:46.091 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.6433.0>, Ref: #Ref<0.2698514698.3832807425.107980>, Peer: "/p2p/1Z9DrDd2MKxwip5qFBs7zm1bjKkUuzp4CbKchcYk1TKAsnV6fVn"
2021-08-20 09:46:50.025 [info] <0.1437.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 09:47:00.028 [info] <0.1437.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 09:47:00.511 [notice] <0.1246.0>@libp2p_transport_tcp:handle_info:473 session identification failed for "/ip4/13.237.200.108/tcp/443": invalid_binary
2021-08-20 09:47:00.511 [warning] <0.1245.0>@libp2p_swarm_server:handle_info:66 ignoring session after failed identify "/ip4/13.237.200.108/tcp/443": invalid_binary
2021-08-20 09:47:05.752 [notice] <0.1250.0>@libp2p_group_gossip_server:handle_info:284 Failed to identify stream <0.6488.0>: invalid_binary
2021-08-20 09:47:10.030 [info] <0.1437.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 09:47:20.032 [info] <0.1437.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions
2021-08-20 09:47:20.898 [warning] <0.1245.0>@libp2p_swarm_server:handle_info:66 ignoring session after failed identify "/ip4/72.224.176.69/tcp/56897": invalid_binary
2021-08-20 09:47:26.211 [warning] <0.1245.0>@libp2p_swarm_server:handle_info:66 ignoring session after failed identify "/ip4/72.224.176.69/tcp/60323": invalid_binary
2021-08-20 09:47:30.034 [info] <0.1437.0>@be_db_pending_txn:handle_info:136 Submitting 0 pending transactions

Some unsupported protocol

2021-08-20 11:45:19.397 [notice] <0.1589.0>@libp2p_multistream_client:negotiate_handler:24 Failed to negotiate handler for "blockchain_sync/1.2.0": protocol_unsupported
2021-08-20 11:45:20.803 [notice] <0.1589.0>@libp2p_multistream_client:negotiate_handler:24 Failed to negotiate handler for "blockchain_sync/1.1.0": protocol_unsupported
2021-08-20 11:45:20.804 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.1597.0>, Ref: #Ref<0.1874676760.633077762.34341>, Peer: "/p2p/112QnmBsb7YZ6eJLuEGjThwaomT7Tb9Ep7Hf8n7SuCnSBFLsFmwk"
2021-08-20 11:45:20.907 [info] <0.1275.0>@blockchain_worker:start_sync:758 new block sync starting with Pid: <0.1598.0>, Ref: #Ref<0.1874676760.633077762.34765>, Peer: "/p2p/112VFG7Rm1ziraPQadWJ3bQPbdqPTpz3CULiZiXpgLs47sbefw8q"

Answer 2 · 2021-08-22T02:47:21.000Z

@nikoes I follow instruction that's pinned message in Discoard#blockchain-development and it solved my issue

Download https://snapshots.helium.wtf/blockchain-797990.db.tgz
STOP your etl (make stop)
cd to _build/dev/rel/blockchain_etl/data/
Delete (rm -rf) blockchain.db
Untar the tar while in the data folder tar xzvf blockchain-794942.db.tgz
START your etl

Now my db has synced up to the tip of the mainnet.

Answer 3 · 2021-08-22T10:56:04.000Z

@varshard Thank you very much for your reply. I’ll try it out later today! Thanks!

Answer 4 · 2021-08-22T20:34:03.000Z

@varshard Hello,

I've tried what you've described but still my DB is empty.
Other than that, the log looks a lot cleaner

heap_size: 376
stack_size: 27
reductions: 30206
neighbours:
2021-08-22 20:17:39 =ERROR REPORT====
** Generic server <0.2326.0> terminating
** Last message in was {'EXIT',<0.1299.0>,killed}
** When Server state == {state,{connection,libp2p_transport_tcp,{tcp_state,{"/ip4/145.255.200.115/tcp/44158","/ip4/136.53.57.93/tcp/44158"},#Port<0.239>,undefined,ranch_tcp}},blockchain_swarm,<0.2328.0>,4,#{},<0.2330.0>,#{},1,#Ref<0.332153784.2499280900.63521>,#Ref<0.332153784.2499280904.38134>,60000,none,{ident,{libp2p_signed_identify_pb,{libp2p_identify_pb,{libp2p_signed_peer_pb,{libp2p_peer_pb,<<0,208,149,129,47,30,233,52,182,111,121,135,128,109,117,81,84,143,175,217,212,185,80,219,88,216,248,243,128,34,232,203,27>>,[<<164,3,38,0,0,47,61,157,230,22,38,253,194,197,155,61,121,151,193,15,111,158,49,20,235,130,114,19,138,18,153,9,10,248,88,64,233,241,156,87,64,162,2,56,112,50,112,47,49,49,50,97,114,122,117,98,84,121,77,121,110,75,122,98,113,84,57,71,109,122,100,84,67,49,84,90,78,102,116,115,74,104,54,112,51,69,78,85,100,117,77,86,78,101,113,119,74,112,50,97>>],[<<0,142,24,95,158,139,179,184,96,147,47,34,55,75,63,84,241,248,218,4,21,69,76,41,74,143,145,160,99,154,127,214,128>>,<<0,44,46,119,5,168,64,71,102,6,14,224,2,99,251,248,75,146,196,27,98,203,141,125,66,80,114,103,103,189,0,193,128>>,<<0,51,88,212,15,143,93,68,62,13,247,80,74,65,190,91,40,0,31,248,64,127,73,225,238,210,252,71,7,153,168,188,28>>,<<0,3,237,114,90,88,241,159,149,42,4,221,62,150,8,54,52,39,164,145,237,16,77,177,73,200,4,49,55,151,232,228,78>>,<<0,118,107,245,197,15,195,78,242,192,146,60,84,59,128,222,217,143,118,36,243,20,210,112,168,85,65,220,14,119,225,100,194>>,<<0,91,82,237,60,69,148,233,42,90,205,130,72,144,125,80,40,191,0,37,171,82,214,172,15,202,52,211,21,167,134,218,236>>,<<0,210,158,10,86,218,229,9,232,230,114,90,245,68,230,247,154,245,121,207,107,102,116,86,156,160,43,165,247,88,111,126,215>>,<<0,47,61,157,230,22,38,253,194,197,155,61,121,151,193,15,111,158,49,20,235,130,114,19,138,18,153,9,10,248,88,64,233>>],symmetric,1629663221785,[],<<45,174,143,185,95,52,62,207,106,76,253,55,223,211,240,155,174,42,223,247,94,49,57,47,232,147,168,215,110,117,16,146>>,[{"last_block_add_time",{libp2p_metadata_value_pb,{int,1629662853}}},{"release_info",{libp2p_metadata_value_pb,{bin,<<"unknown">>}}},{"height",{libp2p_metadata_value_pb,{int,977437}}}]},<<48,68,2,32,84,239,115,182,149,84,138,89,210,217,115,216,212,0,116,74,107,39,62,254,234,145,236,145,240,135,28,234,102,197,136,182,2,32,107,138,91,71,153,247,58,210,239,55,175,0,124,130,206,47,207,100,105,246,220,32,186,33,185,213,97,80,106,54,250,31>>,[]},<<4,145,255,200,115,6,172,126>>,<<70,224,104,204,110,200,146,7,206,88,33,41,129,37,124,225,161,116,102,181>>},<<48,69,2,33,0,207,78,173,165,202,70,20,217,25,55,4,217,64,52,167,231,56,246,119,84,134,208,129,199,207,132,57,243,243,34,212,191,2,32,48,127,79,3,108,19,36,19,58,182,221,226,211,36,59,173,154,10,146,218,139,138,33,131,172,21,99,10,69,77,175,224>>},undefined,[]}}
** Reason for termination ==
** killed

Answer 5 · 2021-08-23T06:36:51.000Z

@varshard Hi,

After 10 hours it seems to be working, the database is now receiving data.
Thank you for the help!