Concordium/Testnet4-Challenges

Node not appearing on network dashboard

concordium-cl opened this issue · 4 comments

concordium-cl commented

It is possible that your node is running fine without appearing on the dashboard.

  • Run consensus status and then compare the finalized height and the best block height to the ones on the dashboard. If they are the same, or very close, your node is running correctly.

    • Windows: concordium-client consensus status
    • *nix/macOS: ./concordium-client consensus status

  • Run raw GetPeerData to see if your node has peers.

    • Windows: concordium-client raw GetPeerData
    • *nix/macOS: ./concordium-client raw GetPeerData

  • Check your nodeID on the network dashboard with https://dashboard.testnet.concordium.com/node/<yourNodeID> and if it is reachable, it will show your information.

  • If you want us to investigate why your node does not appears on the dashboard, you would need to retrieve your full logs (concordium-node-retrieve-logs) and then upload them somewhere from where we can get them (too big to be sent).

pablozsc commented

https://drive.google.com/drive/folders/1XJdC1niL6A2Z04Ah24ZxBngrNvWrtmbx?usp=sharing

The node is constantly disappearing from the dashboard.
Synchronized, everything is fine with the chain.
After restarting, it appears on the dashboard, then disappears after a while. And then everything repeats again.

abizjak commented

The logs show that the service that reports to the node dashboard keeps being killed, but it is unclear why.

The way to restart the service is to run

docker exec concordium-client supervisorctl restart collector

Unfortunately there is no useful logging for the collector normally. You could try running

docker exec concordium-client supervisorctl tail collector stdout                               
docker exec concordium-client supervisorctl tail collector stderr

to see whether anything specific triggered the problem.

I can see in the log that the failure of the collector starting happened on 2021-04-04 at around 10, which is after you restarted the node. Can you clarify what the sequence of events was?

Did you first notice that the node disappeared from the dashboard and then restarted, or the other way around?

mhmtozback commented

Hi. I cant see my node on dashboard ---- https://dashboard.testnet.concordium.com --- MODEKAL1

I see that on Linux screen .. How can I fix this problem ?

CallStack (from HasCallStack):
error, called at src/Concordium/GlobalState/Persistent/BlobStore.hs:158:27 in concordium-consensus-0.1.0.0-8r37AgxhHDq8bBZMuXhUwf:Concordium.GlobalState.Persistent.BlobStore
��`�U: warning: too many hs_exit()s
2021-04-13 08:01:57,693 INFO exited: concordium-node (exit status 0; expected)
2021-04-13 08:01:58,694 INFO spawned: 'concordium-node' with pid 160756
2021-04-13T08:01:58.716496462Z: INFO: Starting up concordium_node version 0.4.11!
2021-04-13T08:01:58.716508263Z: INFO: Application data directory: "/var/lib/concordium/data"
2021-04-13T08:01:58.716513765Z: INFO: Application config directory: "/var/lib/concordium/config"
2021-04-13T08:01:58.716516498Z: INFO: Network: enabled
2021-04-13T08:01:58.716517706Z: INFO: Log level: info
2021-04-13T08:01:58.716632541Z: INFO: My Node ID is a823360e60fcc373
2021-04-13T08:01:58.764558300Z: INFO: Starting consensus layer
2021-04-13T08:01:58.764569348Z: INFO: Starting up the consensus thread
2021-04-13T08:01:58.765090760Z: INFO: Starting up the consensus layer
2021-04-13 08:01:59,766 INFO success: concordium-node entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
concordium-node: too few bytes
From: demandInput

abizjak commented

@mhmtozback it looks like your node's database is corrupt so it cannot start.

The only solution is to stop the node, run concordium-node-reset-data and start the node again.

Note that this will delete any baker keys you may have.