Operator is not eligible for application

Question

Operator is not eligible for application

c29r3 opened this issue 4 years ago · 12 comments

Problem:

WARNI keep-ecdsa: operator is not eligible for application [0x14dC06F762E7f4a756825c1A1dA569b3180153cB] registration.go:138

It's been about a day since the launch of ECDSA node and I still see this error
I'm configure my ECDSA node according this manual https://gist.github.com/afmsavage/8fc19937a6b263f05c3e215d8860629c
Client version: keepnetwork/keep-ecdsa-client:v1.1.2-rc
-staked 300K test KEEP
-authorized contracts in dasboard
Port 3919 is available if i check it from another ip

nc -t -vv $EXTERNAL_IP 3919
Connection to MY_IP 3919 port [tcp/*] succeeded!
/multistream/1.0.0

Log file: ecdsa_node_log.txt

Config file:

[ethereum]
  URL = "wss://ropsten.infura.io/ws/v3/$INFURA_TOKEN"
  URLRPC = "https://ropsten.infura.io/v3/$INFURA_TOKEN"


[ethereum.account]
  Address = "0xc2C5b1F115b4c82fB0c1ce6208AcD7089e3B506D"
  KeyFile = "/mnt/keep-ecdsa/keystore/keep_wallet.json"

[ethereum.ContractAddresses]
  BondedECDSAKeepFactory = "0xe7BF8421fBE80c3Bf67082370D86C8D81D1D77F4"

[SanctionedApplications]
  Addresses = ["0x14dC06F762E7f4a756825c1A1dA569b3180153cB"]

[Storage]
  DataDir = "/mnt/keep-ecdsa/persistence"
  
[LibP2P]
  Peers = ["/dns4/testnet.keep-client.hashd.dev/tcp/3920/ipfs/16Uiu2HAmJsBiNVFNxsJ27NSQEByv39B1M7AKx5FrAc1htqYhHGhU","/ip4/3.23.88.229/tcp/3919/ipfs/16Uiu2HAmEZpkf1Td8rSBMmgPoa66si2kJLb83Rd2eztJ6f5oLvhp","/dns4/ecdsa-2.test.keep.network/tcp/3919/ipfs/16Uiu2HAmNNuCp45z5bgB8KiTHv1vHTNAVbBgxxtTFGAndageo9Dp",	
"/dns4/ecdsa-3.test.keep.network/tcp/3919/ipfs/16Uiu2HAm8KJX32kr3eYUhDuzwTucSfAfspnjnXNf9veVhB12t6Vf","/dns4/bootstrap-1.ecdsa.keep.test.boar.network/tcp/4001/ipfs/16Uiu2HAmPFXDaeGWtnzd8s39NsaQguoWtKi77834A6xwYqeicq6N"]
Port = 3919
 AnnouncedAddresses = ["/ip4/$EXTERNAL_IP/tcp/3919"]

[TSS]
  PreParamsGenerationTimeout = "2m30s"

Answer 1 · 2020-07-17T16:25:45.000Z

I have the same issues with all my ECDSA nodes in Ropsten. The nodes staked 300K test KEEP each and a large amount of Ropsten ETH (200-300). Node restart does not help to resolve this warning.

Answer 2 · 2020-07-17T16:34:09.000Z

Experiencing the same issue operator is not eligible for application [0x14dC06F762E7f4a756825c1A1dA569b3180153cB]

Answer 3 · 2020-07-17T20:14:23.000Z

Thanks for reporting, we're looking into it. To be clear bonded > 100 eth, means you have 100 eth available for bond, or 100 ETH that's already been bonded?

Answer 4 · 2020-07-17T20:22:54.000Z

Thanks for reporting, we're looking into it. To be clear bonded > 100 eth, means you have 100 eth available for bond, or 100 ETH that's already been bonded?

Answer 5 · 2020-07-17T20:51:40.000Z

Thanks for reporting, we're looking into it. To be clear bonded > 100 eth, means you have 100 eth available for bond, or 100 ETH that's already been bonded?

Both > 100 eth: already bonded and available to bond, look at screenshot, for example (ecdsa operator 0x6b2535194feb6f5834a3579889a702d4498aa5cc)

Answer 6 · 2020-07-21T14:07:08.000Z

Thanks for reporting it. I looked at the operator address provided by @c29r3 and I see it's now both eligible for the application and registered to the application. I have also looked at the operator address provided by @k0kk0k and I see it's now eligible for the application but not yet registered (maybe the client is not running).

I think the issue here is related to #506. We accidentally tweaked the price to be very high when we meant it to be very low, so most bonds became insufficient. When we opened a deposit under the very high price, the bond expectations in the sortition pool skyrocketed and a lot of clients became no longer eligible for tBTC pool. Over the weekend, we fixed the price (tx 0xe8275ee1d935e949678354cc4020200f8235382775e7ae2524d312c23d859646) and I think it helped - we had 12 operators in the pool this morning and I see we have 16 now and new deposits opened.

I'll wait for confirmation from @c29r3, @k0kk0k, and @JaviAnibarro - if everything works as expected now, I would consider this issue as fixed. I am also very interested if your clients automatically joined the pool once they became eligible or if you had to restart them - please see #500.

Answer 7 · 2020-07-21T14:19:27.000Z

I'll wait for confirmation from @c29r3,

That's right, today I rebooted the ECDSA operator and it successfully registered (reboot didn't help before).

I am also very interested if your clients automatically joined the pool once they became eligible or if you had to restart them

No, it can't join automatically. Manual register via js script or docker container restart ecdsa

Answer 8 · 2020-07-21T14:25:17.000Z

@c29r3 Can you please post your client logs from the moment before and after the restart in #500? I can't replicate the issue about the client not registering in the pool automatically and any hints are helpful.

Answer 9 · 2020-07-21T15:27:56.000Z

@pdyraga this is what my operator shows me before and after restarting the node. It changed from operator is not eligible for application into this automatically.

Answer 10 · 2020-07-21T15:51:20.000Z

@c29r3 Can you please post your client logs from the moment before and after the restart in #500? I can't replicate the issue about the client not registering in the pool automatically and any hints are helpful.

Such messages were today before manual registration by a script (I reboot the rest of the ecdsa nodes). I noticed these messages appeared after someone did a lot of mint in a row
https://discord.com/channels/590951101600235531/723194152925397062/733267579405074512

ecdsa_log_2020-07-21.zip

Answer 11 · 2020-07-21T17:50:56.000Z

the message Operator is not eligible for application was automatically disappear on my nodes. a restart was not required. I attach a log file, but it will be difficult to find the moment where this change occurred.
logs.tar.gz

Answer 12 · 2020-07-22T12:54:54.000Z

Sorry folks, we think we know what happened here:

Late last week, I made an off-hand suggestion to reduce the apparent ETHBTC price on Ropsten to reduce the bond requirement, allowing more deposits to be opened while requiring less testnet ETH from operators.
Unfortunately, because I was doing this while doing several other things, I actually increased the ETHBTC price 2 orders of magnitude instead of decreasing it. This led to a situation where most available bonds were insufficient to cover any deposits.
We then fixed this, reducing the ETHBTC price as intended. At this point, however, the sortition pool that handles choosing operators was set to expect a higher bond amount, and this amount adapts based on new deposits---so we needed to be able to open a deposit with the lower amount to let all operators rejoin.
We juiced up a few of our operators to get past that hurdle. Here, we saw new deposits failing to open due to a different issue (“Insufficient signer bonds to cover setup fee”). This was where I finally devoted more than “off hand” time to the problem, and remembered the fact that our bottom limit on bonds isn't just limited by the apparent ETHBTC price, but also by the opening cost of a deposit.

We've reverted the apparent ETHBTC price to what it was originally (which is a realistic amount based on relative mainnet prices from a couple of months ago), so new instances of “operator is not eligible for application” should not be related to this issue.

Thanks for bearing with us y'all… We've been focused on getting the relevant bits for a redeploy out the door, so we've had our attention reduced on the community and the Ropsten deploy the last few weeks. We expect that to start changing this week.