apache/incubator-horaedb-meta

The leader node of CeresMeta will OOM when the CeresDB node restarts, and it will also fail when electing new leader.

ZuLiangWang opened this issue · 0 comments

Describe this problem
The leader node of CeresMeta will OOM when the CeresDB node restarts, and it will also fail when electing new leader.

Steps to reproduce

  1. Deploy version 1.0 of CeresDB and CeresMeta.
  2. There are a large number of read and write requests.
  3. Restart some CeresDB nodes.

Expected behavior

  1. CeresMeta will not experience OOM when CeresDB node is restarted.
  2. Under no circumstances should it affect the normal election of CeresMeta.

Additional Information
The preliminary reason for the current positioning is that under the implementation of the current Java Client, when Route fails, it will cause all Routes to be refreshed and send a Route request to CeresDB. This will result in CeresMeta receiving a large number of Route requests in a short period of time, and CeresMeta currently does not have buffering for the Route interface. Each request will read etcd, which causes this issue with etcd.