linkedin/dynamometer

Add the ability to trigger block reports on simulated DataNodes

Closed this issue · 1 comments

xkrogen commented

Due to the way blocks are injected into the simulated DataNodes in Dynamometer, sometimes it is possible for the DataNode to send its initial block report completely empty. This is because the blocks are injected after the DataNode is started, due to the setup of a MiniDFSCluster. If this happens, another block report will not be sent until the block report interval has passed, which can be a very long time. This can result in (a) test timeouts (b) long setup times.

We can add the ability for the ApplicationMaster to monitor which nodes have not reported full block reports, and trigger block reports on those nodes.