Factual/skuld

Random test failure: stress-test: task already claimed

Closed this issue · 1 comments

eric commented

https://travis-ci.org/Factual/skuld/jobs/29787355#L1682

2014-07-12T19:30:54.036 INFO  skuld.node: 127.0.0.1:13003: claim-local: claiming id from queue: #<Bytes 000001472c0e7edd800000010000000000000828>
2014-07-12T19:30:54.039 WARN  skuld.node: 127.0.0.1:13003: caught while claiming #<Bytes 000001472c0e7edd800000010000000000000828> from vnode 127.0.0.1:13003/skuld_1
java.lang.IllegalStateException: task already claimed
    at skuld.task$request_claim.invoke(task.clj:69) ~[na:na]
    at skuld.task$claim.invoke(task.clj:78) ~[na:na]
    at skuld.db.level.Level.claim_task_BANG_(level.clj:27) ~[na:na]
    at skuld.vnode$claim_BANG_.invoke(vnode.clj:490) ~[na:na]
    at skuld.node$claim_local_BANG_$fn__4892.invoke(node.clj:305) ~[na:na]
    at skuld.node$claim_local_BANG_.invoke(node.clj:304) [na:na]
    at skuld.node$claim_BANG_.invoke(node.clj:320) [na:na]
    at skuld.node$handler$handler__4933.invoke(node.clj:420) [na:na]
    at skuld.net$compile_handler$compiled_handler__4077.invoke(net.clj:297) [na:na]
    at skuld.net$handler$fn__4027$fn__4028$fn__4029.invoke(net.clj:110) [na:na]
    at skuld.net$handler$fn__4027$fn__4028.invoke(net.clj:110) [na:na]
    at clojure.core$binding_conveyor_fn$fn__4107.invoke(core.clj:1836) [clojure-1.5.1.jar:na]
    at clojure.lang.AFn.call(AFn.java:18) [clojure-1.5.1.jar:na]
    at java.util.concurrent.FutureTask.run(FutureTask.java:262) [na:1.7.0_51]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_51]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_51]
    at java.lang.Thread.run(Thread.java:744) [na:1.7.0_51]
eric commented

This actually wasn't a fatal exception. During shifting leadership of a partition, a node who was previously the leader will still have all of the task IDs in the nodes queue, but it will no longer be able to serve those tasks.

I've cleaned up how these exceptions are logged in #85.