Deleting and recreating db causes pending timeouts
Closed this issue · 3 comments
We were struggling with strange behavior where our job was looping with pending timeouts right as segments should be have going to the Datomic output plugin (we saw all segments prior coming out of their tasks in the logs). It wouldn't happen every time and there were no exceptions thrown in the logs. We narrowed it down to the plugin by swapping in the core.async output and all was fine. It turned out that the issue was that we were deleting and recreating the database between job submissions (without restarting the peers or datomic). The job submission happened right after the call to d/create-database. When we stopped dropping and recreating the database between job submissions, we stopped experiencing this behavior. I know that's not a lot to go on but I thought I would make you aware of what we experienced.
Thanks for the report. We have previously seen bad behaviour when deleting / creating datomic DBs, however these have generally been caused by datomic's implementation as described here:
https://groups.google.com/forum/m/#!topic/datomic/6BDHEFmczps
As well as in other posts to the group (there can be a timeout before the DB become active).
If this is something you do a lot, I would suggest creating a new DB name each time.
Lucas
On 19 Jul 2016, at 6:39 AM, Roman Pearah notifications@github.com wrote:
We were struggling with strange behavior where our job was looping with pending timeouts right as segments should be have going to the Datomic output plugin (we saw all segments prior coming out of their tasks in the logs). It wouldn't happen every time and there were no exceptions thrown in the logs. We narrowed it down to the plugin by swapping in the core.async output and all was fine. It turned out that the issue was that we were deleting and recreating the database between job submissions (without restarting the peers or datomic). The job submission happened right after the call to d/create-database. When we stopped dropping and recreating the database between job submissions, we stopped experiencing this behavior. I know that's not a lot to go on but I thought I would make you aware of what we experienced.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.
I've seen the same behavior elsewhere. It's unrelated to Onyx, as Lucas said.
Got it. It's not normal operating procedure for us, so not a big deal.