Bot stops being able to fetch status over time

[cproxy]

Running docker 1.6.0 in unRaid 6.9.2
Initial setup works great, then ceases to update overnight.
Removing the game server and adding it again resolves the issue.

java.util.concurrent.CompletionException: java.util.concurrent.RejectedExecutionException
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.completion(SourceQueryMessenger.java:368) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.access$1000(SourceQueryMessenger.java:279) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.lambda$send$1(SourceQueryMessenger.java:240) ~[agql-source-query-1.0.5.jar:1.0.5]
at java.base/java.util.concurrent.CompletableFuture.uniHandle(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.uniHandleStage(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.handle(Unknown Source) ~[na:na]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.send(SourceQueryMessenger.java:240) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.send(SourceQueryMessenger.java:62) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.core.AbstractClient.send(AbstractClient.java:111) ~[agql-lib-core-1.0.5.jar:1.0.5]
at com.ibasco.agql.core.AbstractClient.send(AbstractClient.java:87) ~[agql-lib-core-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryClient.getPlayers(SourceQueryClient.java:228) ~[agql-source-query-1.0.5.jar:1.0.5]
at de.darkatra.vrising.discord.ServerQueryClient.getPlayerList(ServerQueryClient.kt:39) ~[classes/:1.6.0]
at de.darkatra.vrising.discord.ServerStatusMonitorService$launchServerStatusMonitor$1.invokeSuspend(ServerStatusMonitorService.kt:80) ~[classes/:1.6.0]
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.6.21.jar:1.6.21-release-334(1.6.21)]
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
Caused by: java.util.concurrent.RejectedExecutionException: null
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.execute(SourceQueryMessenger.java:303) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.send(SourceQueryMessenger.java:239) ~[agql-source-query-1.0.5.jar:1.0.5]
... 12 common frames omitted

2022-07-07 07:21:05.038 ERROR 1 --- [atcher-worker-4] d.d.v.d.ServerStatusMonitorService : Exception while fetching the status of bdf8b1eb-fd61-11ec-a5b9-8518966c8d10

[DarkAtra]

Looks like the Threadpool used by the ServerQueryClient was somehow terminated. Which version of the bot are you using?

[cproxy]

Looks like the Threadpool used by the ServerQueryClient was somehow terminated. Which version of the bot are you using?

1.6.0

let me know if I can help with anything else.

[DarkAtra]

i'll debug for a bit and see if i can reproduce the issue with 1.6.x.

[DarkAtra]

There a new dev release that could potentially fix the issue. Feel free to test and report back: https://github.com/DarkAtra/v-rising-discord-bot/releases/tag/v1.6.2-next.1

The fix basically gets rid of the custom executor used to perform the server queries and delegates everything to the query lib.

[cproxy]

I updated the. Removed the server and added it again. Unfortunately it still fails at some point. If you’d like me to run some debug code, let me know.

java.util.concurrent.CompletionException: com.ibasco.agql.core.exceptions.MessengerException: com.ibasco.agql.core.exceptions.ReadTimeoutException
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.completion(SourceQueryMessenger.java:368) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.access$1000(SourceQueryMessenger.java:279) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.lambda$send$1(SourceQueryMessenger.java:240) ~[agql-source-query-1.0.5.jar:1.0.5]
at java.base/java.util.concurrent.CompletableFuture.uniHandle(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.postComplete(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.completeExceptionally(Unknown Source) ~[na:na]
at com.ibasco.agql.core.NettyChannelContext.markInError(NettyChannelContext.java:361) ~[agql-lib-core-1.0.5.jar:1.0.5]
at com.ibasco.agql.core.NettyMessenger.receive(NettyMessenger.java:340) ~[agql-lib-core-1.0.5.jar:1.0.5]
at com.ibasco.agql.core.NettyChannelContext.receive(NettyChannelContext.java:375) ~[agql-lib-core-1.0.5.jar:1.0.5]
at com.ibasco.agql.core.transport.handlers.MessageRouter.exceptionCaught(MessageRouter.java:208) ~[agql-lib-core-1.0.5.jar:1.0.5]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at com.ibasco.agql.protocols.valve.source.query.common.handlers.SourceQuerySplitPacketAssembler.exceptionCaught(SourceQuerySplitPacketAssembler.java:149) ~[agql-source-query-1.0.5.jar:1.0.5]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at com.ibasco.agql.core.transport.handlers.MessageDecoder.exceptionCaught(MessageDecoder.java:93) ~[agql-lib-core-1.0.5.jar:1.0.5]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273) ~[netty-transport-4.1.78.Final.jar:4.1.78.Final]
at com.ibasco.agql.core.transport.handlers.ReadTimeoutHandler.readTimedOut(ReadTimeoutHandler.java:78) ~[agql-lib-core-1.0.5.jar:1.0.5]
at com.ibasco.agql.core.transport.handlers.ReadTimeoutHandler.channelIdle(ReadTimeoutHandler.java:63) ~[agql-lib-core-1.0.5.jar:1.0.5]
at io.netty.handler.timeout.IdleStateHandler$ReaderIdleTimeoutTask.run(IdleStateHandler.java:503) ~[netty-handler-4.1.78.Final.jar:4.1.78.Final]
at io.netty.handler.timeout.IdleStateHandler$AbstractIdleTask.run(IdleStateHandler.java:475) ~[netty-handler-4.1.78.Final.jar:4.1.78.Final]
at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[netty-common-4.1.78.Final.jar:4.1.78.Final]
at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:153) ~[netty-common-4.1.78.Final.jar:4.1.78.Final]
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[netty-common-4.1.78.Final.jar:4.1.78.Final]
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[netty-common-4.1.78.Final.jar:4.1.78.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[netty-common-4.1.78.Final.jar:4.1.78.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:394) ~[netty-transport-classes-epoll-4.1.78.Final.jar:4.1.78.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) ~[netty-common-4.1.78.Final.jar:4.1.78.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.78.Final.jar:4.1.78.Final]
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.78.Final.jar:4.1.78.Final]
at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
Caused by: com.ibasco.agql.core.exceptions.MessengerException: com.ibasco.agql.core.exceptions.ReadTimeoutException
at com.ibasco.agql.core.NettyMessenger.wrapException(NettyMessenger.java:285) ~[agql-lib-core-1.0.5.jar:1.0.5]
at java.base/java.util.concurrent.CompletableFuture.biApply(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture$BiApply.tryFire(Unknown Source) ~[na:na]
... 33 common frames omitted
Caused by: com.ibasco.agql.core.exceptions.ReadTimeoutException: null
at com.ibasco.agql.core.exceptions.ReadTimeoutException.<clinit>(ReadTimeoutException.java:27) ~[agql-lib-core-1.0.5.jar:1.0.5]
... 16 common frames omitted

2022-07-08 12:15:31.593 ERROR 1 --- [atcher-worker-4] d.d.v.d.ServerStatusMonitorService : Exception while fetching the status of dd614c11-fef0-11ec-b3c4-955e6b24f2bd


java.util.concurrent.CompletionException: java.util.concurrent.RejectedExecutionException
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.completion(SourceQueryMessenger.java:368) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.access$1000(SourceQueryMessenger.java:279) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.lambda$send$1(SourceQueryMessenger.java:240) ~[agql-source-query-1.0.5.jar:1.0.5]
at java.base/java.util.concurrent.CompletableFuture.uniHandle(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.uniHandleStage(Unknown Source) ~[na:na]
at java.base/java.util.concurrent.CompletableFuture.handle(Unknown Source) ~[na:na]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.send(SourceQueryMessenger.java:240) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.send(SourceQueryMessenger.java:62) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.core.AbstractClient.send(AbstractClient.java:111) ~[agql-lib-core-1.0.5.jar:1.0.5]
at com.ibasco.agql.core.AbstractClient.send(AbstractClient.java:87) ~[agql-lib-core-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryClient.getPlayers(SourceQueryClient.java:228) ~[agql-source-query-1.0.5.jar:1.0.5]
at de.darkatra.vrising.discord.ServerQueryClient.getPlayerList(ServerQueryClient.kt:35) ~[classes/:1.6.2-next.1]
at de.darkatra.vrising.discord.ServerStatusMonitorService$launchServerStatusMonitor$1.invokeSuspend(ServerStatusMonitorService.kt:80) ~[classes/:1.6.2-next.1]
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) ~[kotlin-stdlib-1.6.21.jar:1.6.21-release-334(1.6.21)]
at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:749) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) ~[kotlinx-coroutines-core-jvm-1.6.3.jar:na]
Caused by: java.util.concurrent.RejectedExecutionException: null
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger$RequestContext.execute(SourceQueryMessenger.java:303) ~[agql-source-query-1.0.5.jar:1.0.5]
at com.ibasco.agql.protocols.valve.source.query.SourceQueryMessenger.send(SourceQueryMessenger.java:239) ~[agql-source-query-1.0.5.jar:1.0.5]
... 12 common frames omitted

2022-07-08 12:33:15.117 ERROR 1 --- [atcher-worker-5] d.d.v.d.ServerStatusMonitorService : Exception while fetching the status of dd614c11-fef0-11ec-b3c4-955e6b24f2bd

[DarkAtra]

are you certain that the bot is not updating the discord post after those messages appeared? one of them is just a read timeout which happens if the server is not responding in time (e.g. a temporary issue). the second one also happened to me from time to time but never led to long term issues for me.

[cproxy]

are you certain that the bot is not updating the discord post after those messages appeared? one of them is just a read timeout which happens if the server is not responding in time (e.g. a temporary issue). the second one also happened to me from time to time but never led to long term issues for me.

I was not certain! It seems to still be working. Thanks!

[DarkAtra]

closing this issue for now. please reopen if you encounter the same issue again.

btw, you should be able to switch back to the main branch by using version 1.7.0. it includes the same fix as the dev release.