Recurring exceptions connection to localhost:80 when only one Function is defined
oceanlewis opened this issue · 3 comments
oceanlewis commented
Describe the bug
We're using Spring Cloud Function at JOB and we've run into an interesting bug where: if we have only defined one Function for our project to run, we'll get recurring connection error exceptions and our application refuses to boot. I say interesting since we can avoid this issue by simple defining a second function, in which case the issue no longer presents itself.
Sample
I'll provide the log output here:
. ____ _ __ _ _
/\\ / ___'_ __ _ _(_)_ __ __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
\\/ ___)| |_)| | | | | || (_| | ) ) ) )
' |____| .__|_| |_|_| |_\__, | / / / /
=========|_|==============|___/=/_/_/_/
:: Spring Boot :: (v2.3.2.RELEASE)
2020-12-17 16:56:23.150 INFO 16262 --- [ main] a.d.m.f.my.function.Application$Main : Starting Application.Main on armstronglewis with PID 16262 (/home/david/src/.../my.function)
2020-12-17 16:56:23.185 INFO 16262 --- [ main] a.d.m.f.my.function.Application$Main : The following profiles are active: development
2020-12-17 16:56:24.548 WARN 16262 --- [ main] s.a.a.p.internal.ProfileFileReader : Ignoring profile 'test-deploy-profile' on line 37 because it did not start with 'profile ' and it was not 'default'.
2020-12-17 16:56:24.548 WARN 16262 --- [ main] s.a.a.p.internal.ProfileFileReader : Ignoring profile 'test-source-profile' on line 41 because it did not start with 'profile ' and it was not 'default'.
2020-12-17 16:56:24.548 WARN 16262 --- [ main] s.a.a.p.internal.ProfileFileReader : Ignoring profile 'test-destination-profile' on line 45 because it did not start with 'profile ' and it was not 'default'.
2020-12-17 16:56:25.514 INFO 16262 --- [ main] o.s.c.f.web.flux.FunctionHandlerMapping : FunctionCatalog: org.springframework.cloud.function.context.catalog.BeanFactoryAwareFunctionRegistry@10959ece
2020-12-17 16:56:25.808 INFO 16262 --- [ main] o.s.c.f.web.source.SupplierExporter : Starting
2020-12-17 16:56:25.825 INFO 16262 --- [ main] o.s.c.f.c.c.SimpleFunctionRegistry : Looking up function 'origin|' with acceptedOutputTypes: [application/json]
2020-12-17 16:56:25.919 INFO 16262 --- [ main] reactor.Flux.MonoRepeatPredicate.1 : onSubscribe(FluxRepeatPredicate.RepeatPredicateSubscriber)
2020-12-17 16:56:25.921 INFO 16262 --- [ main] reactor.Flux.MonoRepeatPredicate.1 : request(256)
2020-12-17 16:56:26.125 INFO 16262 --- [ main] o.s.b.web.embedded.netty.NettyWebServer : Netty started on port(s): 9109
2020-12-17 16:56:26.146 INFO 16262 --- [ main] a.d.m.f.my.function.Application$Main : Started Application.Main in 3.525 seconds (JVM running for 4.024)
2020-12-17 16:56:26.169 ERROR 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 : onError(io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:80)
2020-12-17 16:56:26.179 ERROR 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 :
io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:80
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
|_ checkpoint ⇢ Request to GET http://localhost/2018-06-01/runtime/invocation/next [DefaultWebClient]
Stack trace:
Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused
at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124) ~[netty-transport-native-unix-common-4.1.46.Final.jar!/:4.1.46.Final]
at io.netty.channel.unix.Socket.finishConnect(Socket.java:243) ~[netty-transport-native-unix-common-4.1.46.Final.jar!/:4.1.46.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:672) [netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:649) [netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:529) [netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:465) ~[netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.46.Final.jar!/:4.1.46.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.46.Final.jar!/:4.1.46.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.46.Final.jar!/:4.1.46.Final]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_252]
2020-12-17 16:56:26.182 INFO 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 : onSubscribe(FluxRepeatPredicate.RepeatPredicateSubscriber)
2020-12-17 16:56:26.182 INFO 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 : request(256)
2020-12-17 16:56:26.186 ERROR 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 : onError(io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:80)
2020-12-17 16:56:26.187 ERROR 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 :
io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:80
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
|_ checkpoint ⇢ Request to GET http://localhost/2018-06-01/runtime/invocation/next [DefaultWebClient]
Stack trace:
Caused by: java.net.ConnectException: finishConnect(..) failed: Connection refused
at io.netty.channel.unix.Errors.throwConnectException(Errors.java:124) ~[netty-transport-native-unix-common-4.1.46.Final.jar!/:4.1.46.Final]
at io.netty.channel.unix.Socket.finishConnect(Socket.java:243) ~[netty-transport-native-unix-common-4.1.46.Final.jar!/:4.1.46.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.doFinishConnect(AbstractEpollChannel.java:672) [netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:649) [netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:529) [netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:465) ~[netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) ~[netty-transport-native-epoll-4.1.46.Final-linux-x86_64.jar!/:4.1.46.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[netty-common-4.1.46.Final.jar!/:4.1.46.Final]
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[netty-common-4.1.46.Final.jar!/:4.1.46.Final]
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[netty-common-4.1.46.Final.jar!/:4.1.46.Final]
at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_252]
2020-12-17 16:56:26.187 INFO 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 : onSubscribe(FluxRepeatPredicate.RepeatPredicateSubscriber)
2020-12-17 16:56:26.187 INFO 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 : request(256)
2020-12-17 16:56:26.189 ERROR 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 : onError(io.netty.channel.AbstractChannel$AnnotatedConnectException: finishConnect(..) failed: Connection refused: localhost/127.0.0.1:80)
2020-12-17 16:56:26.189 ERROR 16262 --- [or-http-epoll-1] reactor.Flux.MonoRepeatPredicate.1 :
which continues throwing the same exception until the server is killed.
See this repository for an example: https://github.com/davidarmstronglewis/spring-cloud-function-issues-623
Search for a TODO to find where to comment/uncomment the @Component annotation and trigger the issue in question.
Happy to help further if I can 🙂
olegz commented
Was able to reproduce it with provided sample, thanks!
Let's see what's going on. . . .
olegz commented
This bug exists only on 3.0.x branch. I just confirmed that it works as expected in 3.1.0.RELEASE which as been released before the end of the year.
That said I am fixing it for 3.0.x anyway