[Bug] exchangis执行数据交换成功,但结束任务时报错

Question

[Bug] exchangis执行数据交换成功,但结束任务时报错

Closed this issue a year ago · 5 comments

coombe commented 2 years ago

Search before asking

I searched the issues and found no similar issues.

Exchangis Component

exchangis-plugins

What happened + What you expected to happen

sqoop 引擎报如下错误:

2023-02-10 10:23:34.252 INFO [SpringContextShutdownHook] org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService 85 executorShutDownHook - executorShutDownHook start to execute.
2023-02-10 10:23:34.252 WARN [SpringContextShutdownHook] org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService 91 executorShutDownHook - had stop, do not shutdown
2023-02-10 10:23:34.262 INFO [SpringContextShutdownHook] org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService 85 executorShutDownHook - executorShutDownHook start to execute.
2023-02-10 10:23:34.262 WARN [SpringContextShutdownHook] org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService 91 executorShutDownHook - had stop, do not shutdown
2023-02-10 10:23:34.263 INFO [SpringContextShutdownHook] com.netflix.util.concurrent.ShutdownEnabledTimer 67 cancel - Shutdown hook removed for: NFLoadBalancer-PingTimer-linkis-cg-engineconnmanager
2023-02-10 10:23:34.263 INFO [SpringContextShutdownHook] com.netflix.util.concurrent.ShutdownEnabledTimer 72 cancel - Exception caught (might be ok if at shutdown) java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) ~[?:1.8.0_311]
at java.lang.Runtime.removeShutdownHook(Runtime.java:239) ~[?:1.8.0_311]
at com.netflix.util.concurrent.ShutdownEnabledTimer.cancel(ShutdownEnabledTimer.java:70) ~[netflix-commons-util-0.3.0.jar:0.3.0]
at com.netflix.loadbalancer.BaseLoadBalancer.cancelPingTask(BaseLoadBalancer.java:632) ~[ribbon-loadbalancer-2.3.0.jar:2.3.0]
at com.netflix.loadbalancer.BaseLoadBalancer.shutdown(BaseLoadBalancer.java:883) ~[ribbon-loadbalancer-2.3.0.jar:2.3.0]
at com.netflix.loadbalancer.DynamicServerListLoadBalancer.shutdown(DynamicServerListLoadBalancer.java:285) ~[ribbon-loadbalancer-2.3.0.jar:2.3.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_311]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_311]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_311]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_311]
at org.springframework.beans.factory.support.DisposableBeanAdapter.invokeCustomDestroyMethod(DisposableBeanAdapter.java:286) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:220) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:587) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:559) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1092) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:520) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1085) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1061) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1030) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:979) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.cloud.context.named.NamedContextFactory.destroy(NamedContextFactory.java:93) ~[spring-cloud-context-2.2.9.RELEASE.jar:2.2.9.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:205) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:587) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:559) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1092) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:520) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1085) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1061) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1030) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.doClose(ServletWebServerApplicationContext.java:170) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:949) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]

2023-02-10 10:23:34.271 INFO [SpringContextShutdownHook] org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService 85 executorShutDownHook - executorShutDownHook start to execute.
2023-02-10 10:23:34.271 WARN [SpringContextShutdownHook] org.apache.linkis.engineconn.acessible.executor.service.DefaultAccessibleService 91 executorShutDownHook - had stop, do not shutdown
2023-02-10 10:23:34.272 INFO [SpringContextShutdownHook] com.netflix.util.concurrent.ShutdownEnabledTimer 67 cancel - Shutdown hook removed for: NFLoadBalancer-PingTimer-linkis-cg-linkismanager
2023-02-10 10:23:34.272 INFO [SpringContextShutdownHook] com.netflix.util.concurrent.ShutdownEnabledTimer 72 cancel - Exception caught (might be ok if at shutdown) java.lang.IllegalStateException: Shutdown in progress
at java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:82) ~[?:1.8.0_311]
at java.lang.Runtime.removeShutdownHook(Runtime.java:239) ~[?:1.8.0_311]
at com.netflix.util.concurrent.ShutdownEnabledTimer.cancel(ShutdownEnabledTimer.java:70) ~[netflix-commons-util-0.3.0.jar:0.3.0]
at com.netflix.loadbalancer.BaseLoadBalancer.cancelPingTask(BaseLoadBalancer.java:632) ~[ribbon-loadbalancer-2.3.0.jar:2.3.0]
at com.netflix.loadbalancer.BaseLoadBalancer.shutdown(BaseLoadBalancer.java:883) ~[ribbon-loadbalancer-2.3.0.jar:2.3.0]
at com.netflix.loadbalancer.DynamicServerListLoadBalancer.shutdown(DynamicServerListLoadBalancer.java:285) ~[ribbon-loadbalancer-2.3.0.jar:2.3.0]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_311]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_311]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_311]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_311]
at org.springframework.beans.factory.support.DisposableBeanAdapter.invokeCustomDestroyMethod(DisposableBeanAdapter.java:286) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:220) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:587) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:559) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1092) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:520) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1085) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1061) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1030) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:979) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.cloud.context.named.NamedContextFactory.destroy(NamedContextFactory.java:93) ~[spring-cloud-context-2.2.9.RELEASE.jar:2.2.9.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:205) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:587) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:559) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:1092) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:520) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:1085) ~[spring-beans-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:1061) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:1030) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.doClose(ServletWebServerApplicationContext.java:170) ~[spring-boot-2.3.12.RELEASE.jar:2.3.12.RELEASE]
at org.springframework.context.support.AbstractApplicationContext$1.run(AbstractApplicationContext.java:949) ~[spring-context-5.2.22.RELEASE.jar:5.2.22.RELEASE]

2023-02-10 10:23:37.277 WARN [DiscoveryClient-0] com.netflix.discovery.TimedSupervisorTask 90 run - task supervisor shutting down, can't accept the task

Relevent platform

exchangis-1.1.2-webank/dss-1.1.1/linkis-1.3.1/sqoop-1.4.6/centos7.9

Reproduction script

无

Anything else

No response

Are you willing to submit a PR?

Yes I am willing to submit a PR!

Answer 1 · 2023-02-11T07:02:26.000Z

看看字段映射里头是不是有字段类型不匹配了，可以只保留一个string字段试试看

Answer 2 · 2023-02-15T01:15:44.000Z

看看字段映射里头是不是有字段类型不匹配了，可以只保留一个string字段试试看

感谢您。应该不是这个问题，我看了一下，我这4个字段，都是字符串类型的，字段名称完全相同。而且，数据已经传输到目标表里了。我看日志是关闭任务时失败了，感觉应该是linkis升级之后，exchangis的某些地方需要适配。

Answer 3 · 2023-03-16T13:48:11.000Z

同样的问题，exchangis dev-1.1.1, linkis 1.3.1

Answer 4 · 2023-05-17T00:36:13.000Z

同样的问题，数据同步成功，返回状态失败，在sqoop引擎日志的最后有两个报错

Answer 5 · 2023-08-14T12:08:22.000Z

It had been resolved by pr #408 . The latest version is 1.1.2, the latest dss version is 1.1.2, and the latest linkis version is 1.4.0