apache/seatunnel

[Bug] [k8s_Deploy] The client submits tasks and the home directory of the cluster must be consistent？？

Opened this issue · 0 comments

reformdai commented

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

客户端提交任务到集群，报错 source 的connector 的jar 包 class not found
When the client submits a task to the cluster, an error message is displayed: 【com.hazelcast.nio.serialization.HazelcastSerializatcionException: java.lang.ClassNotFoundException: orgg.apache.seatunnel.connectors.seatunnel.feishu.source.Feishusource】

image

由于启动的 client 的 SEATUNNEL_HOME 目录是本地的，然后本地启动cluster，提交成功！
Since the SEATUNNEL_HOME directory of the started client is local, and then the cluster is started locally, the submission is successful!

最后发现原因是client提交的任务，SEATUNNEL_HOME 是需要与 cluster 中的保持一致。
Finally, I found that the reason was the task submitted by the client. SEATUNNEL_HOME needs to be consistent with that in the cluster.

除了使用 REST 提交，有没有任何可以解决的办法呢？
Is there any solution besides using REST submission?

SeaTunnel Version

2.3.8

SeaTunnel Config

env {
  job.mode = "BATCH"
  parallelism = 1
}

source {
  Feishu {
    url = "https://open.feishu.cn/open-apis"
    appId = "xxxxx"
    appSecret = "xxxxx"
    nodeToken = "xxxxx"
    tableId = "xxxxx"
    format = "json"
    pageing = {
      page_field = "page_token"
      batch_size = "200"
    }
}

transform {
}

sink {
  Console {
  }
}

Running Command

ClientJobExecutionEnvironment jobExecutionEnv = seaTunnelClient.createExecutionContext(filePath, jobConfig, seaTunnelConfig);
ClientJobProxy clientJobProxy = jobExecutionEnv.execute();

Error Exception

com.hazelcast.nio.serialization.HazelcastSerializatcionException: java.lang.ClassNotFoundException: orgg.apache.seatunnel.connectors.seatunnel.feishu.source.Feishusource

Zeta or Flink or Spark Version

Zeta

Java or Scala Version

Java

Screenshots

image

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct