[Feature] Spark3问题和Spark2集成建议
954031894 opened this issue · 3 comments
Search before asking
- I had searched in the feature and found no similar feature requirement.
Description
在使用1.2.1版本的过程中通过datasophon部署安装spark3中没有master角色，看了一下安装Spark3指南，说没有master角色，感觉应该是Spark3没有适配完成吗？
希望集成一个Spark2版本，因为安装的很多组件对spark的要求还是2.x
当然，我自己也会尝试一下我的这些建议
Usage Scenario
No response
Related issues
No response
Are you willing to submit a PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct
The onYarn mode should be used here. In theory, this is just a client, but the role of historyserver should be missing currently.
If the on Yarn mode is indeed like this, but when I deploy Spark3, even the workers cannot deploy it. There is no such option, only the client. I don't know if it's a version issue on my end or what caused it
I understand the reason now. In On Yarn mode, there is no need for a master or work, only the client needs to submit tasks. The past deployment methods have formed this mindset in me. Sorry to bother you with this issue.