Ability to associate kernelspec with a jobdefinition
akshaychitneni opened this issue · 0 comments
Problem
Currently, a notebook job is executed using the kernelspec that is associated with the notebook file which is generally used in an interactive session. However, users may require to specifically link a kernel spec to a job definition which can be useful in a scenario when users manage kernelspecs with different properties for interactive and non-interactive usecases.
Example: Experiment using an interactive session with a kernelspec on small amount of data but run in background or pipeline with large data that require higher resources configured in the kernelspec.
Proposed Solution
Provide ability for the user to choose a kernelspec when creating a job definition
Additional context
This is a feature we are using internally in our organization and I am happy to contribute to this project