kubeflow/notebooks

Kubeflow Notebooks 2.0 (Kubeflow Workspaces)

Opened this issue · 43 comments

thesuperzapper commented

/kind feature

Here is my proposal for Kubeflow Notebooks 2.0, which can be called "Kubeflow Workspaces".

Frequently Asked Questions

What is Notebooks 2.0?

Kubeflow Notebook 2.0 is the next evolution of Kubeflow Notebooks.
It makes running IDEs on Kubernetes for ML/AI significantly better for users and cluster admins alike.

Where can I find Notebooks 2.0?

We are developing Notebooks 2.0 in kubeflow/notebooks on the notebooks-v2 branch.

Here are direct links to each component's code:

How can I contribute to Notebooks 2.0?

We welcome code contributions and ideas! The best way to get involved is participating on the #kubeflow-notebooks Slack channel, and joining our weekly notebooks community calls (currently at 8AM PT each Thursday).

If you don't see any unassigned tasks, take a look at the current state of the code and make some suggestions (by creating issues on the kubeflow/notebooks repo or starting a thread in Slack).

When will Notebooks 2.0 be ready?

We hope to ship a widely available alpha version as an optional component of Kubeflow 1.11 (ETA mid 2025).

However, we will release a testing version as soon as we have an end-to-end working platform (ETA late 2025), subscribe to this issue for updates and to help us test.

Motivation

The main idea has always been to make the Notebook CRD not just a wrapper around PodSpec, with the goal of abstracting away the Kubernetes resources from end users, while also giving cluster admins the ability to define a set of "templates" that end users can choose from.

The main benefits of this approach are:

  • end-users can create Workspaces without needing to know anything about Kubernetes, because the UI literally becomes 3 dropdowns and 2 volume mounts:
    • Choose a WorkspaceKind (e.g. "jupyter-lab", "vs-code", "rstudio")
    • Choose an image from the approved list
    • Choose a pod-config (e.g. "small_cpu", "big_gpu")
    • Create/Mount a home volume (optional)
    • Create/Mount data volumes (optional)
  • cluster-admins can update the definitions of WorkspaceKinds without breaking existing Workspaces:
    • they simply add a new image/pod-config option, and redirect the old one to the new one
    • we can even provide a config to make the controller wait for a Workspace to restart before applying the redirect
      (we can also make the Spawner UI display a warning that the Workspace needs to be restarted to get the new config)
  • scheduling Pods with GPU resources often requires a number of Pod configs to be correctly aligned (e.g. tolerating a taint, and setting a resource limit), collecting all these configs into a single "pod-config" allows cluster-admins to provide drop-down options, rather than requiring users to understand the structure of their Kubernetes cluster.
  • resizing an existing notebook is effectively impossible right now, because all the nice "spawner configs" are only available at the time you create the Notebook, once it's spawned you have a PodSpec.
    • With WorkspaceKinds, we can make an "edit" button on the UI which allows you to pick from the current "pod-configs" and "images" available for the WorkspaceKind.

Implementation

For more detailed information about the design, please see:

New Components

  • a new Workspace Controller:
    • Will manage and reconcile the new CRDs
    • Will provide webhooks (for validation of CRD patches)
    • Will be written in GoLang with Kubebuilder
  • a new Workspace Backend API
    • Will be the interface between the frontend and Kubernetes
    • Will allow programmatic management of Notebooks, and allow easier replacement of frontend.
    • Will be a REST API written in Golang
  • a new Workspace Frontend:
    • Will let users overview the workspaces in a namespace
    • Will let users spawn, edit and connect to Workspaces
    • Will be written in JS (React)

New CRDs

The high-level overview is to split the Notebook CRD into these two CRDs:

  • Workspace (namespaced resource)
    • this is the resource that end-users create via the "Workspace Spawner UI" or kubectl
    • it is NOT a wrapper around PodSpec
  • WorkspaceKind (cluster resource)
    • this is the resource that cluster-admins create
    • it specifies the template for a Workspace (e.g. "JupyterLab", "VSCode", "RStudio")
    • initially, we would only support a "podTemplate" kind, which is very similar to the existing Notebook CRD,
      but in the future, we could support other types of templates (e.g. "helmTemplate")
google-oss-prow commented

@thesuperzapper: The label(s) kind/feature cannot be applied, because the repository doesn't have them.

In response to this:

/kind feature

Here is my proposal for Kubeflow Notebooks 2.0, which can be called "Kubeflow Workspaces".

Frequently Asked Questions

What is Notebooks 2.0?

Kubeflow Notebook 2.0 is the next evolution of Kubeflow Notebooks.
It makes running IDEs on Kubernetes for ML/AI significantly better for users and cluster admins alike.

Where can I find Notebooks 2.0?

We are developing Notebooks 2.0 in kubeflow/notebooks on the notebooks-v2 branch.

Here are direct links to each component's code:

How can I contribute to Notebooks 2.0?

We welcome code contributions and ideas! See our tracking board for what is currently being worked on, look for unassigned tasks and request to be assigned them.

If you don't see any unassigned tasks, take a look at the current state of the code and make some suggestions (by creating issues on the kubeflow/notebooks repo).

When will Notebooks 2.0 be ready?

We hope to ship a widely available alpha version as an optional component of Kubeflow 1.10 (ETA early 2025).

However, we will release a testing version as soon as we have an end-to-end working platform (ETA late 2024), subscribe to this issue for updates and to help us test.

Motivation

The main idea has always been to make the Notebook CRD not just a wrapper around PodSpec, with the goal of abstracting away the Kubernetes resources from end users, while also giving cluster admins the ability to define a set of "templates" that end users can choose from.

The main benefits of this approach are:

  • end-users can create Workspaces without needing to know anything about Kubernetes, because the UI literally becomes 3 dropdowns and 2 volume mounts:
    • Choose a WorkspaceKind (e.g. "jupyter-lab", "vs-code", "rstudio")
    • Choose an image from the approved list
    • Choose a pod-config (e.g. "small_cpu", "big_gpu")
    • Create/Mount a home volume (optional)
    • Create/Mount data volumes (optional)
  • cluster-admins can update the definitions of WorkspaceKinds without breaking existing Workspaces:
  • they simply add a new image/pod-config option, and redirect the old one to the new one
  • we can even provide a config to make the controller wait for a Workspace to restart before applying the redirect
    (we can also make the Spawner UI display a warning that the Workspace needs to be restarted to get the new config)
  • scheduling Pods with GPU resources often requires a number of Pod configs to be correctly aligned (e.g. tolerating a taint, and setting a resource limit), collecting all these configs into a single "pod-config" allows cluster-admins to provide drop-down options, rather than requiring users to understand the structure of their Kubernetes cluster.
  • resizing an existing notebook is effectively impossible right now, because all the nice "spawner configs" are only available at the time you create the Notebook, once it's spawned you have a PodSpec.
    • With WorkspaceKinds, we can make an "edit" button on the UI which allows you to pick from the current "pod-configs" and "images" available for the WorkspaceKind.

Implementation

For more detailed information about the design, please see:

New Components

  • a new Workspace Controller:
    • Will manage and reconcile the new CRDs
    • Will provide webhooks (for validation of CRD patches)
    • Will be written in GoLang with Kubebuilder
  • a new Workspace Backend API
    • Will be the interface between the frontend and Kubernetes
    • Will allow programmatic management of Notebooks, and allow easier replacement of frontend.
    • Will be a REST API written in Golang
  • a new Workspace Frontend:
    • Will let users overview the workspaces in a namespace
    • Will let users spawn, edit and connect to Workspaces
    • Will be written in JS (React)

New CRDs

The high-level overview is to split the Notebook CRD into these two CRDs:

  • Workspace (namespaced resource)
  • this is the resource that end-users create via the "Workspace Spawner UI" or kubectl
  • it is NOT a wrapper around PodSpec
  • WorkspaceKind (cluster resource)
  • this is the resource that cluster-admins create
  • it specifies the template for a Workspace (e.g. "JupyterLab", "VSCode", "RStudio")
  • initially, we would only support a "podTemplate" kind, which is very similar to the existing Notebook CRD,
    but in the future, we could support other types of templates (e.g. "helmTemplate")

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

thesuperzapper commented

@kubeflow/wg-notebooks-leads I would appreciate any feedback or ideas on this proposal.

  • 👍1
thesuperzapper commented

@kimwnasptd we (that is, @juliusvonkohout, @tzstoyanov, and me) discussed this in the NB meeting today and are thinking that we can target this for Kubeflow 1.9.

I would love to get your general feedback on this proposal, and if you think it looks ok, then we can discuss some specifics and get a finalized design doc.

I think the general approach would be that we keep "kubeflow notebooks" as a "deprecated" component for 1.9, and add a new component called "kubeflow workspaces", which will have:

  1. A new controller for the Workspace and WorkspaceKind CRDS
  2. A new UI to manage spawning/editing/pausing/etc them
  3. A new "component" section on the docs website
  • 👍4
kimwnasptd commented

@thesuperzapper @juliusvonkohout @tzstoyanov I'm good with the way of implementing this as a new CRD, and web app since this will be the least intrusive implementation. Users will be able to pick and choose which one they'll want, until we are fully confident on using Workspaces by default.

What I'd like to confirm before committing to this effort would be what are the criteria for for considering this as the default?
* Have culling?
* Have a web app that allows us to have the same functionality as the current one?

  • 👍1
kabartay commented

Thanks for this new changes. Looks good.

thesuperzapper commented

Calling anyone who wants to help move this feature forward!

Please reply if you are willing/able to help with any of the following areas:

  • Frontend (JS - Angular) (Python - Flask):
    • We will need to add 2 new crud-web-apps, to manage and view the new resources.
      • The Workspace app will behave similarly to the existing Notebooks one, and will mostly be about spawning and managing the state of Workspace instances in each profile namespace.
      • The WorkspaceKind app can start with the goal of only showing the current WorkspaceKinds registered in the cluster, and in the future can allow admins to change them graphically.
  • Backend (Golang - Kubebuilder) (Istio):
    • We will need to build a new controller for the Workspace and WorkspaceKind resources.
      • We should use Kubebuilder like we already do in the notebook-controller
      • The Workspace controller will also need to create the Istio VirtualServices/etc to enable access to the notebook web interfaces.
    • We will also probably need to create Webhooks to achieve some of what we are doing, which can also be created using Kubebuilder.
  • Docs (Markdown and Images):
    • Docs are just as important as the code! Otherwise, people won't know about the awesome new features!
    • We will need to add a new "component" section of the Kubeflow website for "Kubeflow Workspace".

Once we have a few people willing to help, we can organize a specific call to flesh out a doc with:

  • User Stories
  • Controller Architecture/Behaviour
  • UI Mockups
  • Alternatives / Problems

@mishraprafful and @guohaoyu110 mentioned that they might be interested in contributing to this, also, there are some others who might be like @punkerpunker @helloericsf @wjhhuizi @wadhah101

  • 👍2
guohaoyu110 commented
  • Backend (Golang - Kubebuilder) (Istio)

I am interested in this part: Backend (Golang - Kubebuilder) (Istio)

thesuperzapper commented

Also, @vvnpan @apo-ger may be interested in helping this effort, given they have raised large PRs on the old Notebooks.

mishraprafful commented

I am interested to take up any of these following these areas:

  • Frontend (Python - Flask)
  • Backend (Golang - Kubebuilder) (Istio)
  • Docs (Markdown and Images)
SachinVarghese commented

I'm happy to collaborate on creating the backend+docs (workspace controller) for this new proposal

vikas-saxena02 commented

I am happy to contribute to the documentation if you guys need help with that.

thesuperzapper commented

Hey all, now that Kubeflow 1.8 is out, we are going to start seriously working on Kubeflow Notebooks 2.0!

The first step will be to organize and regular meetings with the people who want to contribute and/or give feedback.

I think the first few meetings will be to finalize the design and requirements. Then next year, we can split everything down into tasks and assign them to specific people, so the meeting just becomes a check-in.

Join the next community call (12 December 2023 @ 8:00am PT) so we can organize the first few meetings.

Once we have organized the meeting we'll send it out on the Kubeflow-discuss mailing list.

Also please join the Kubeflow slack and the #kubeflow-notebooks channel which we can use for discussing ideas.

Also just some notes so that I don't forget:

  • we need to discuss configurable service account bindings for the pod (rather than just giving default-editor to all notebooks), at very least we should allow the admin to set all of notebooks in a specific profile to viewer only
  • we need to discuss if we want to have more granular assignments of access to specific notebooks for specific users (rather than just everyone in a profile can connect to all of the notebooks)
thesuperzapper commented

For those wanting to help with Kubeflow Notebooks 2.0, please see the previous comment:

Please try and get to the next kubeflow community call, so we can organize a meeting time.

Also please join the kubeflow slack (especially if you can't attend the community call), and make yourself known on the #kubeflow-notebooks channel, so we can find a good time that works for the "Notebooks 2.0 catchup" meetings).

@SachinVarghese
@Talador12
@apo-ger
@guohaoyu110
@helloericsf
@juliusvonkohout
@kimwnasptd
@mishraprafful
@punkerpunker
@tzstoyanov
@vikas-saxena02
@vvnpan
@wadhah101
@wjhhuizi

  • 👍1
vikas-saxena02 commented

@thesuperzapper I am more than happy to be part of the initiative. I will try my best to attend the community call tonight.

StefanoFioravanzo commented

@thesuperzapper I have a high-level observation on the terminology: why did you choose the name "Workspace"? I think the proposed rename to "Workbench" was well received and agreed upon here:

The term "workspace" is generally used to refer to a project or a logical space that contains resources and user data. In Kubeflow, I believe the closest thing to a workspace we have is the namespace.

  • 👍8
umka1332 commented

I find this proposal very bad, since you are removing options that user have before. Now they will be limited to configurations that were created by admins instead of having the ability to create the thing that they need.
It's like adding complexity and removing flexibility at the same time.
It would be much better to have some presets, from which user can choose, but still be able to specify all needed values by themselves.

  • 👀1
thesuperzapper commented

Hey all,

Please attend the Notebooks Working Group Meeting (Thursday, 18 January, 2024 @ 8:00am PT) if you want to help with Kubeflow Notebooks 2.0!

This will be the first meeting, and our goal will be to start an architecture doc and discuss ideas.

I think the overall steps to get Notebooks 2.0 shipped are:

  • Step 1: organize a "Kubeflow Notebooks 2.0" catchup meeting.
  • Step 2: finalize the design (by discussing requirements and implementation)
  • Step 3: split the design up into tasks, and allocate people
  • Step 4: continue regular catch-ups (to monitor progress)
  • Step 5: ship as part of Kubeflow 1.9

@SachinVarghese
@Talador12
@apo-ger
@guohaoyu110
@helloericsf
@juliusvonkohout
@kimwnasptd
@mishraprafful
@punkerpunker
@tzstoyanov
@vikas-saxena02
@vvnpan
@wadhah101
@wjhhuizi
etc.

  • 👍1
thesuperzapper commented

Hey all,

We have started writing up the design document for Kubeflow Notebooks 2.0.
Please review it and feel free to make comments/suggestions.

The next meeting is Thursday 25th January @ 8 AM PT / Thursday 25th January @ 4 PM PT
Where we will continue finalizing the document and to start assigning tasks to contributors.

PS: we will use the Notebooks WG meeting, to get the invite, either join kubeflow-discuss (if you use Google Calendar) or manually add the "Kubeflow Community Calendar" to yours

juliusvonkohout commented

@thesuperzapper Thursday 25th January @ 8 AM PT is the manifests wg meeting. are you sure you do not mean Febuary 1 ?

vikas-saxena02 commented
thesuperzapper commented

@juliusvonkohout I did not see the Manifests WG meeting at that time!

Currently, the Notebooks WG Meeting only runs every two weeks, we need to make it weekly while we work on Kubeflow Notebooks 2.0.

I think we can add a new bi-weekly 4:00PM PT meeting starting next week, which would be aimed at US + APAC attendees (but impossible for EU attendees). It gives everyone a chance to attend and does not overlap with the existing Manifests WG meeting.

I have made a PR that will update the community calendar to reflect this:

thesuperzapper commented

@vikas-saxena02 that would mean the next meeting is on an Indian/Australian holiday, but I think it's fine if fewer people attend for that specific meeting, we need to keep the momentum up for the Notebooks 2.0 planning.

thesuperzapper commented

So, for anyone who wants to attend next week, expect the meeting next week to be on:

Thursday 25th Janurary @ 4:00 PM PT.

Either way, everyone please comment/contribute to the design document whenever you have time:

  • 👍2
  • 🚀1
WYGIN commented

@thesuperzapper I am willing to contribute to this feature out side the GSoC, willing to know if it is fine. Thank You.

thesuperzapper commented

@WYGIN what areas do you have experience in that could help us?

WYGIN commented

@WYGIN what areas do you have experience in that could help us?

around 9 months of experience on containerization and GoLang and more than 2 years learning curve on js(react, mainly building backend services) and not quite familiar with python(but always wanted to get in).

This feature strongly aligns with my interests and might be the best PR to work around to explore kubeflow and to challenge myself, Happy to hand it over to GSoC mentee at anytime if needed. Thank You.

You can also point me to any core feature(hopefully large/challenging issue and focuses on one of the core features on kubeflow) for me to work around.
cc: @thesuperzapper

rockyzhou-zeyu commented

@thesuperzapper I am a cs undergrad from CMU. I am really interested in Kubeflow Notebooks 2.0. I have experiences of Golang Development and good amount of understanding of Kubernetes at School. The design doc of Kubeflow Notebooks 2.0 is just inspiring to me. I am willing to contribute for the Backend as a GSoC contributor, specifically in Workspace Controller and Validation Webhooks parts.

thesuperzapper commented

For those watching, we are nearly ready to start commiting code for Notebooks 2.0!

We have mostly finalized the new CRDs (see the new example repo for reference):

We have also made good progress on the design document:

We only need to finalize the UI mock ups for the new spawner, which we plan to share shortly.

Once we have a final design, we can start by creating folders in the repo for the new controller and front-end (with only mock code and CI/CD build pipelines), after that we can start making PRs to build out each component!

  • 🎉1
  • 🚀1
thesuperzapper commented

For those watching, we are nearly ready to start commiting code for Notebooks 2.0!

We have mostly finalized the new CRDs (see the new example repo for reference):

We have also made good progress on the design document:

We only need to finalize the UI mock ups for the new spawner, which we plan to share shortly.

Once we have a final design, we can start by creating folders in the repo for the new controller and front-end (with only mock code and CI/CD build pipelines), after that we can start making PRs to build out each component!

  • 🎉2
  • 🚀1
WYGIN commented

For those watching, we are nearly ready to start committing code for Notebooks 2.0!

We have mostly finalized the new CRDs (see the new example repo for reference):

@thesuperzapper Have you finalized whether to use GoLang or Python? Also, have you decided between using React or Angular?

thesuperzapper commented

@thesuperzapper Have you finalized whether to use GoLang or Python? Also, have you decided between using React or Angular?

@WYGIN, for the front end, we have a number of people more familiar with React who are planning to contribute, so unless that changes, we probably will go with React.

It's still not clear which is going to be better (Go or Python) for the back-end of the front-end (not the controller, which will be in Go + Kubebuilder). .

What we choose will largely depend on who's available to work on it. However, because it's much easier to interact with Kubernetes via Go, and Go is a lot more scalable, I do have a slight preference for Go, if both options have contributors available.

  • ❤️2
juliusvonkohout commented

@thesuperzapper

i saw https://github.com/thesuperzapper/kubeflow-notebooks-v2-design/blob/667275bdbf62e6f8a3af73ea302724f8430cad22/crds/workspace-kind.yaml#L83

but i think as discussed in the meetings and documents something additional like maxSeconds: 2*86400 or so is useful in a lot of enterprise environments where we want to prevent Workspaces being abused for long-running jobs and terminate them independent of the activity after 48 hours or so.

thesuperzapper commented

Just tagging @apo-ger based on the discussion in kubeflow/kubeflow#6927 (comment), we might consider adding a third activity probe based on "istio metrics" (e.g. the last time the Pod got an HTTP request).

Note, this will not be in the initial release of Notebooks 2.0 (unless you want to contribute it @apo-ger), but might be in a future one, once we agree on the implementation details.

  • 👍2
krishnakaushik195 commented

Hi, this is kaushik
there is a small idea from my side for the future development
how about adding the integrated LLM or any multimodel like AI

  • 👍1
krishnakaushik195 commented

hlo team
i want to contribute to the flask + react and LLM
so can some one say how do i start the contributing to contribute

thesuperzapper commented

@krishnakaushik195 we are currently developing Notebooks 2.0 in the notebooks-v2 branch of the kubeflow/notebooks repo.

Please be aware, it's NOT ready for usage yet (we are still actively developing it). If you want to help, you should attend the Notebooks Working Group (WG) meetings know what we are actively working on:

Get invited to the Notebooks WG meeting (on Google Calander) by joining the kubeflow-discuss Google Group. (NOTE: this will also invite you to many other community events, which you can decline or attend)

krishnakaushik195 commented

Yeah sir got it
Thank you

andreyvelich commented

/transfer notebooks

shalberd commented

Hi all, is there a way in the design currently in the new kind: WorkSpaceKind or whatever to define common labels for pods centrally? podSpec.metadata.labels .... added for all pods ever spawned? See my request at kubeflow/community#800

thesuperzapper commented

Hi all, is there a way in the design currently in the new kind: WorkSpaceKind or whatever to define common labels for pods centrally? podSpec.metadata.labels .... added for all pods ever spawned? See my request at kubeflow/community#800

@shalberd yes, it's possible to define common labels/annotations for all pods of a WorkspaceKind in Notebooks 2.0.

This is achieved by setting the following fields:

apiVersion: kubeflow.org/v1beta1
kind: WorkspaceKind
metadata:
  name: jupyterlab
spec:
  ...

  podTemplate:

    ## metadata for Workspace Pods (MUTABLE)
    ##
    podMetadata:
      labels:
        my-workspace-kind-label: "my-value"
      annotations:
        my-workspace-kind-annotation: "my-value"

  ...
github-actions commented

This issue has been automatically marked as stale because it has not had recent activity.
It will be closed if no further activity occurs.
Thank you for your contributions.

Members may comment /lifecycle frozen to prevent this issue from being marked as stale.

thesuperzapper commented

/lifecycle frozen