Add support for notebooks

[karthiknadig]

In notebook scenario, each cell is treated as a notebook. With events handing add, remove, move, change of each cell as notebook events. And the content changes as document events.

[alcarney]

For anyone who's interested, you can kind of use pygls today with notebooks. If you're willing to bend the rules a little.
I was able to get the example inlay_hint.py server running with zero (server) code changes, just by adding VSCode's notebook cell uri scheme to the documentSelector on the client side.

function getClientOptions(): LanguageClientOptions {
    return {
        // Specify which documents you'd like to be sent to the language server
        //
        // For a list of available schemes in VSCode see: https://github.com/microsoft/vscode/blob/main/src/vs/base/common/network.ts#L10
        documentSelector: [
            { scheme: "file" },
            { scheme: "untitled" },
            { scheme: "vscode-notebook-cell" }
        ],
        outputChannelName: "[pygls] JsonLanguageServer",
        synchronize: {
            // Notify the server about file changes to '.clientrc files contain in the workspace
            fileEvents: workspace.createFileSystemWatcher("**/.clientrc"),
        },
    };
}

However to quote the LSP spec

However since the URI of a notebook cell’s text document should be opaque, servers can not know its scheme nor its path

This obviously won't be a viable long term solution 😅 but perhaps useful if you want to start playing around with notebooks and are waiting for pygls to catch up

[karthiknadig]

This is what I use with black/autopep8 extensions, to format cells.

The limitation is when we need to see notebook as a whole document. Like when using linting, we may have to provide a merged document to the linter containing only the python cell contents. Or in cases where the linter can handle notebook itself.

[alcarney]

Trust me to spot the non-hacky way to do it immediately after! 😅 - ask for the language you're interested in and not the scheme!

        documentSelector: [
            { language: "python" },
        ],

---

Also posting this here for future me (and anyone else who's interested) to refer to.

To quote the spec

The protocol will therefore support two modes when it comes to synchronizing cell text content:

cellContent: in this mode only the cell text content is synchronized to the server using the standard textDocument/did* notification. No notebook document and no cell structure is synchronized. This mode allows for easy adoption of notebooks since servers can reuse most of it implementation logic.

notebook: in this mode the notebook document, the notebook cells and the notebook cell text content is synchronized to the server. To allow servers to create a consistent picture of a notebook document the cell text content is NOT synchronized using the standard textDocument/did* notifications. It is instead synchronized using special notebookDocument/did* notifications. This ensures that the cell and its text content arrives on the server using one open, change or close event.

It sounds like there are 2 levels of sync available - a "high level" and a "low level" api if you like.
In the "low-level" view of the world, the client pretends each cell is its own text document and syncs the content across as normal - which we can support today as confirmed by @karthiknadig above.

What's missing is the "higher level" API were we can process groups of cells as complete notebooks!