/copilot-explorer

Hacky repo to see what the Copilot extension sends to the server

Primary LanguageJavaScript

Copilot-Explorer

This is a tool meant for exploring the codebase of Github Copilot (the client side, not the model itself).

It's work in progress. See the Journal for details on what's been done so far and my observations.

There's a blog post that explains the high level findings, and provides necessary code pointers.

Screenshots

Screenshot 1

Just show me the tool

You can access a version of the tool here.

This explores version 1.57.7193 of the VSCode extension of Copilot.

The webpage shows 3 panels:

  • Left most shows the module-level dependency graph of the extension. The edges are based on imports.
  • Middle panel shows the prettified code of the selected module. By default the main module is selected.
  • Right panel shows some information about the model (specifically, the module id, exports, imports and modules that import this module).

Module Names: I've also named modules -- some of them manually, while most of them were automatically named using codex in a few-shot manner. Some names are not unique, and I've not dealt with that. IDs of every module are unique though.

Module Categories: I've similarly also categorized modules automatically (again using codex and a bit of manual labeling). The categories are not perfect, but they serve as a very good guideline for segregating modules.

Manual annotations: Some of the modules were quite interesting, so I manually annotated them (added comments and renamed variables) to make them easier to understand. These are now directly viewable in the middle panel. Links below.

If you want to start browsing, below are some interesting starting points.

Navigation tips:

  • You can use the forward and back buttons to navigate through the history of modules you've visited.
  • Within a module, use your regular VSCode shortcuts (e.g., ctrl+click will work for definitions within the same file). They don't work across modules. But I've renamed the modules to make them easier to navigate. So, e.g., in module A, if you see M_prompt_extractor.getPrompt(), then you know to look for prompt-extractor module in the right panel.
  • The modules have been color coded so you can choose to ignore the modules you don't care about.

Interesting modules

Some interesting modules I've found so far (these notes were written before I wrote the blog post, so some comments might be outdated):

How to run locally

If you want to play with the code (different transformations, different visualizations, etc.), you can run the tool locally and modify the code.

  1. Clone the repo
  2. Run npm install
  3. Run node index.js -- this processes the copilot extension code and produces modules by doing some automatic reverse engineering.

Steps 2 and 3 are optional. They're only needed if you modify the extension code or the transformation code. The repo already contains the processed modules.

  1. Install manifest using pip install manifest-ml.
  2. Create a file named codeviz/.openai-api-key storing your OpenAI API key. You can keep this empty if you're not going to predict names and categories, or going to use the select-snippet -> right-click -> Describe this snippet feature (which doesn't work that well anyway).
  3. Run python3 codeviz/app.py to start the app.
  4. Head to localhost:5000 in your browser.

Manual annotations

The tool supports 3 types of annotations: (a) module names (b) module categories (c) module code.

You can change the first two annotations (rename/recategorize modules) by the UI. These annotations persist in your localstorage. If you want to export them, you can use browser console to do so. I know this isn't ideal, but hey this is a hacky repo. I doubt anyone will reannotate stuff seriously.

For module code -- if you find a particular module interesting and want to add your comments, you can do that and save the annotated module under codeviz/data/manually_annotated_modules/<module_id>.js. Finally, run codeviz/compress_annotated_codes.js which takes all the annotated modules and puts them in a single file that's loaded in the browser...so that your annotations show up in the UI.

Support and Warranty

lmao

But seriously, this is all on a best-effort basis, I might've misunderstood a few things and so the conclusions might be slightly off. If you find something, do raise an issue or a PR.