This is a demonstrator of an integration of ChatGPT in MPS. The idea is that users specify their model in prose, and ChatGPT produces the source code.
Click on this picture to see a demo video you Youtube.
- This is an MPS project that needs to be opened in MPS version 2021.3.2
- Create a directory on your drive; say it's called
demo - Clone the reposistory into the
chatgptdemodirectory insidedemo - Make sure you have the mbeddr.platform and IETS3.opensource libraries installed in your MPS installation; alternatively, put these into a
libfolder underdemoand make thechatgptdemopath variable in MPS point todemo - Open the chatgptdemo project in MPS and rebuild everything for validation of the setup.
ChatGPT has a bunch of configuration parameters that you'll have to set; they can all be configured in the ChatGPTSetup class.
CHATGPT_ENDPOINT: the service endpoint. As of this writing, this can be left unchanged.CHATGPT_MODEL: the model used by ChatGPT. Currently this is set togpt-3.5-turbo. If you have access to more recent models, you might want to try changing this to get better results.
The API key is read from a file that is not committed (via gitignore):
- create a file named
API_KEY(no extension) in this directory. - You have to enter your own API key into that file. Get an API key here.
It is also a good idea to run MPS in a way where you can see the
system console (run java instead of javaw in the batch or sh file that
starts MPS). Many error messages are printed to stderr.
I have been working for years and years with subject matter experts, people who are not programmers but need to specify the core subject matter of a domain in a way that is formal and complete enough to allow direct execution by software. You can't teach them a general-purpose language, because it is too low-level and too burdened with technical concerns that are unrelated to the subject matter at hand. Instead, I follow an approach based on the Subject Matter First Manifesto, where software developers create domain-specific languages and tools that allow the subject matter experts to {program|model|specify} the subject matter formally. I have helped many companies use this approach in practice, at real-world scale. It works.
However, there has been a long standing dream by which these subject matter experts aren't required to express themselves in any formal language, domain-specific or not. Instead, why not just express what the system should do in prose. There are lots of tradeoffs here, which I describe in detail in an article that I will publish soon. The bottom line is that for the subject matter experts use case, I think letting the AI translate the prose into a DSL is better than having it translate directly into programming language source code. The code in this repository demos this approach.
The approach demonstrated here is of course technically specific to MPS. The conceptual approach is not. In fact, if a real textual DSL was used (instead of the projection technology provided by MPS) then the translation between nodes and XML (see below) would not be needed.
The code is separated into a demo application (in the lang.demo
virtual folder) and a generic, reusable set of utilities (in the
lang.chatgpt virtual folder). The demo application focuses on forms
with fields and calculated values; such forms are used, for example, as
assessments in healthcare. Here is an example:
(By default it might look less fancy; push the editor hint nicer to make it nicer.
The language also includes tests for forms that can be executed in MPS with the help of an interpreter:
For more details on the example domain see the video above. The forms language uses KernelF for expressions.
The forms language is a normal MPS DSL; it currently has a few technical dependencies on the generic ChatGPT framework, for example here or here. (The previous link and many of those below are MPS node URLs; click on them while the project is open in MPS to jump directly to the code).
- The user enters a prompt into the Swing UI that is shown at the bottom of the ChatGPTConsole.
- The system sends the following to ChatGPT:
- system messages that instruct GPT to only respond with XML code, no prose, no explanations
- a set of user messages explain to ChatGPT the XML format we use to express the forms (we use XML as the common denominator between MPS and ChatGPT) this "tutorial" is written as a text file.
- the history of the MPS-ChatGPT interactions so far as a means of emulating a stateful conversation (see below)
- the prompt entered by the user
- ChatGPT sends back a new form expressed as XML
- We parse the XML and build the MPS model The demo-language-specific config node acts as a factory for the parser. If ChatGPT makes up syntax that was not defined in the tutorial, we cannot parse it. We capture these problems in the created model with annotations and error concepts for expressions and types.
- The ChatGPTConsole does not just store the latest form; instead it stores the history (see also the video). The prompt and the textual reply is stored as an annotation on each of the constructed forms.
- Importantly, when we send back the history, we do not send back the XML that we received from ChatGPT before. Instead we serialise the MPS model back to XML. If the user did not make changes to the model after it was received from ChatGPT, this does not make a difference. But it does make a difference if the user made a change: we make it appear as if ChatGPT suggested this change, which allows ChatGPT to make subsequent changes on the basis for the model changed by the user.
- 11 July: I have extended the system to also generate tests. See this screenshot As I was suspecting, the tests are often wrong because ChatGPT can't "execute" the expressions it had generated earlier. It can syntactically predict expression structure, but not "understand" their semantics. At least not when the semantics involve numeric calculations.
- 11 July: After pusing my API key to github again, I have moved it to a file that is never committed. See Setup section above.
- 11 July: A few internal refactorings to make it easier to replace ChatGPT with other LLMs.
- 9 July: You can use a context menu action on the ChatGPT annotation to let ChatGPT explain the form.
This code is licensed under Apache 2.0. See LICENSE file.