A GitHub Action that analyses a Pull Request and adds unit tests if necessary / possible
Minimal example to put into your action YAML file:
name: Suggest tests for new functions
on: [pull_request]
jobs:
build:
name: Suggest tests for new functions
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Create patch file & store in workspace
run: git diff origin/${GITHUB_BASE_REF} origin/${GITHUB_HEAD_REF} &> ${GITHUB_WORKSPACE}/patch
- name: Ask ChatGPT for unit tests for new functions
uses: zebroc/chatgpt-create-unit-tests@master
env:
OPENAI_TOKEN: ${{ secrets.OPENAI_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
The important parts are the "with: fetch-depth: 0" option for the checkout action so that he entire history is there and providing your OpenAI token as a secret called "OPENAI_TOKEN". GITHUB_TOKEN is automatically provided by GitHub, it just needs the right permissions:
To allow the comment on PRs, you need to go to Settings --> Actions --> General --> Workflow permissions and select the option "Read and write permissions". This action will not fail if you don't do this, but no commenting will happen.
If you want more output, you can set a DEBUG environment variable, like so:
…
- name: Ask ChatGPT for unit tests for new functions
uses: zebroc/chatgpt-create-unit-tests@master
env:
OPENAI_TOKEN: ${{ secrets.OPENAI_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DEBUG: true
An example using custom prompts and a very small max patch size:
name: Code review & security review
on: [pull_request]
jobs:
build:
name: Suggest tests for new functions
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Create patch file & store in workspace
run: git diff origin/${GITHUB_BASE_REF} origin/${GITHUB_HEAD_REF} &> ${GITHUB_WORKSPACE}/patch
- name: Ask ChatGPT for a code and security review
uses: zebroc/chatgpt-create-unit-tests@master
with:
prompts: '{"code":"Do a code review for this:\n\n%s","sec":"Do a security review for this:\n\n%s"}'
maxpatchsize: '50'
env:
OPENAI_TOKEN: ${{ secrets.OPENAI_TOKEN }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
You can use inputs to provide the prompts that you want to use. They go into the "prompts" input in the form of a JSON object which is marshalled into a map[string]string, example:
{
"codereview": "Please do a code review for this patch:\n\n%s",
"security": "Are there any security issues this patch:\n\n%s"
}Review prompts must be written a little differently, as they produce a JSON output which is programmatically used and adds code reviews to the PR which makes it easier to apply suggestions from ChatGPT
{
"Code review": "Given the following patch:\\n\\n%s\\n\\nplease perform a code review and create GitHub Review comments suggesting code changes and fill each one into a JSON object like: { \"path\": \"\", \"body\": \"FILL IN SUGGESTION\\n\\\\u0060\\\\u0060\\\\u0060suggestion\\nCODE\\\\u0060\\\\u0060\\\\u0060\", \"start_side\": \"RIGHT\", \"side\": \"RIGHT\", \"start_line\": STARTING_LINE, \"line\": ENDING_LINE } and then return just those objects in an array."
}A prompt has exactly one placeholder (the %s). That is the content of the patch / diff.
Only ask ChatGPT if your patch is below this size: Serves both as a reminder to keep PRs small and also limits requests to ChatGPT that are too large anyways.