Automatic Bi-modal Question Title Generation for Stack Overflow with Prompt Learning

Paper

📜 Automatic Bi-modal Question Title Generation for Stack Overflow with Prompt Learning

@article{yang2024automatic,
  title={Automatic bi-modal question title generation for Stack Overflow with prompt learning},
  author={Yang, Shaoyu and Chen, Xiang and Liu, Ke and Yang, Guang and Yu, Chi},
  journal={Empirical Software Engineering},
  volume={29},
  number={3},
  pages={63},
  year={2024},
  publisher={Springer}
}

Introduction

The framework of our proposed approach SOTitle+

Corpus & Model

We publish our dataset and trained model on Zenodo.

Experimental Replication Tutorials

In ./model_model, We shared the script to replicate the experimental data in our paper

Replication step

Clone the repo

git clone https://github.com/shaoyuyoung/SOTitlePlus.git

Mkdir a data catalogue in the root directory and download the datasets in data catalogue.
Make sure your version of python is python3.9 (Due to compatibility issue with the nlg-eval library, we do not support python3.10 or later)
Install the dependencies according to the requirements file
```
pip install requirements.txt
```
Training and evaluating the model (fine-tuning and prompt-tuning in this phase)
```
python model_code/main.py 
```
Calculating the metrics
```
python results/metrics.py
```
If you have any questions on replication, please feel free to report in the issue🤗

Results

In ./results, run metrics.py to calculate ROUGE, METEOR, BLEU and CIDEr

SOQTG of ChatGPT

If you want to use ChatGPT to generate Stack Overflow question titles, we share scripts in ./run_ChatGPT_API.

You need to put your APIKEY and design your own prompt. We keep the original prompts from our experiment in the script.

Tool and Demo

We developed a browser plugin based on SOTitle and integrated it into the Chrome browser. Instruction for use:

Download and install the plugin from the SOTitlePlusPlugin folder.
Enter this website: https://stackoverflow.com/questions/ask
After you provide the problem description and code snippet, press Ctrl + Q to generate candidate titles.

We provide a demo video on youtube:https://www.youtube.com/watch?v=_KgUISAT74M