Large Language Model (LLM) for Multiobjective Evolutionary Algorithm (MOEA)
This Rep consists the implementation of:
- MOEA/D-LLM: LLM (GPT) serves as a blackbox search operator for decomposition-based MOEA (MOEA/D)
- MOEA/D-LO: learning from the LLM behavior, we further design an explicit white-box operator with randomness and propose a new version of decomposition-based MOEA, termed MOEA/D-LO.
Below is an illustration of the general framework of integrating LLM in MOEA/D, for more information and results please refer to manuscript LLM4MOEA
If you find the code helpful, please cite : Fei Liu, Xi Lin, Zhenkun Wang, Shunyu Yao, Xialiang Tong, Mingxuan Yuan, and Qingfu Zhang. "Large language model for multi-objective evolutionary optimization." arXiv preprint arXiv:2310.12541 (2023).
If you are interested in LLM for algorithm design, we recommend:
- A Platform for LLM for using automatic algorithm design Evolution of Heuristics
- A collection on references and research papers on LLM for optimization (LLM4Opt)
MOEA/D-LLM
Implemented in Pymoo
Set your LLM Endpoint, key, and model before start !
cd MOEAD-LLM
python run.py
MOEA/D-LO
Implemented in PlatEMO
copy [MOEAD-LO.m] to fold "Algorithms/Multi-objective optimization/MOEA-D-LO10" in PlatEMO
copy [OperatorLO.m] to fold "Algorithms/Utility functions" in PlatEMO
Test it using PlatEMO GUI or terminal command
API for LLM
In our implementation, we use API2D API to request response from GPT3.5
Step 1: Create your key from API2D
Step 2: Copy your own API2D API Key to run.py. No additional settings are required.
You can also
-
Use official API from OpenAI
-
Or local open-source LLM model, such as Llama2, Gemma
cd LLM2LO
python LinearReg.py
python PolyReg.py
where two regressions are in two levels
- LinearReg.py performs linear regression on samples (X to Y) from data.txt. 10 weights are produced
- PolyReg.py performs polynomial regression on weight (Weight to Rank). Weights for poly regression are produced
The resulted regression of weight vs. rank should be as follows (slightly different from the results in manuscript with different random seeds):
