minqi/cma_mae

A python implementation of Covariance Matrix Adaptation MAP-Annealing

PythonMIT

Covariance Matrix Adaptation MAP-Annealing

This repository contains code for the experiments of the Covariance Matrix Adaptation MAP-Annealing (CMA-MAE) paper.

The project contains a modified version of pyribs a quality diversity optimization library from the Differentiable Quality Diversity paper (github). We implement the CMA-MAE and CMA-MAEGA algorithms in pyribs. The AnnealingEmitter (see ribs/emitters/_annealing_emitter.py) implements the CMA-MAE algorithm and the GradientAnnealingEmitter (see ribs/emitters/_gradient_annealing_emitter.py) implements the CMA-MAEGA algorithm. We modify ArchiveBase (see ribs/archives/_archive_base.py) to implement acceptance thresholds needed by both algorithms.

Requirements

The project builds in Anaconda.

Once installed, create the conda environment with the following command:

conda env create -f experiments/environment.yml

Next activate the conda environment and install pyribs:

conda activate cma_mae_exps
pip3 install -e .[all]

Pretrained Models

To run LSI experiments, you must first download the StyleGAN pretrained models from the StyleGAN repo. Place the .pt file in the folder experiments/lsi_clip.

CLIP automatically installs with the conda environment.

Running Experiments

For each experiment you pick an identifier for the algorithm you want to run.

Quality Diversity Algorithm	Identifier
MAP-Elites	map_elites
MAP-Elites (line)	map_elites_line
CMA-ME	cma_me
CMA-ME*	cma_me_star
CMA-MAE	cma_mae
CMA-MEGA	cma_mega
CMA-MAEGA	cma_maega

Linear Projection (sphere)

To run an experiment with MAP-Elites:

conda activate cma_mae_exps
cd experiments/lin_proj

python3 lin_proj.py map_elites --objective sphere

To run a different algorithm replace map_elites with another identifier from the above table.

For additional options see:

python3 lin_proj.py --help

Linear Projection (Rastrigin)

To run an experiment with MAP-Elites:

conda activate cma_mae_exps
cd experiments/lin_proj

python3 lin_proj.py map_elites --objective Rastrigin

To run a different algorithm replace map_elites with another identifier from the above table.

For additional options see:

python3 lin_proj.py --help

Linear Projection (plateau)

To run an experiment with MAP-Elites:

conda activate cma_mae_exps
cd experiments/lin_proj

python3 lin_proj.py map_elites --objective plateau

To run a different algorithm replace map_elites with another identifier from the above table.

For additional options see:

python3 lin_proj.py --help

Arm Repertoire

To run an experiment with MAP-Elites:

conda activate cma_mae_exps
cd experiments/arm

python3 arm.py map_elites

To run a different algorithm replace map_elites with another identifier from the above table.

For additional options see:

python3 arm.py --help

Latent Space Illumination (LSI)

To run an experiment with MAP-Elites:

conda activate cma_mae_exps
cd experiments/lsi_clip

python3 lsi.py map_elites

To run a different algorithm replace map_elites with another identifier from the above table.

For additional options see:

python3 lsi.py --help

Results

The following tables contain the reported results from the paper and commands to run each experiment.

Linear Projection (sphere)

Quality Diversity Algorithms	QD-score	Coverage	Experiment Command
MAP-Elites	41.64	50.80%	`python3 lin_proj.py map_elites --objective sphere`
MAP-Elites (line)	49.07	60.42%	`python3 lin_proj.py map_elites_line --objective sphere`
CMA-ME	36.50	42.82%	`python3 lin_proj.py cma_me --objective sphere`
CMA-MAE	64.86	83.31%	`python3 lin_proj.py cma_mae --alpha 0.01 --objective sphere`

Linear Projection (Rastrigin)

Quality Diversity Algorithms	QD-score	Coverage	Experiment Command
MAP-Elites	31.43	47.88%	`python3 lin_proj.py map_elites --objective Rastrigin`
MAP-Elites (line)	38.29	56.51%	`python3 lin_proj.py map_elites_line --objective Rastrigin`
CMA-ME	38.02	53.09%	`python3 lin_proj.py cma_me --objective Rastrigin`
CMA-MAE	52.65	80.46%	`python3 lin_proj.py cma_mae --alpha 0.01 --objective Rastrigin`

Linear Projection (plateau)

Quality Diversity Algorithms	QD-score	Coverage	Experiment Command
MAP-Elites	47.07	47.07%	`python3 lin_proj.py map_elites --objective plateau`
MAP-Elites (line)	52.20	52.20%	`python3 lin_proj.py map_elites_line --objective plateau`
CMA-ME	34.54	34.54%	`python3 lin_proj.py cma_me --objective plateau`
CMA-MAE	79.27	79.29%	`python3 lin_proj.py cma_mae --alpha 0.01 --objective plateau`

Arm Repertoire

Quality Diversity Algorithms	QD-score	Coverage	Experiment Command
MAP-Elites	71.40	74.09%	`python3 arm.py map_elites`
MAP-Elites (line)	74.55	75.61%	`python3 arm.py map_elites_line`
CMA-ME	75.82	75.89%	`python3 arm.py cma_me`
CMA-MAE	79.03	79.24%	`python3 arm.py cma_mae --alpha 0.01`

Latent Space Illumination (LSI)

Quality Diversity Algorithms	QD-score	Coverage	Experiment Command
MAP-Elites	12.85	19.42%	`python3 lsi.py map_elites`
MAP-Elites (line)	14.40	21.11%	`python3 lsi.py map_elites_line`
CMA-ME	14.00	19.57%	`python3 lsi.py cma_me`
CMA-MAE	17.67	25.08%	`python3 lsi.py cma_mae --alpha 0.1`

See the paper and supplementary materials for full data and standard error bars.

License

pyribs and this project are both released under the MIT License.

pyribs MIT License