Value Kaleidoscope

This repository is a companion for Value Kaleidoscope: Engaging AI with Pluralistic Values, Rights, and Duties. Here are some other links of interest: demo, dataset, and models (small, base, large, xl, xxl).

Warning

Kaleido is intended for research purposes only. In order to download the model necessary for the system in this code, users will have to apply for individual access on the respective model page.

Example usage

To intiialize the system, you can use the following code:

from KaleidoSys import KaleidoSys
system = KaleidoSys(model_name='tsor13/kaleido-xl') # sizes: small, base, large, xl, xxl

Generate Values, Rights, and Duties

From here, you can use the system to generate a candidate set of values, rights, and duties:

system.get_candidates('Biking to work instead of driving')

Output:

                              action    vrd                                          text  relevant  supports  opposes  either     label
0  Biking to work instead of driving   Duty        Duty to be environmentally responsible      1.00      1.00     0.00    0.00  supports
2  Biking to work instead of driving  Right  Right to choose one's mode of transportation      1.00      0.26     0.00    0.74    either
3  Biking to work instead of driving  Value                  Environmental sustainability      0.99      1.00     0.00    0.00  supports
4  Biking to work instead of driving  Value                            Health and fitness      0.99      1.00     0.00    0.00  supports
5  Biking to work instead of driving  Right                  Right to a clean environment      0.99      1.00     0.00    0.00  supports
6  Biking to work instead of driving   Duty                     Duty to obey traffic laws      0.99      0.13     0.01    0.86    either
7  Biking to work instead of driving  Value                                   Convenience      0.98      0.03     0.75    0.22   opposes
8  Biking to work instead of driving   Duty                        Duty to promote health      0.98      1.00     0.00    0.00  supports

Evaluate Relevance

system.get_relevance('Do a pushup', 'Value', 'Health')

Output:

tensor([0.9975, 0.0025]) # first number is p(relevant), second is p(not relevant)

Evaluate Valence

system.get_valence('Do a pushup', 'Value', 'Health')

Output:

tensor([9.9961e-01, 6.0631e-06, 3.8288e-04]) # p(Supports), p(Opposes), p(Either)

Explanation

system.get_explanation('Do a pushup', 'Value', 'Health')

Output:

"Doing pushups can improve one's physical fitness and overall well-being."

Citation

If you use Kaleido, please cite it with:

@misc{sorensen2023value,
      title={Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties}, 
      author={Taylor Sorensen and Liwei Jiang and Jena Hwang and Sydney Levine and Valentina Pyatkin and Peter West and Nouha Dziri and Ximing Lu and Kavel Rao and Chandra Bhagavatula and Maarten Sap and John Tasioulas and Yejin Choi},
      year={2023},
      eprint={2309.00779},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}