Jarviswang94/Multilingual_safety_benchmark

Multilingual safety benchmark for Large Language Models

Multilingual Safety Benchmark for LLMs

Paper: All Languages Matter: On the Multilingual Safety of Large Language Model.

RESEARCH USE ONLY✅ NO MISUSE❌

💡Framework

We build the first multilingual safety benchmark for LLMs, XSafety, in response to the global deployment of LLMs in practice. XSafety covers 14 kinds of commonly used safety issues across 10 languages that span several language families. We utilize XSafety to empirically study the multilingual safety for 4 widely-used LLMs, including both close-API and open-source models. Experimental results show that all LLMs produce significantly more unsafe responses for non-English queries than English ones, indicating the necessity of developing safety alignment for non-English languages

📃Results

👉 Paper and Citation

For more details, please refer to our paper here.

Community Discussion:

• Twitter: AIDB, Jiao Wenxiang

Citation

If you find our paper&data interesting and useful, please feel free to give us a star and cite us through:

@article{wang2023all,
  title={All Languages Matter: On the Multilingual Safety of Large Language Models},
  author={Wang, Wenxuan and Tu, Zhaopeng and Chen, Chang and Yuan, Youliang and Huang, Jen-tse and Jiao, Wenxiang and Lyu, Michael R},
  journal={arXiv preprint arXiv:2310.00905},
  year={2023}
}