SPEAR: A Simulator for Photorealistic Embodied AI Research

Interactive simulators are becoming powerful tools for training embodied agents, but existing simulators suffer from limited content diversity, physical interactivity, and visual fidelity. We address these limitations by introducing SPEAR: A Simulator for Photorealistic Embodied AI Research. To create our simulator, we worked closely with a team of professional artists for over a year to construct 300 unique virtual indoor environments with 2,566 unique rooms and 17,234 unique objects that can be manipulated individually. Each of our environments features detailed geometry, photorealistic materials, and a unique floor plan and object layout designed by a professional artist, i.e., we do not rely on remixing existing layouts to create additional content. Our environments are implemented as Unreal Engine assets, and we provide an OpenAI Gym interface for interacting with the environments via Python.

The SPEAR code is released under an MIT License, and the SPEAR assets are released under various licenses that permit academic use.

Citation

If you find SPEAR useful in your research, please cite this repository as follows:

@misc{spear,
    author       = {Mike Roberts AND Rachith Prakash AND Renhan Wang AND Quentin Leboutet AND
                    Stephan R. Richter AND Stefan Leutenegger AND Rui Tang AND Matthias
                    M{\"u}ller AND German Ros AND Vladlen Koltun},
    title        = {{SPEAR}: {A} Simulator for Photorealistic Embodied AI Research},
    howpublished = {\url{http://github.com/spear-sim/spear}}
}

Getting Started

Minimum and recommended system specifications

Minimum and recommended system specifications for the Unreal Engine are given here.

Precompiled binaries

See our latest release notes for download links. The easiest way to start working with SPEAR is to download a precompiled binary for your platform. Our precompiled binaries come pre-packaged with the scene pictured above. You can start interactively navigating around this scene with the keyboard and mouse simply by running the downloaded binary with no additional arguments.

Documentation

Our Getting Started tutorial explains how to interact with multiple scenes and our Python interface.
Our Building SpearSim tutorial explains how to build from source.
Our Importing and Exporting Assets tutorial explains how to import and export assets.
Our Coding Guidelines document describes our coding standard.
Our Contribution Guidelines document contains information on how to contribute effectively.

Licenses

Code

The code in this repository is licensed under an MIT License.
The licenses for all of our third-party code dependencies are given here.

Assets

The apartment, debug, and warehouse scenes are licensed under a CC0 License.
The OpenBot and Fetch assets in this repository are licensed under a CC0 License.
The license for the kujiale scenes is given here.
The license for the StarterContent assets referenced in the debug scenes is given here.

Acknowledgements

From 2021 to 2024, SPEAR was developed with generous support from the Intelligent Systems Lab at Intel and Kujiale. Beginning in 2024, SPEAR is being developed by an independent consortium with generous support from Kujiale.