CarlaEnv-Benchmark

An autopilot benchmark for multi-modality visual reinforcement learning, and world modeling of intelligent transportation system, etc.

Note: Our benchmark is constantly being upgraded, including the addition of extreme weather and challenging traffic scenarios.

Weather/Scenario	highway	narrow	jaywalk	tunnel	merging
hard_high_light
soft_high_light
soft_low_light
hard_low_light
soft_noisy_low_light
hard_noisy_low_light

Requirements
Installation
Usage
Custom Settings
Citation
Acknowledgements

1/Requirements

CarlaEnv-Benchmark is developed and tested under the following settings:

Ubuntu: 18.04
Carla: 0.9.13
Python: 3.7.13

2/Installation

Download the compiled release version and additional maps of CARLA 0.9.13 from here or using the following code:

wget https://carla-releases.s3.eu-west-3.amazonaws.com/Linux/CARLA_0.9.13.tar.gz
mkdir CARLA_0.9.13
tar -zxvf CARLA_0.9.13.tar.gz -C CARLA_0.9.13

cd CARLA_0.9.13/
wget https://carla-releases.s3.eu-west-3.amazonaws.com/Linux/AdditionalMaps_0.9.13.tar.gz
tar -zxvf AdditionalMaps_0.9.13.tar.gz

Installing the Python API of Carla:

cd carla_0.9.13/PythonAPI/carla/dist
easy_install carla-0.9.13-py3.7-linux-x86_64.egg

Download source code of CarlaEnv-Benchmark:

git clone https://github.com/kyoran/CarlaEnv-Benchmark

Installing necessary packages:

pip install -r requirements.txt

3/Usage

We first need to run the rendering engine CARLA server in the background:

cd carla_0.9.13/
DISPLAY= ./CarlaUE4.sh -opengl -RenderOffScreen -world-port=12321

To manipulate the environment through the Python API, we need to add the following code to the start of the script or the main function:

import json
import matplotlib.pyplot as plt

from env.CarlaEnv import CarlaEnv
from utils.VideoRecorder import VideoRecorder

We also need to load config files:

with open('./cfg/weather.json', 'r', encoding='utf8') as fff:
    weather_params = json.load(fff)
with open('./cfg/scenario.json', 'r', encoding='utf8') as fff:
    scenario_params = json.load(fff)

Then, we can create a carla environment with selected weather "hard_high_light" and selected scenario "jaywalk":

carla_env = CarlaEnv(
    weather_params=weather_params,
    scenario_params=scenario_params,
    selected_weather="hard_high_light",
    selected_scenario="jaywalk",
    carla_rpc_port=12321,
    carla_tm_port=18935,
    carla_timeout=8,
    perception_type="dvs+vidar",
    num_cameras=5,
    rl_image_size=256,
    fov=60,
    max_fps=120,
    min_fps=30,
    max_episode_steps=1000,
    frame_skip=1,
    ego_auto_pilot=True,   # testing purpose
    is_spectator=True,     # rendering mode
)

To record what has happened in the process of simulation, we can create a recorder:

video = VideoRecorder("./video", min_fps=30, max_fps=120)

Now the simulation is running by using the following code, with numerous vehicles driving around the map, several pedestrians jaywalking, and a third-person-perspective camera recording data, and five first-person perspective perception data from the ego vehicle. This data can then be used to feed a machine learning algorithm for training an autonomous driving agent.

obs = carla_env.reset()
video.init(True)
for one_step in range(400):
    action = [0, 0.7]
    obs, reward, done, info = carla_env.step(action)

    video.record(obs, carla_env.vehicle)
video.save("test")

Finally, we can get recorded video in the 'video' directory.

4/Custom Settings

We provide five scenarios and six weathers:

4.1/Scenario

highway: is a wide road with four lanes that vehicles on it run relatively faster than the other scenarios.
narrow: is a long and narrow S-bend road that vehicles are forbidden to change lanes.
jaywalk: is an intra-city road that some walkers randomly cross.
tunnel: is a tunnel where challenging illumination inevitably happens when entering and exiting.
merging: is a four-lane road that has three lanes randomly blocked by crashed vehicles.

4.2/Weather

hard_high_light: is the brightest weather of the midday.
soft_high_light: is the twilight when the sun is about to go down.
soft_low_light: is a cloudy and foggy day.
hard_low_light: is the darkest weather of the midnight.
soft_noisy_low_light: is the dust with little rain.
hard_noisy_low_light: rain cats and dogs in the midnight.

kyoran/CarlaEnv-Benchmark