Extensive Sensor Suite

To encourage research in multi-modality sensing, we try to offer the most extensive sensor suite based on CARLA. On carside, the sensors are set up under the configuration shown above. Each vehicle is equipped with 6 RGB/depth/semantic segmentation cameras, one BEV semantic segmentation camera, one LiDAR and semantic LiDAR sensor, one IMU and one GNSS sensor.

On roadside, we attach 4 RGB cameras, one BEV semantic segmentation camera, one LiDAR and one semantic LiDAR to the infrastructure. Both the cameras on vehicles and roadside infrastructures are able to cover 360° to ensure a full-view perception.

Examples of the processed data are included at the bottom of this page.

---

RGB Camera

• On vehicle
  • 70° horizontal FoV (front/front left/front right/back left/back right)
  • 110° horizontal FoV (back)
  • image size: 1600x900 pixels

• On infrastructure
  • 70° horizontal FoV
  • facing downward at 35° diagonally
  • image size: 1600x900 pixels

Depth Camera

• On vehicle
  • 70° horizontal FoV (front/front left/front right/back left/back right)
  • 110° horizontal FoV (back)
  • image size: 1600x900 pixels

Semantics Segmentation Camera

• On vehicle
  • 70° horizontal FoV (front/front left/front right/back left/back right)
  • 110° horizontal FoV (back)
  • image size: 1600x900 pixels
  • raw images are available

BEV Semantics Segmentation Camera

• On vehicle
  • 1 BEV semantic segmentation camera at the top facing downward
  • image size: 900x900 pixels
  • raw images are available

• On infrastructure
  • 1 BEV semantic segmentation camera at the top facing downward
  • image size: 900x900 pixels
  • raw images are available

IMU and GNSS Sensor

• On vehicle
  • 1 IMU/GNSS sensor at the top
  • stored in .npy files

LiDAR and Semantic LiDAR Sensor

• On vehicle
  • 1 LiDAR and 1 semantic LiDAR sensor at the top
  • 32 channels
  • 70m max range
  • 250,000 points per second
  • 20 Hz rotation frequency

• On infrastructure
  • 1 LiDAR and 1 semantic LiDAR sensor at the top
  • 32 channels
  • 70m max range
  • 250,000 points per second
  • 20 Hz rotation frequency

Town Maps

Currently, CARLA's Town 3, Town 4, and Town 5 are considered in our dataset. Town 3, the most complex town in CARLA, is a mid-sized town with features such as slopes, small junctions, and roundables. Town 4 is a small town with a highway system that encloses it. While Town 5 is the largest town among the three with multiple lanes per direction. We use the above-mentioned CARLA-SUMO co-simulation to spawn multiple types of vehicles such as cars, bicycles, motorcycles, and emergency vehicles to the town as retrieve sensor data from the recording files.

More details about the towns in CARLA can be found at CARLA's documentations.