RoadsideRadar: A Roadside 3+1D Automotive Radar Point Cloud Dataset for Semantic and Instance Segmentation

Abstract The RoadsideRadar Dataset contains recordings from a 3+1D automotive Radar sensor, which was mounted on one intelligent roadside measurement setup, shown in the left side of above figure. The synchronized anonymized camera images are also added for visualization pupose. This dataset accompanies the paper Deep segmentation of 3+1D radar point cloud for real-time roadside traffic user detection and the master's thesis Real-time semantic and instance segmentation of 3D radar point cloud for smart infrastructure-based road user detection. The dataset contains 5399 frames of radar point clouds. In addition to the point cloud data from the radar sensor, semantic and instance annotations on a point-wise level from 5 different classes are provided. This dataset supports research in: Roadside Radar-based perception Semantic segmentation of radar point clouds Instance segmentation of radar point clouds Dataset Structure The dataset is provided in train, val, and test splits. Each split includes radar pcds, corresponding synchronized and anonymized camera images, and the annotation jsons. RoadsideRadar ├─────sensor.json # contains information about the sensor used. ├─────README.md └─────data ├───bg_sub_grids/ # contains .npy background subtraction grid. └───splits ├───train │ ├───images/ # contains camera .png images. │ ├───pcds/ # contains radar .pcd files. │ └───annotations/ # contains .json annotaion files. ├───val │ ├───images/ │ ├───pcds/ │ └───annotations/ └───test ├───images/ ├───pcds/ └───annotations/ Labeling Information This dataset is an extended version of the INFA-3DRC dataset, with an increased number of frames and a subset of class categories. Specifically, the following modifications are made to the INFRA-3DRC dataset: The scenes with parking spots are removed. The Radar frames with object category group are removed. Further frames of adverse weather and poor lighting conditions are added. Please refer to this manuscript for labeling information of the INFRA-3DRC dataset. The additional frames were labeled manually. Naming Convention Note that this dataset is mainly intended for radar focus research only. As such, the name of the camera images do not resemble the name of the radar frames. However, the names of the radar pcd and the corresponding annotation json file are the same. Moreover, each json annotation file contains information about the corresponsing synchronized image. The synchronized image for any given frame can be obtained using the annoatation json file (annotation_dict["cam_image"]["file_name"]). Also note that software based time-synchronization was used for obtaining the radar-camera pairs, leading to difference in the time stamps of radar and camera files. The filename include timestamps in the format YYYY-MM-DD-HH-MM-SS-MSEC. Additionally, the radar frames also include the index of the background subtraction grid at the end *_bg{idx}.pcd. radar_01__2023-06-02-21-23-51-372_bg0.pcd # radar pcd camera_01__2023-06-02-21-23-51-376.png # camera image radar__2023-06-02-21-23-51-372_bg0.json # annotation json Dataset Statistics The dataset contains total of 5399 frames of radar point clouds. The training, val, and test splits contain 3780, 810, and 809 frames respectively. Details about the class distribution of the labeled objects in each split are given in below table. A comprehensive breakdown of radar points and objects across the Train, Val, and Test splits. Class Total Points Total Objects Avg. Points / Object Train Val Test Train Val Test Train Val Test Person 4297 896 889 1920 408 392 2.24 2.20 2.26 Bicycle 6517 1403 1274 2723 578 544 2.40 2.42 2.35 Motorcycle 679 140 179 214 47 53 3.17 2.97 3.38 Car 15964 3337 3612 3786 785 880 4.22 4.25 4.10 Bus 12483 2420 2632 680 145 137 18.35 16.69 19.21 Background 110909 26746 25507 — — — — — — Total (Σ) 150939 34093 34942 9323 1963 2006 — — — Further statistics of the number of static and dynamic points per class in the dataset is provided in below table. Class-wise static and dynamic radar points distribution Category Person Bicycle Motorcycle Car Bus Static points 285 638 78 2370 0 Dynamic points 5797 8556 920 20543 17535 Total (Σ) 6082 9194 998 22913 17535 For more information, the readers are requested to explore our paper, and this master's thesis. Radar Point Cloud Fields Each radar frame is stored as a .pcd (point cloud data) file, and contains information about the radar points in that frame. Each radar point contains the following fields: Specifications of the raw and processed features provided for each radar point (detection). Parameter Unit Dtype Definition index no unit uint16 Unique index given to each radar point in one frame. range meter float32 Direct distance of the radar point with respect to origin of the radar sensor (Polar). azimuth_angle radians float32 Horizontal angle with respect to sensor origin (Polar). elevation_angle radians float32 Vertical angle with respect to sensor origin (Polar). range_rate m/sec float32 Doppler speed. Negative: approaching; Positive: receding. rcs dBsm float32 Radar Cross Section (signal strength). x, y, z meter float32 Longitudinal, lateral, and vertical distance from sensor origin (Cartesian). x_ground, y_ground, z_ground meter float32 Longitudinal, lateral, and vertical distance from the ground origin (Cartesian). u, v pixels float32 Projected pixel coordinates in the 2D image plane. v_x, v_y m/sec float32 Longitudinal and lateral velocity components of the radial velocity. Annotation Format Annotations are stored in JSON format and provide point-level semantic as well as object-level instance labels for radar points. Each object contains radar points belonging to that instance. Note that there are annotations present for image (i.e. bouning box, segmentation mask). The class definitions are given below. Category ID 1 4 5 6 7 Class Name Adult Bicycle Motorcycle Car Bus Below is an examplary annotation JSON structure. { "info": { "description": "RoadsideRadar_Dataset", "year": "2025", "version": "1" }, "objects": [ { "category_id": 7, "points": [ [101, 54.9962, -0.4125, ...], [102, 55.9972, -0.4575, ...], ] } ], "categories": [ { "category_id": "7", "supercategory": "vehicle", "name": "bus", } ], "cam_image": { "file_name": "camera_01__2023-06-02-21-23-51-376.png", "height": 1216, "width": 1920 }, "pcd_metadata": { "pcd_name": "radar_01__2023-06-02-21-23-51-372.pcd", "points": 505, "fields": "['index', 'range', 'azimuth_angle', ...]", "dtypes": "['uint16', 'float32', 'float32', ...]", } }Furthermore, the json annotation files also include the background points. Citation If this dataset has contributed to your work, we would appreciate citing our paper / dataset and giving the Github repository a star. @dataset{bhanderi_2025_19056521, author={Bhanderi, Savankumar andAgrawal, Shiva and Elger, Gordon}, title={RoadsideRadar: A Roadside 3+1D Automotive RadarPoint Cloud Dataset for Semantic and Instance Segmentation}, month=march, year=2025, publisher={Zenodo}, version={1.0}, doi={10.5281/zenodo.19056521}, url={https://doi.org/10.5281/zenodo.19056521}, } @article{bhanderi2025radar, title={Deep segmentation of 3+1D radar point cloud for real-time roadside traffic user detection}, author={Bhanderi, S. and Agrawal, S. and Elger, G.}, journal={Scientific Reports}, volume={15}, pages={38489}, year={2025}, doi={10.1038/s41598-025-23019-6} } Contact For questions regarding the dataset or collabotations, please contact the dataset authors. Savankumar Bhanderi📧