A Spatio-Temporal Multi-Task Framework for iRAP Road Attribute Classification from Street-Level Videos

Road safety is a significant global issue. Each year approximately 1.19 million people die and 20 to 50 million suffer injuries in road crashes. Existing proactive road infrastructure safety assessment tools play an important and increasing role in identifying and reducing infrastructure-related risks. Such tools depend on reliable identification of road safety related attributes (e.g., area type, land use surrounding roads, sidewalks, lane count and width, and roadside hazards, etc.) for preventing traffic crashes and improving infrastructure design and planning. Automated classification of these attributes from readily available sources, such as street-level imagery, has the potential to rapidly accelerate the use of these tools across road networks world-wide. However, achieving reliability using these methods is challenging due to inter-attribute dependencies and the need to capture temporal information from video sequences.Our study aimed to use street-level imagery to classify road attributes used in the iRAP road safety "Star Rating" assessment models. In this study, we used MaxViT as the backbone and apply a Temporal Transformer after extracting features from the backbone. Our experiments on five road attributes used in the iRAP models (Land Use driver side, Land Use passenger side, Area Type, Sidewalk driver side and Sidewalk passenger side) demonstrate that our proposed approach, which incorporates learned temporal relationships, improves classification performance, achieving Macro F1 scores of 0.48876 land use driver, 0.6065 for land use passenger, 0.6910 for Sidewalk driver, 0.6794 for Sidewalk passenger and 0.8554 for area type classification.