Unified Data Engineering for Smart Mobility: Real-Time Integration of Traffic, Public Transport, and Environmental Data

Urbanization and environmental deterioration are both increasing, requiring sustainable solutions, real-time, for urban mobility. In this study, a unified data engineering framework for smart mobility is provided, combining three key data streams: traffic congestion, public transportation operations (GTFS), and environmental pollutants (AQI). The architecture uses Apache Kafka and Flink for high-throughput streaming processing, performs semantic transformations using JSON-LD and GeoSPARQL, and stores spatiotemporal data with PostGIS and InfluxDB. A fusion engine aligns data streams to a common time and geo-spatial grid and screens out anomalies based on Z-score thresholds. Predictive modeling uses XGBoost and LSTM to predict AQI based on using multimodal input such as PM2.5, NO<inf xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</inf>, and vehicle count. The results of the evaluation show that XGBoost significantly outperforms LSTM, with a lower MAE (0.032) and RMSE (0.061), and higher R<sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> (0.9999), which makes it suitable for real-time operation. The connected platform is visualized on a Grafana dashboard, providing real-time guidance for decision-making to city planners and the city. This work delivers a scalable, interoperable, and AI-ready smart cities infrastructure that will help reduce congestion, emissions, and improve public health outcomes. The framework aligns with the UN Sustainable Development Goals and is essential in the framework for smart data-driven urban transportation systems.