Zero-Shot Vision-Language Model for Event Detection in Smart Surveillance

Vision-Language Models (VLMs) have shown remarkable capabilities in processing visual data through natural language, yet their potential in smart surveillance for abnormal event detection remains underexplored. Traditional anomaly detection systems rely on predefined event classes, limiting their ability to identify novel or ambiguous anomalies in diverse scenarios. To address this, we propose VEZA (Vision Encoding for Zero-shot Anomalies), a novel architecture lever-aging pre-trained VLMs for zero-shot anomaly detection. With VLMs, we integrate vision and language modalities to encode and caption video streams, enabling flexible and context-rich anomaly detection without surveillance-specific training. This dual-modality approach enhances robustness across varied real-world settings. Evaluated on the UCF -Crime dataset, VEZA achieved up to 89% in open-vocabulary anomaly detection and video-text retrieval tasks, with 45% R @ 1 on the Ubnormal dataset as an additional evaluation. These results demonstrate VEZA's ability to identify and retrieve a wide variety of abnormal events without the need for prior training on surveillance-specific scenarios.