Intelligent Transportation Systems (ITS) have made significant advances in areas like autonomous driving, traffic monitoring and trip planning. But just like your car needs winter tires, these systems also require adjustments to perform effectively in snowy conditions. Many ITS applications rely on cameras and are trained using vast amounts of images and videos. However, their performance drops in extreme weather because these conditions are often missing or underrepresented in the training data. A comprehensive visual dataset for winter and snow conditions serves as “winter tires” for these systems, making applications like winter road maintenance and traffic estimation more robust.
In this article, we introduce “ViSnow” [1], an image dataset of snow-covered urban streets destined for computer vision applications in ITS. ViSnow consists of nearly 300,000 images captured by traffic surveillance cameras across Montreal, Quebec, a city that receives in average 210 cm of snow annually. This makes it an ideal environment for urban snow-related research. The images were collected throughout the winters of 2022 and 2023, before, during, and after snowfall events. They were publicly accessible via an online service from the City of Montreal, which provided updated snapshots every 5 to 10 minutes. We enriched each image with a text file containing information about the time and weather conditions.
The dataset includes images from various weather conditions (clear, rainy, snowy, cloudy), times of day (day, night, dusk, dawn) and locations (downtown, neighbourhoods, industrial areas). It also captures different traffic scenarios (congested, smooth, no traffic) and challenging vision conditions, such as poor visibility or cameras obstructed by snow or rain. Examples of these images are illustrated in the figure below.
ViSnow's diverse content can enhance the performance of various applications and enable new ones. Examples include optimizing snow removal planning, trip-planning systems that recommend the best routes and modes of transport during snowfall, winter traffic analysis for detecting congestion and vulnerable roads, and more accurate predictions of public transportation arrival times and delays.
To develop applications using ViSnow, the images need to be properly annotated. For example, in winter road maintenance, images can help classify road surface conditions. This requires annotating images based on snow cover to use them for the classifier training. In our published article [2], we introduced an automated method for this task. It uses image processing and data clustering techniques to detect different categories based on latent characteristics within the images. The specific classes are clear surface, light-covered surface, medium-to-heavy-covered surface and plowed surface.
To focus on road conditions, the images were first grouped by day or night, and by camera. They were then converted in binary mode (black and white) to highlight the snow component. Next, they were processed through an autoencoder neural network, which learns hidden features from the images. These features exist in a new space where key characteristics of each category are easily distinguishable. We used the Louvain algorithm, a community detection method, to cluster the images and annotate them into the desired categories. The resulting annotated dataset consists of about 40,000 images distributed as 17,422 clear surface images, 3,726 light-covered surface images, 14,725 medium-to-heavy covered surface images, and 3,512 plowed surface images. This distribution is due to the different representations of each category during snowfall events.
Afterwards, we used the ViSnow annotated version to train convolutional neural networks (CNN) classifying the images into one of the four categories. We trained the EfficientNet and ResNet50 models with different training settings. We obtained the best results when we trained the EfficientNet model on separate day and night image sets, with a training function that takes into account the skewed distribution of the categories. The model achieves an accuracy of 97% in classifying new images from the cameras.
Using the annotated ViSnow dataset, we trained several convolutional neural networks (CNN) to classify images into four categories. We used models like EfficientNet and ResNet50 under different training settings. The best results were achieved by training EfficientNet on separate day and night image sets, using a function that accounted for the skewed distribution of categories. This model reached an accuracy of 97% in classifying new images from the cameras.
The ViSnow dataset, both in annotated and unannotated formats, aims to encourage and advance research and technologies related to transportation systems in snowy conditions. The dataset has allowed us to propose and implement valuable use-cases relevant to the public, notably snow removal planning. We encourage further work using ViSnow, which we have made available online [3].
References:
[1] Karaa, Mohamed, Hakim Ghazzai et Lokman Sboui. 2024. "ViSnow: Snow-Covered Urban Roads Dataset for Computer Vision Applications". IEEE Open Journal of Systems Engineering, vol. 2, p. 62‑70.
[2] Karaa, Mohamed, Hakim Ghazzai et Lokman Sboui. 2024. « A Dataset Annotation System for Snowy Weather Road Surface Classification ». IEEE Open Journal of Systems Engineering, vol. 2, p. 71‑82.
[3] Karaa, Mohamed Karaa, Hakim Ghazzai Ghazzai et Lokman Sboui Sboui. ViSnow: Snow-covered Urban Roads Dataset for Computer Vision Applications. IEEE Dataport. https://doi.org/10.21227/9DYZ-....