BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Behavioural Change
Recognizing complex emotions linked to ambivalence and hesitancy (A/H) can play a critical role in the personalization and effectiveness of digital behaviour change interventions. These subtle and conflicting emotions are manifested by a discord between multiple modalities, such as facial and vocal expressions, and body language. Although experts can be trained to identify A/H, integrating them into digital interventions is costly and less effective. Automatic learning systems provide a cost-effective alternative that can adapt to individual users, and operate seamlessly within real-time, and resource-limited environments. However, there are currently no datasets available for the design of ML models to recognize A/H.
About the BAH Dataset
We've developed the first Behavioural Ambivalence/Hesitancy (BAH) dataset designed to recognize subtle emotions in videos through multimodal analysis.
Dataset Highlights
- 300 participants from 9 Canadian provinces, representing diverse ages and ethnicities
- 1,427 videos totaling 10.6 hours of content (including 1.8 hours of A/H moments)
- Participants answered 7 carefully designed questions via webcam and microphone
- Expert annotations marking A/H cues at both frame and video levels
- Complete video transcripts with timestamps
- Cropped and aligned facial images for each frame
- Rich participant metadata
What's Included
Our behavioural team has meticulously annotated each video to identify where ambivalence and hesitancy occur, providing researchers with precise timestamp segments and detailed A/H cues.
The dataset includes preliminary benchmarking results for baseline models, covering:
The modest performance of baseline models underscores the real-world challenges of detecting these subtle emotional states in natural video settings.
Access the Dataset
Data, code, and pretrained weights are available here.
License / Download
BAH dataset is licensed under proprietary license for research only. To request the dataset please follow these instructions: BAH-dataset-request.
Acknowledgments
This work was supported in part by the Fonds de recherche du Québec – Santé, the Natural Sciences and Engineering Research Council of Canada, Canada Foundation for Innovation, and the Digital Research Alliance of Canada. We thank interns that participated in the dataset annotation: Jessica Almeida (Concordia University, Université du Québec à Montréal), and Laura Lucia Ortiz (MBMC).