BAH Dataset for Ambivalence and Hesitancy Recognition in Videos for Behavioural Change
Recognizing complex emotions linked to ambivalence and hesitancy (A/H) can play a critical role in the personalization and effectiveness of digital behaviour change interventions. These subtle and conflicting emotions are manifested by a discord between multiple modalities, such as facial and vocal expressions, and body language. Although experts can be trained to identify A/H, integrating them into digital interventions is costly and less effective. Automatic learning systems provide a cost-effective alternative that can adapt to individual users, and operate seamlessly within real-time, and resource-limited environments. However, there are currently no datasets available for the design of ML models to recognize A/H.
About the BAH Dataset
We've developed the first Behavioural Ambivalence and Hesitancy (BAH) dataset designed to recognize subtle emotions in videos through multimodal analysis.
Dataset Highlights
- 300 participants from 9 Canadian provinces, representing diverse ages and ethnicities.
- 1,427 videos totaling 10.6 hours of content (including 1.8 hours of A/H moments).
- Participants answered 7 carefully designed questions via webcam and microphone.
- Expert annotations marking A/H cues at both frame and video levels.
- Complete video transcripts with timestamps.
- Cropped and aligned facial images for each frame.
- Rich participant metadata.
What's Included
Our behavioural team has meticulously annotated each video to identify where ambivalence and hesitancy occur, providing researchers with precise timestamp segments and detailed A/H cues.
The dataset includes preliminary benchmarking results for baseline models, covering:
The modest performance of baseline models underscores the real-world challenges of detecting these subtle emotional states in natural video settings.
License / Download
BAH dataset is licensed under proprietary license for research only. To request the dataset please follow these instructions: BAH dataset download link.
Voir la citation BibTeX
@article{gonzalez-25-bah,
title="{BAH} Dataset for Ambivalence/Hesitancy Recognition in Videos for Behavioural Change",
author="González-González, M. and Belharbi, S. and Zeeshan, M. O. and Sharafi, M. and Aslam, M. H. and Pedersoli, M. and Koerich, A. L. and Bacon, S. L. and Granger, E.",
journal="CoRR",
volume="abs/2505.19328",
year="2025"
}
Additional material
Contact us
For any questions, please write to the director of LIVIA: eric.granger@etsmtl.ca.
Acknowledgments
This work was supported in part by the Fonds de recherche du Québec – Santé, the Natural Sciences and Engineering Research Council of Canada, Canada Foundation for Innovation, and the Digital Research Alliance of Canada. We thank interns that participated in the dataset annotation: Jessica Almeida (Concordia University, Université du Québec à Montréal), and Laura Lucia Ortiz (MBMC).