General aims
Our group aims to understand the dynamics of social interaction in both humans and nonhuman animals using cutting-edge tracking technologies including motion-capture, gaze-tracking, and computer vision techniques. Our goal is to 1) establish techniques to track postures and eye/head directions across species and then 2) identify key interactive components among individuals, as well as key cognitive and emotional traits of individuals, that lead to successful synchronization and joint action of a collective. Our focus is to study real-life social interaction across species, including birds (pigeons and crows), primates (great apes and monkeys), and humans, while they are engaged in various types of social behavior, such as team cooperation (in humans), vigilance (in pigeons), and collective foraging (in all animals).
Who is looking at whom? Gaze-tracking of individuals in a group during natural social interactions in pigeons, crows, monkeys, and great tits: One key challenge of studying animal social cognition is to study cognition in a natural environment where animals freely interact with both the environment and conspecifics. We tackle this challenge by tracking the “gaze” of freely-behaving animals – “gaze” as a window to their mind.
Pigeons:
Using a large-scale motion-capture system, we reconstruct the gaze of pigeons and study the dynamics of attention during vigilance and feeding - a key multitasking situation critical to the animals’ survival.
Project Lead: Mathilde Delacoux
Great tits:
Studies of animal culture often use association networks to infer whether there is social transmission of novel information. However, association is only a proxy for attention, and it remains to be seen how closely attention networks might correlate with association. In this project we aim to use cutting edge machine learning methods to quantify birds’ gaze during a social learning task in a wild population of great tits. From this fine-scale data we can obtain an attention network, and use it to explore how variation in individuals’ attention to one another shapes the spread of a cultural trait throughout the population.
Project Lead: Michael Chimento
Humans:
What are the secrets about human coordination? Reverse-engineering key interaction components during human joint actions. When people are doing something together, such as playing sports, learning from and teaching others, and simply watching movies together, the coordination with partners sometimes works but sometimes does not. What are the secrets about successful human coordination? This project aims to reveal key interaction components during human joint action using various tracking technologies (wearable eye-trackers, motion-capture system, heart-rate sensors) and modeling approaches.
Project Lead: Prasetia Putra
Markerless 3D Posture Tracking in Birds
With the rapid development of computer vision and machine learning frameworks for animal posture tracking in recent years, tracking animals automatically with the use of video cameras in captivity and the wild is becoming more accessbile. In this project, we take advantage of the motion tracking systems to automatically generate training data for markerless posture tracking in pigeons and crows, with the goal of developing a framework that could reconstruct 3D postures of bird flocks in captivity and the wild.
Project Lead: Alex Chan
Monkeys:
Using drones and computer vision, we reconstruct the visual fields of wild Japanese monkeys at Kohshima island, Japan, during foraging and study the attention network in relation to their dominance and social relationships.
Crows:
Crows are well known for sophisticated social cognition comparable to great apes’. Using a custom motion-capture system (built by ourselves), we reconstruct the visual field of crows and study their socio-cognitive skills, such as gaze following.