Our #CVPR2025 work on understanding movie characters by predicting their emotions (happy, angry, ...) and mental states (honest, helpful, ...) is now available! arxiv.org/abs/2304.05634 A first big publication for my MS students Dhruv Srivastava and rodo at IIIT Hyderabad! 🧵1/N

Excited to be giving a talk at the #CVPR2023 Workshop on Learning with Limited Labelled Data for Image and Video Understanding tomorrow! Come to East 3 at 9:10 to hear about reducing supervision to enable detailed video understanding.

Exciting new #lab at UvA_DSC on human-aligned #video #AI, Vacancies for 7 #PhD students expected to open in the fall. We are looking for a special breed of multidisciplinary talent, who combine AI-tech with AI-society expertise. #Amsterdam #AI

Every CV guy I know has privately admitted at some point that current video datasets do not really seem to care about time. That the video tasks are "too short" & don't test much time understanding We introduce EgoSchema -- A litmus test for truly long-form video understanding

🏆 7 MSc AI Students received ELLIS Honours Certificates as recognition of their distinguished accomplishments ivi.fnwi.uva.nl/ellis/2023/10/… Congratulations🌟 #ELLISunitAmsterdam #AI #ML #ELLISforEurope

Our paper "CounTX: Open-world Text-specified Object Counting" won a Best Poster Award at British Machine Vision Conference (BMVC) 2023 last week! Congrats to Niki Amini-Naieni! The model counts the number of objects from free-form text queries. Code and weights are available here: github.com/niki-amini-nai…

Check out our ICLR 2026 [oral] paper on learning state-of-the-art ViTs from a single video from scratch! One of the coolest things is that multi-object tracking emerges from the different heads in the plain ViTs (three heads visualised below in R,G,B).

Interested in the field of unsupervised learning, check out our new publication on improving general purpose contrastive learning models. 🔊 Audio-Language multi-modal LLM focused.

Humans learn from one continuous visual stream, but large video models have to be trained on billions of web videos. We found that learning from such sequential streams is challenging for video models—and we introduce a family of "orthogonal optimizers" to bridge the gap!

Introducing JEGAL👐 JEGAL can match hand gestures with words & phrases in speech/text. By only looking at hand gestures, JEGAL can perform tasks like determining who is speaking, or if a keyword (eg beautiful) is gestured More about our latest research on co-speech gestures 🧵👇

🔔New #CVPR2025 paper evaluating compositional reasoning of Video-LLMs on 10s, action-packed clips! 🥁 VELOCITI features 7 tests to disentangle and assess the comprehension of people, actions, and their associations across multiple events. katha-ai.github.io/projects/veloc… 🧵 1/9 #CVPR2025

Bharat Ratna for Mohammed Siraj!