Huge thanks to my truly awesome mentor Ruofei Du and collaborators! Looking forward to seeing everyone in Hamburg!

What if #AI could assists you w/ relevant realtime visuals in #AR during conversations? Check out Project #VisualCaptions: duruofei.com/projects/augme… Kudos to Xingyu Bruce Liu, Vlad, Xiuxiu, Alex Olwal Xiang 'Anthony' Chen Peggy Chi for the wonderful teamwork! ACM SIGCHI #CHI2023 #GPT3 #LLM

Are you tired of jittery captions in Google Meet, Zoom, Microsoft Teams, #FaceTime, Skype? Check out our latest research in text stability in live captions ACM SIGCHI #CHI2023! #TextStability paper available at in duruofei.com/projects/augme… #accessbility #a11y #HCI #UX #AR

Congrats to Xingyu Bruce Liu on winning the Edward K. Rice Outstanding Master’s Student award!

#VisualCaptions CHI'23 #opensource at github.com/google/archat! Feel free to star+fork! A big kudos to Xingyu Bruce Liu and the #ARChat + #Rapsai teams for an amazing year of dedication and hard work! You may need to apply API keys to run, but we verified it :) #AR ACM CHI Conference ACM UIST

What if #AI could show context-relevant visuals in online meetings? #VisualCaptions in #ARChat is an interactive system to augment communication w/ interactive visuals suggested by #LLM + a search →goo.gle/42yYgBi Fork & star our code & dataset →github.com/google/archat

Congrats to our 11 UCLA doctoral students from computer science UCLA CS Department, bioengineering UCLA Bioengineering, and electrical and computer engineering UCLA Electrical and Computer Engineering who have been awarded Amazon fellowships by Amazon Science to conduct #AI research. sciencehub.ucla.edu/2023-amazon-fe…

Congrats to Xingyu Bruce Liu on receiving the Amazon PhD Fellowship!

traveling to #UIST2023 from LA along the beautiful west coast w/ Xingyu Bruce Liu & Nazlı Doğan

Hearty congratulations to Xun, Xingyu Bruce Liu, Siyou Pei (on job market), and the team on the #CHI2024 papers in #XR & #AI! Can't wait to finalize the revision and share the latest invention with the world! Thanks Purdue Mechanical Engineering UCLA Computer Science UCLA Electrical and Computer Engineering for the AMAZING collaboration with Google AI!

🤔Ever missed an important call in a noisy place? Our recent work Human I/O introduces a unified approach that rethinks Situational Impairments as the availability of human input/ouput channels, e.g. hands👋, vision👀, hearing👂 and speaking👄. Preprint: arxiv.org/pdf/2403.04008

💡Ever missed a breathtaking view while driving? Tired of juggling apps to perform digital actions? #CHI2024 Meet OmniActions: an egocentric, multimodal AI agent in #PervasiveAR that predicts your next move during real-world interaction. Preprint: arxiv.org/abs/2405.03901

Human I/O is a unified approach that uses egocentric vision, multimodal sensing, and LLM reasoning to detect situational impairments and assess a user’s ability to interact with their hands, vision, hearing or speech in a given situation. Learn more at: goo.gle/3KJEP1T

Check out our latest blog post in #HumanIO that introduces a unified approach that rethinks Situational Impairments as the availability of human input/ouput channels, e.g. hands👋, vision📷, hearing📷. 👄 #AR #accessbility #LLM #a11y #XR #metaverse

I’ll be presenting Inner Thoughts tomorrow (Tuesday) 2:10 PM room G415 at #CHI2025. Come and say hi!