🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

Only 33 days left to submit your paper to #SIGIRAP2025! Don't miss the abstract deadline.

Let's talk about Mixture-of-Transformers (MoT) and heterogeneous omni-model training. 1. Inspired by prior architectures consisting of modality-specific parameters—such as Flamingo, CogVLM, BEIT-3, and MoMA—MoT (arxiv.org/abs/2411.04996) pushes this idea further by using

hello twittersphere! i am planning to graduate in a few months, so i am officially ✨ Looking For A Job ✨ if you know of a role that'd be a good fit, or just want to chat, please reach out! here are some projects i've worked on that i'm most proud of 👇

Multimodal RAG: Just use ColPali/DSE then pass your screenshots to the LLM This is the dream, but how well do LLMs read text contained in images? We wanted to know, so we tried a simple thing: do results change on evals when using screenshots rather than text as input? Yes.

Among papers I wrote, GradCache is one of my favorite. Very glad to see it still being useful🚀

🎉Our Spurious Rewards is available on ArXiv! We added experiments on - More prompts/steps/models/analysis... - Spurious Prompts! Surprisingly, we obtained 19.4% gains when replacing prompts with LaTex placeholder text (\lipsum) 😶‍🌫️ Check out our 2nd blog: tinyurl.com/spurious-prompt

Congrats on the release!

It’s been ~4 weeks since we launched Yupp – a consumer-first approach to robust & trustworthy AI evaluation. We’re still early but have already gathered 2M+ high-quality human preference feedback datapoints on 500+ models across diverse use cases. 🧵 x.com/pankaj/status/…

It’s been 36 hours since Grok 4 launched and we have an early verdict based on 6K+ preferences of Yupp users globally on real use cases. ‼️ Grok 4 is worse than other leading models: OpenAI o3, Claude Opus 4, and Gemini 2.5 Pro. Grok 4 is liked even less than Grok 3. 🧵

Got my visa approved right before departure 😂. Heading to Padova for #SIGIR2025, looking forward to chatting about multi-modality/reasoning for information retrieval.

NeuralOS Towards Simulating Operating Systems via Neural Generative Models a generative OS that predicts screen images from user inputs, combining an RNN for computer state modeling and a diffusion model for rendering

Can we build an operating system entirely powered by neural networks? Introducing NeuralOS: towards a generative OS that directly predicts screen images from user inputs. Try it live: neural-os.com Paper: huggingface.co/papers/2507.08… Inspired by Andrej Karpathy's vision. 1/5

This is a good opportunity to announce that I recently joined the research team at Databricks where I will be working alongside Jonathan Frankle Rishabh Singh Matei Zaharia Erich Elsen, and many others on the hardest problems at the intersection of information retrieval and AI.

ScholarCopilot (led by Yubo Wang) is now accepted at COLM 2025! Lots of great work has emerged over the past half year on improving interleaved search and reasoning. Looking forward to seeing them apply to the scientific writing tasks.