Automating AI research is exciting! But can LLMs actually produce novel, expert-level research ideas? After a year-long study, we obtained the first statistically significant conclusion: LLM-generated ideas are more novel than ideas written by expert human researchers.

Do you work in AI? Do you find things uniquely stressful right now, like never before? Haver you ever suffered from a mental illness? Read my personal experience of those challenges here: docs.google.com/document/d/1aE…

It's cool to see Google DeepMind's new research to show similar findings as we did back in April. IsoBench (isobench.github.io, accepted to Conference on Language Modeling 2024) was curated to show the performance gap across modalities and multimodal models' preference over text modality.

💡We find that models “think” 💭 in English (or in general, their dominant language) when processing distinct non-English or even non-language data types 🤯 like texts in other languages, arithmetic expressions, code, visual inputs, & audio inputs ‼️ 🧵⬇️arxiv.org/abs/2411.04986

Like how we might have a semantic "hub" in our brain, we find models tend to process🤔non-English & even non-language data (text, code, images, audios.etc) in their dominant language, too! Thank you Zhaofeng Wu for the wonderful collaboration!

Can't be at EMNLP this year... but can celebrate on being recognized as an outstanding reviewer!! ❤️ I'm so happy!!

Julien Chaumond Qwen This is relevant to our finding that models tend to "think"/"reason" in their dominant pretraining language 💭 x.com/zhaofeng_wu/st…

Akari is extremely intelligent, caring, thoughtful, organized, and supportive. Her work is very impactful as well. Any school that hires her secures a gem!!

This is not funny 😠

We have released our code at github.com/ZhaofengWu/sem…. We hope that this could be useful for future studies understanding the how LMs work!

Introducing METAGENE-1🧬, an open-source 7B-parameter metagenomics foundation model pretrained on 1.5 trillion base pairs. Built for pandemic monitoring, pathogen detection, and biosurveillance, with SOTA results across many genomics tasks. 🧵1/

To appear @ #ICLR2025! We show that LMs represent semantically-equiv. input across languages, modalities, etc similarly. This shared repr. space is structured by the LM's dominant language, which is also relevant to recent phenomena where LMs "think" in Chinese🀄️in Eng🔠 contexts

x.com/i/article/1892…

[1/x] Humans naturally understand implicit cultural values in conversation—but can LLMs do the same? We are excited to introduce CQ-Bench, a benchmark for evaluating LLMs’ cultural intelligence through dialogue. Details below 🧵👇

Super sad to miss ICLR, but please check out our poster if you have time!! 🙌

🚨Do passage rerankers really need explicit reasoning?🤔—Maybe Not! Our findings: ⚖️Standard rerankers outperform those w/ step-by-step reasoning! 🚫Disable reasoning from reasoning reranker actually improves reranking accuracy!🤯 👇But, why? 📰arxiv.org/abs/2505.16886 (1/6)

What happens when an LLM is asked to use information that contradicts its knowledge? We explore knowledge conflict in a new preprint📑 TLDR: Performance drops, and this could affect the overall performance of LLMs in model-based evaluation.📑🧵⬇️ 1/8 #NLProc #LLM #AIResearch

I'll be in Vienna at #acl2025! Please lmk if you'd like to chat 🙂