🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

New paper (to appear at ACL 2023)! We present Marked Personas, an unsupervised way to measure stereotypes in LLMs for any intersectional identity. Paper: arxiv.org/pdf/2305.18189… Joint work with the wonderful Esin Durmus Dan Jurafsky Stanford NLP Group 🧵1/6

🚨 New policy brief: Millions of images are generated each day using text-to-image AI systems. Our latest brief examines how major image generation models encode a wide range of dangerous biases about demographic groups. Read or download here: hai.stanford.edu/policy-brief-d…

The way we talk about AI matters. “The model understands how to…” implies much more powerful capabilities than “The model is used to…” We present AnthroScore, a measure of how much tech is anthropomorphized, i.e talked about in human-like ways. #EACL2024 anthroscore.stanford.edu

💥 New paper 💥 We discover a form of covert racism in LLMs that is triggered by dialect features alone, with massive harms for affected groups. For example, GPT-4 is more likely to suggest that defendants be sentenced to death when they speak African American English. 🧵

causalgym won an area chair award and outstanding paper award at ACL 😁 thanks to my very cool advisors Christopher Potts and Dan Jurafsky

It's back-to-school time and so here's the Fall '24 release of draft chapters for Speech and Language Processing! web.stanford.edu/~jurafsky/slp3/

Stanford Linguistics is hiring!!! We have an open area, open rank faculty position! Apply here: facultypositions.stanford.edu/en-us/job/4947…

Beyond excited to share that this is now out in nature! We show that despite efforts to remove overt racial bias, LLMs generate covertly racist decisions about people based on their dialect. Joint work with amazing co-authors ria / pratyusha ria kalluri, Dan Jurafsky, and Sharese King.

Content moderation algorithms can mistakenly flag stories of experiencing racism as toxic content. Human users also flag discrimination disclosures for removal more often than stories about negative interpersonal experiences that don’t involve race. PNAS: ow.ly/f9v850TmWCp

Great new paper from Jennifer Eberhardt and co.: pnas.org/doi/10.1073/pn…

The AI community lacks consensus on AI policy. With colleagues across academia, we argue that we need to advance scientific understanding to build evidence-based AI policy. understanding-ai-safety.org

Since the EVIDENCE Act was signed in 2019, evidence-based policy has been the official mandate of US government. It's crucial that research is brought to bear on AI policy to mitigate harm and realize potential good. Honored to be working w these researchers to advance this goal.

1/ Can modern language models (LMs) accurately distinguish fact, belief, and knowledge? Our new study systematically explores this question, identifying several key limitations that have serious implications for LM applications in healthcare, law, journalism, and education.

Happy New Year everyone! Jim and I just put up our January 2025 release of Speech and Language Processing! Check it out here: web.stanford.edu/~jurafsky/slp3/

Our article on using LLMs to improve health equity is out in NEJM AI! 85% of equity-related LLM papers focus on *harms*. But equally vital are the equity-related *opportunities* LLMs create: detecting bias, extracting structured data, and improving access to health info.

Dear ChatGPT, Am I the Asshole? While Reddit users might say yes, your favorite LLM probably won’t. We present Social Sycophancy: a new way to understand and measure sycophancy as how LLMs overly preserve users' self-image.

new paper! 🫡 why are state space models (SSMs) worse than Transformers at recall over their context? this is a question about the mechanisms underlying model behaviour: therefore, we propose using mechanistic evaluations to answer it!

I'm excited to announce that I’ll be joining the Computer Science department at Johns Hopkins University as an Assistant Professor this Fall! I’ll be working on large language models, computational social science, and AI & society—and will be recruiting PhD students. Apply to work with me!