Come and enjoy on Fri, Oct 27 at 6 pm, as The Bands of HM Royal Marines and K-TIGERS E&C Official do a joint performance at 📍Gwanghwamun Plaza in front of the Sejong Center to celebrate 🇬🇧🇰🇷140yrs.

Pleasure to host Ohad Rubin at KAIST AI to hear about his ongoing work of retrieval augmented language models, long range modeling and RPT arxiv.org/abs/2306.13421

⭐️Congratulations to everyone at KAIST AI who have a total of 20 papers accepted to #NeurIPS2023! Check them here 🧵docs.google.com/document/d/1t5…

⭐️Congratulations to everyone at KAIST AI who has a total of 11 papers accepted to #EMNLP2023! Check them here 📷🧵 docs.google.com/document/d/1YB…

A group photo from the poster presentation of »AmbiFC: Fact-Checking Ambiguous Claims with Evidence«, co-authored by @Max_Glockner (UKP Lab), Ieva Staliūnaitė (Cambridge Computer Science), James Thorne (KAIST AI) , Gisela Vallejo (@unimelb), Andreas Vlachos (Cambridge Computer Science) and Iryna Gurevych. #EMNLP2023

Congratulations Seonghyeon!

Pleased to announce that Theia Insights has raised $6.5M to develop foundational AI technologies for the investment industry. Well done team!

I can’t keep up with all these new papers and datasets from my students 😊 James Thorne and I wanted to give #kaist undergrad students research experience with LLMs in their own languages, and this is one of the outcomes of that project!

New preprint: Align LLMs without reference models using ORPO ⬇️ Code and checkpoints available on Hugging Face

This is actually huge: - No SFT stage (e.g., Zephyr used 200k examples) - Preference tuning with 7K examples only (other models trained with at least 60k samples) I've put a lot of care & love building the DPO version of the amazing Capybara dataset from LDJ so I'm

Aligning a diffusion model on preference data WITHOUT a reference model could be nice no? So, Kashif Rasul and I are ideating the use of ORPO to align SDXL 1.0 on PickAPic. Diffusion ORPO with LoRA 💫 Code and model ⬇️ huggingface.co/sayakpaul/diff…

Great work! I’m glad our benchmarks KoBBQ, KOLD, and CLIcK are used to evaluate HyperCLOVA_X. Making progress in Korean LLM! 🇰🇷🇰🇷🇰🇷 arxiv.org/abs/2307.16778 arxiv.org/abs/2205.11315 arxiv.org/abs/2403.06412

🔥Mixtral-8x22B-base + ORPO🔥 7k data & 1.3 hours to build a strong human-aligned 140B chat model🦾 👉IFEval: 65% 👉BBH: 59% 👉MT-Bench: 8.17 More models will be added to the Zephyr-ORPO collection with Argilla and Hugging Face , stay-tuned😃

We are hosting wonderful NLP colleagues at KAIST on their way to ACL Bangkok! 🤩 On-site registration is closed, but the talks will be broadcast on Zoom. Please join us! Date/Time: Aug 10, 2024, 10:05-12:30 KST (UCT+9) Parallel Session 1: Advanced Language Models and AI

📢 Excited to share our new paper with Fabrizio Silvestri: "Beyond Position: How Rotary Embeddings Shape Representations and Memory in Autoregressive Transformers"! arxiv.org/abs/2410.18067 Keep reading to find out how RoPE affects Transformer models beyond just positional encoding 🧵

NVIDIA's earnings call wasn’t about semiconductors — it was a vision for AI factories, 24/7 inference, and industry transformation. Using TIIC, we reveal $NVDA's diversification shows the future is broader than chips. What's your take? #AI #NVIDIA #LLMs linkedin.com/pulse/nvidias-…

Looking forward to attending #NeurIPS2024 this week. Come meet Namgyu Ho who will present on block-transformer, which speeds up decoding by up to 20x.

Pleased to announce the next FEVERworkshop at ACL2025! Regular workshop papers (ARR and direct submissions) due 15th of April! And new shared task focusing on reproducible and efficient verification of real world claims! Check fever.ai and get keen!

⁉️Why do reward models suffer from over-optimization in RLHF? We revisit how representations are learned during reward modeling, revealing “hidden state dispersion” as the key, with a simple fix! 🧵 Meet us at ICML Conference! 📅July 16th (Wed) 11AM–1:30PM 📍East Hall A-B E-2608