In our new paper, we introduce Latxa, a family of LLMs for Basque from 7 to 70B parameters that outperform open models and GPT3.5. Models and datasets Hugging Face hf.co/collections/Hi… Code: github.com/hitz-zentroa/l… Blog: hitz.eus/en/node/343 Paper: arxiv.org/abs/2403.20266

Data contamination is a huge problem for LLM evals right now. At Scale, we created a new test set for GSM8k *from scratch* to measure overfitting and found evidence that some models (most notably Mistral and Phi) do substantially worse on this new test set compared to GSM8k.

good bye #ICLR2024 and congrats to organizers, they managed to make a 5000 conference a pleasure to attend!

The CONDA Data Contamination Workshop deadline was delayed to May 31st to give more buffer following ACL notifications - consider submitting if this gives you the chance to! conda-workshop.github.io

Our paper about Latxa has been accepted at #ACL2024NLP 🎉 Congratulations to all the coauthors and see you in Bangkok!

Today I presented Medical mT5 at LREC COLING 2024!!! We have released a lot of multilingual models, training data and evaluation benchmarks for the medical domain here, check them out!! huggingface.co/collections/Hi…

Back in the day, XLM-RoBERTa was too big for many people to run. It doesn't matter if you cannot run a 400B model today; in a few years, we will all laugh remembering how we thought that a "400B model was big" in the same way we laugh about how we thought that BERT was big.

Our colleagues accompanied by @orai_nlp and Vicomtech are working hard at #lreccolling2024

Reimagining table representation! In our new #ACL2024NLP paper we introduce PixT3: a family of image-based Table-to-Text Generation models that scale better at generating text from large tables, outperforming traditional text-based baselines. arxiv.org/abs/2311.09808

How Much Do Language Models Know About Local Culture? We introduce BertaQA, a multiple-choice trivia dataset parallel in English and Basque. It consists of a local subset about the Basque culture, and a global subset with questions of broader interest. arxiv.org/abs/2406.07302

Great work!!! It's nice to see GoLLIE being useful for the community 😀

Concerned about data contamination? We asked the community for known contamination in different datasets and models, and summarized these finding in this report. arxiv.org/pdf/2407.21530

Thanks to everyone who came to our poster session! It was nice to talk about Latxa. I gave out many stickers! #ACL2024NLP

I'll be discussing 'emergent properties' at 11am in this lovely workshop tomorrow. I found even more definitions for what this means during this ACL and also ICML!

There is now an LLM Leaderboard for one of the most spoken language worldwide: Spanish! 🚀 (+ Catalan, Basque and Galician) Congrats to María Grandury for setting it up, and to SomosNLP for gathering super high quality datasets from many partners! huggingface.co/spaces/la-lead…

El trabajo de HiTZ sobre Latxa ha recibido un premio internacional, aumentando el peso del euskera en la investigación. ehu.eus/es/-/centro-hi…

Además, Iker García-Ferrero ha presentado el dataset NoticIA dataset y sistemas para la detección de clickbaits

Join us in the First Shared Task on the Multilingual Counterspeech Generation against Hate Speech! To be held COLING 2025 Info: rb.gy/jfm481 - Aim: given a HS (and any additional knowledge the participants may like to use), generate a CN to counteract the HS.