🔥The 9th Round of Easy Loan, Earn $40 Reward is in progress❗️ ⏰ Promotion Period: January 15th - Feburary 15th, 2025 👉 Register now and check more details at gate.io/campaigns/358

print("Hello world! 🎉") Excited to announce the BigCode project led by ServiceNow Research and Hugging Face! In the spirit of BigScience we aim to develop large language models for code in an open and responsible way. Join here: bigcode-project.org/docs/about/joi… A thread with our goals🧵

The BigScience Research Workshop carbon footprint paper is live!! 🎉 Check it out to see how we calculated BLOOM's carbon footprint, covering all steps from the manufacturing of equipment 💻 to deployment! 🚀 arxiv.org/abs/2211.02001

Crosslingual Generalization through Multitask Finetuning 🌸 Demo: huggingface.co/bigscience/blo… 📜 arxiv.org/abs/2211.01786 💻github.com/bigscience-wor… We present BLOOMZ & mT0, a family of models w/ up to 176B params that follow human instructions in >100 languages zero-shot. 1/7

Big day today with two papers out! BLOOM carbon footprint at arxiv.org/abs/2211.02001, new models BLOOMZ and mt0 at huggingface.co/bigscience/blo…

The Bloom paper is out. Looks like it's doing worse than current GPT3 API in zero-shot generation tasks in English but better than other open-source LLMs & better than all in zs multi-lingual (which was the main goal). Proud of the work from the community! arxiv.org/abs/2211.05100

Petals, a system for easy decentralized inference and adaptation of 100B+ LLMs, is now online! 🌸Generate text with BLOOM-176B using Colab or a desktop GPU 🔌Fine-tune large models for your tasks 👥Help others by contributing your GPUs or host a new swarm colab.research.google.com/drive/1Ervk6HP…

(Repost for corrected Arxiv) 🧐What’s the best way to quickly adapt large multilingual language models to new languages? We present our new paper from BigScience Research Workshop 🌸: BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting. 📜 arxiv.org/abs/2212.09535 [1/9]

Worried about benchmark data contamination? Studying LLM memorization or attribution? BigScience Research Workshop BLOOM 🌸 now has exact & fuzzy search over full training data! with Ola Piktus🏆 Christopher Akiki Paulo Villegas Hugo Laurençon @ggdupont Sasha Luccioni, PhD 🦋🌎✨🤗 Yacine Jernite arxiv.org/abs/2302.14035 /1

The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset Documents the data creation and curation efforts of ROOTS corpus, a 1.6TB dataset used to train BLOOM Releases a large initial subset of the corpus data: huggingface.co/bigscience-data abs: arxiv.org/abs/2303.03915

As you already know, I am very proud of the collective work that enabled the development of BigScience Research Workshop's ethical charter. Today I am even more proud to announce that it's part of OECD Innovation's catalog to promote Trustworthy AI: such a milestone! oecd.ai/en/catalogue/t…

Join us tomorrow, Wednesday 22nd (6:30 PM - 8:00PM CET) at the Mozilla Festival Science Fair to learn more about our work in the open and responsible development of large language models (LLMs) for code. schedule.mozillafestival.org/session/TJRU3L… #Mozfest

Introducing: 💫StarCoder StarCoder is a 15B LLM for code with 8k context and trained only on permissive data in 80+ programming languages. It can be prompted to reach 40% pass@1 on HumanEval and act as a Tech Assistant. Try it here: shorturl.at/cYZ06r Release thread🧵

If you wanted to see the fun panel/Q&A we did with Londoners on AI, you can check out the recording here! My preso at the start is also on Open Science, representing Hugging Face & BigScience Research Workshop.

Never thought I'd see the day I'd have a publication in JMLR 🥹 So happy that the BLOOM carbon footprint paper has finally found a home at such an incredible venue! Thank you Shakir Mohamed for being such a great editor, it warms my heart to see your name on this paper 💚

I respect the caution, but also need to stress that efforts that pursue transparency as an operational value in service of actual inclusion and accountability do exist - see for example the writing on this very topic by BigScience Research Workshop, including its ethical charter. 1/3

The top 15 most-liked organizations on Hugging Face 1. Stability AI 20k likes 2. AI at Meta 20k 3. Runway 11k 4. CompVis 10k 5. Tsinghua KEG (THUDM) 7k 6. BigScience Research Workshop 7k 7. Technology Innovation Institute 7k 8. Microsoft 6.5k 9. Google AI 6k 10. OpenAI 4k 11. BigCode 4k 12. . 4k 13. UKP Lab 3k

The Universal Checkpointing paper is out! arxiv.org/abs/2406.18820 If you remember the BigScience Research Workshop BLOOM-176B training, Tunji Ruwase and I co-invented this technology for Megatron-Deepspeed in order to enable to quickly scale up and down node topology while continuing training.

DPhil candidate @cailean_osborne shares reflections on the Open Source Initiative @[email protected] co-design process to define #opensourceAI and recommends next steps, including improving model safety and supporting more grassroots initiatives like BigScience Research Workshop.

Doesn't get enough credit but IMO paved the way for open-source LLMs!

🌸❤️