Check out our new work at #ICML2022

We are happy to share that the paper submission deadline for the 13th edition of the ‘Indian Conference on Computer Vision, Graphics and Image Processing’ (#ICVGIP) has been extended till August 21. Website: bit.ly/3wD4yCR Shanmuganathan Raman Vineet Vashista udit bhatia

We at MIT-IBM Watson AI Lab, are currently looking to hire PhD candidates for 2023 summer internship, to work on efficient training and inference of large language (and/or vision) models. Please DM or send me an email if you are interested.

Happy to share that 3 papers on Efficient AI accepted to ICLR 2026. One as a "notable-top-25%" paper (Spotlight). Huge thanks to all my co-authors. Stay tuned for more details! Work MIT-IBM Watson AI Lab #ICLR2023

We at MIT-IBM Watson AI Lab, are currently looking for a research software engineer, to work on efficient large language models, and develop prototype solutions to real-world problems, while publishing papers in top AI conferences. Apply at: krb-sjobs.brassring.com/TGnewUI/Search… #NLP #efficiency #LLMs

We at MIT-IBM Watson AI Lab, are currently looking for a senior AI researcher, to work on efficient large language models, and develop prototype solutions to real-world problems, while publishing papers in top AI conferences. Apply at: careers.ibm.com/job/18637769/s… #NLP #efficiency #LLMs #AI

Our team is currently looking to hire PhD candidates for 2024 summer internship, to work on efficient training and inference of large language (and/or multimodal) models. Please DM or send me an email if you are interested. MIT-IBM Watson AI Lab IBM Research

Yes, our goal is to create really useful code LLMs for real production use cases, not for just getting some kind of sota on HumanEval (but we still get it 😉).

JetMoE and IBM Granite Code models are now natively available on in Huggingface Transformers v4.41! github.com/huggingface/tr…

Reducing Transformer Key-Value Cache Size with Cross-Layer Attention Shows that it is possible to share key and value heads between adjacent layers w/o perf degradation arxiv.org/abs/2405.12981

We have released 4-bit GGUF versions of all Granite Code models for local inference. 💻 The models can be found here: huggingface.co/collections/ib…

Thanks for posting our work! (1/5) After running thousands of experiments with the WSD learning rate scheduler and μTransfer, we found that the optimal learning rate strongly correlates with the batch size and the number of tokens.

Apply soon if you’re interested in joining a small, growing team to make a large impact together!!

🚨Hiring🚨 We are looking for research scientists and engineers to join IBM Research (Cambridge, Bangalore). We train large language models and do fundamental research on directions related to LLMs. Please DM me your CV and a brief introduction of yourself if you are interested!

📢 (1/16) Introducing PaTH 🛣️ — a RoPE-free contextualized position encoding scheme, built for stronger state tracking, better extrapolation, and hardware-efficient training. PaTH outperforms RoPE across short and long language modeling benchmarks arxiv.org/abs/2505.16381

If you want to fine-tune the Granite 4.0 MoE models, Unsloth has a ready-to-go recipe here! If you're gonna roll your own, I've updated scattermoe to inject a forward pass to the Huggingface implementation that uses scattermoe. github.com/shawntan/scatt…

Amazing work from Zhangchen Xu!