10 Tips for Research and a PhD I've been asked in the past to provide advice on doing research. Here are 10 tips that worked well for me and will hopefully also be useful to others. ruder.io/10-tips-for-re…

Our paper "Sparse Sinkhorn Attention" is accepted to #ICML2020! 😃 In this paper, we propose efficient self-attention via differentiable sorting. Joint work with Dara Bahri @liuyangumass Don Metzler Da-Cheng Juan Preprint: arxiv.org/abs/2002.11296

Excited to share a video describing our approach to learn hyperbolic Knowledge Graph embeddings! youtube.com/watch?v=Yf03-C… Thanks to my amazing collaborators! Adva Wolf Da-Cheng Juan Fred Sala Sujith Ravi hazyresearch

Join us at KDD on Tuesday, 9 AM - 12 PM Pacific time. We’re organizing a hands-on tutorial on Neural Structured Learning. Tutorial outline at github.com/tensorflow/neu…. Find us on whova or Vfairs. #KDD2020 Da-Cheng Juan SIGKDD 2025

Inspired by the dizzying number of efficient Transformers ("x-formers") models that are coming out lately, we wrote a survey paper to organize all this information. Check it out at arxiv.org/abs/2009.06732. Joint work with Mostafa Dehghani Dara Bahri and Don Metzler. Google AI 😀😃

📊 Neural Structured Learning in TFX. In Arjun Gopalan's new blog, you'll learn how graph regularization can be implemented using custom TFX components and how adversarial learning can be deployed seamlessly with TFX. Read more ↓ blog.tensorflow.org/2020/10/neural…

As a companion to our recent efficient Transformer survey, we designed "Long Range Arena" a new challenging benchmark to help understand and analyze trade-offs between recent efficient Transformer models. Check out our paper at arxiv.org/abs/2011.04006. Google AI @DeepMind

[#NeurIPS2020 Talk] Mitigating Forgetting in Online Continual Learning via Instance-Aware Parameterization by researchers from NTHU Google AI An-Chieh Cheng Da-Cheng Juan #paperwithvideo #machinelearning #neuralnetwork #deeplearning #convolution bit.ly/3nE3veY

I have a new paper on how to represent part-whole hierarchies in neural networks. arxiv.org/abs/2102.12627

Excited to share that our Synthesizer paper was recently accepted to #ICML2021. Random Synthesizers are actually "All-MLP" architectures and MLP-Mixers are a form of Random Synthesizers (or vice versa)! preprint: arxiv.org/abs/2005.00743 code: github.com/tensorflow/mesh

#Pathways is a new #ML architecture that will let us train a single model to accomplish thousands or millions of tasks, across modalities - while consuming much less energy than typical models do today. Learn more at goo.gle/3jHFST5, and in the TED Talk below.

I love $REZ!

I love this

About to claim