Thanks! We will keep updating this repo. Any feedback is welcome:)

This guy is looking for a PhD position. Help him😂

I recently gave a tea talk on graph representation learning at milamontreal, which summarized some of the most important work my group have done . I started to work on this back to 2013 and published the first paper LINE. Slides are available at: github.com/tangjianpku/ta…

Welcome to the TsinghuaNLP Twitter feed, where we'll share new researches and information from TsinghuaNLP Group. Looking forward to interacting with you here! #NLProc

KEPLER (arxiv.org/pdf/1911.06136…) is a unified model for knowledge embedding (KE) and pre-trained language model (PLM) representation. It encodes textual entity descriptions with a PLM and then jointly optimize KE and language modeling objectives. #NLProc #AI #TsinghuaNLP

MAVEN is a massive general domain event detection dataset, which contains 4,480 Wikipedia documents, 118,732 event mentions, and 168 event types. Check out our EMNLP 2020 paper (arxiv.org/abs/2004.13590). #NLProc #AI #TsinghuaNLP #EMNLP

HMEAE is an event argument extraction model. It provides effective inductive bias from the concept hierarchy of event argument roles by using a neural module network to imitate the hierarchical structure of concepts. #NLProc #AI #TsinghuaNLP #EMNLP Paper: aclweb.org/anthology/D19-…

Our ACL 2021 paper proposed a contrastive pre-training framework CLEVE for event extraction task, utilizing AMR structures to make PLMs better understand semantic relations between triggers and arguments. Our paper: arxiv.org/abs/2105.14485. #NLProc #ACL2021 #AI #TsinghuaNLP

Wanna tune few-shot NLP tasks with 5 free parameters😉? Check our recent preprint: Exploring Low-dimensional Intrinsic Task Subspace via Prompt Tuning. We explore how can PLMs effectively adapt to broad NLP tasks differing a lot superficially. arxiv.org/abs/2110.07867

🎉Thrilled to introduce our latest work, Delta Tuning: A Comprehensive Study of Parameter Efficient Methods for Pre-trained Language Models. We perform a comprehensive theoretical analysis and experimental validation of the parametrically efficient paradigm for #PLMs.

Welcome to follow our #NAACL-HLT 2022 paper. We empirically investigate the transferability of soft prompts across tasks/PLMs and introduce a new transferability indicator. #NLProc #AI #TsinghuaNLP (1/4) Paper：aclanthology.org/2022.naacl-mai… Data/Code: github.com/thunlp/Prompt-…

Come and try! #nlproc

Thanks! KoLA will be continually updated. Our second season is now open for participation. Stay tuned!

1/5 🚀 Excited to share our latest paper on Configurable Foundation Models! 🧠 Inspired by the human brain's functional specialization, we propose a concept: Configurable Foundation Model, a modular approach to LLMs.

📢Our 2nd Knowledgeable Foundation Model workshop will be at AAAI 25! Submission Deadline: Dec 1st Thanks to the wonderful organizer team Zoey Sha Li Mor Geva Chi Han Xiaozhi Wang Shangbin Feng @silingao and advising committee Heng Ji Isabelle Augenstein Mohit Bansal !

Welcome for submission!

1/4 🚀 Densing Law of LLMs 🚀 OpenAI's Scaling Law showed how model capabilities scale with size. But what about the trend toward efficient models? 🤔 We introduce "capacity density" and found an exciting empirical law: LLMs' capacity density grows EXPONENTIALLY over time!

Our brand new long-context benchmark. Check it!

Check our new work on Agentic RM

🚀 MiniCPM4 is here! 5x faster on end devices 🔥 ✨ What's new: 🏗️ Efficient Model Architecture - InfLLM v2 -- Trainable Sparse Attention Mechanism 🧠 Efficient Learning Algorithms - Model Wind Tunnel 2.0 -- Efficient Predictable Scaling - BitCPM -- Ultimate Ternary Quantization