Glad I am for the first time an instructor for two projects topics for 13 students of computer science at QualityUsabilityLab: "Towards automatic #SignLanguage translation" & "Fine grained evaluation of machine #translation". DFKI TU Berlin #NLProc

(a) Better Human Evaluation: Like last year, our primary human scores will be derived from expert-based MQM annotations. (b) We have 3 new subtasks: (i) QE as a Metric, (ii) Challenge Sets, (iii) MBR Decoding. (2/n)

Similar to last year, we will not only evaluate the performance of metrics on the WMT submissions, but also augment the set of systems with additional MT systems generated by methods like sampling or MBR decoding. (3/n)

For the Challenge Sets task we invite anyone to submit a new test suite and/or an analysis paper on metric behaviour for specific perturbations/phenomena similar to what we did last year! (4/n)

Please check out all details regarding both the main task and all subtasks: wmt-metrics-task.github.io (5/n)

We welcome any additional comments and suggestions from the community regarding these evaluation plans and will take all such feedback into consideration. (6/n)

Also check our last year’s metric task paper: aclanthology.org/2021.wmt-1.73/ . There are a lot of goodies: (7/n)

Panel discussion #EMNLP2022 #emnlp

Best long paper award at #EMNLP2022 on #tangrams!

Excited that through our DFKI #research project we can work on the automatic translation of sign languages. Our first experiments were shown with a technical report at the 7th Conference on #MachineTranslation, in #EMNLP2022. Still long way to go. #nlproc dfki.de/en/web/researc…

Hello everyone! The Automatic Translation for Signed and Spoken Languages (AT4SSL) Workshop will be part of the @EAMT_2023, in Tampere, this year! If you are working on topics related to MT for Signed and Spoken Languages, have a look at our CfP: sites.google.com/tilburgunivers…

My colleague was looking for citations on Bing Chat and got as a response a former non-published document of his with the same title but different made-up authors! #BingChat #ChatGPT

Hello everyone! The deadline for submitting a paper to our workshop has been extended until April 24th. (Which means more time to work on great papers!) We're looking forward to all your contributions and to seeing you at @EAMT_2023 in Tampere! For info: sites.google.com/tilburgunivers…

Really delighted that the #wearable Textile Game Controller of the DFKI department of Interactive Textiles was presented by Esther Zahn and Hannah Fischer at the Interactivity Section of #CHI2023 in Hamburg

The “Test suites” sub-task will be included for the sixth time in the General MT Shared Task of the Conference on Machine Translation (WMT23). Submission deadline: June 19th - more info: www2.statmt.org/wmt23/testsuit… #NLProc #EMNLP2023 #EMNLP #ACL

Meta evaluation of metrics usually does not get much attention. Dan's paper clearly shows that it is important to understand what your meta eval actually measures. Highly recommend to read! Really insightful work! Especially if you think about doing evaluation with LLMs.

traveling to the #ICASSP2023 #conference with to discuss at #SLTAT2023 the #design of our DFKI prototypical #AI #SignLanguage #translation tool employing lightweight #AugmentedReality and smart headphones. dfki.de/en/web/researc…

Happy to announce the WMT23 Metrics Shared Task! New this year amongst other changes: paragraph-level metric task for En-De & a new language pair: Hebrew-English. wmt-metrics-task.github.io #emnlp #NLProc #machinetranslation (1/n)

Our #ACL2023 paper on "Neural #MachineTranslation for Translating Text to #SignLanguage Glosses" was named an outstanding paper! A result of the excellent MSc thesis of Dele Yvan Zhu at TU Berlin with the invaluable help of Vera Czehmann. #NLProc DFKI aclanthology.org/2023.acl-long.…

Help us break LLMs! The test suite sub-task will be included for the sixth time in the General MT Shared Task of the Conference on Machine Translation (WMT24). This year's theme is to reveal weaknesses of LLMs when translating #LLMs #EMNLP #NLProc| www2.statmt.org/wmt24/testsuit…