Model, Task and Data Engineering for NLP
Assistant Professor, Founder of NTU-NLP Group
Nanyang Technological University (NTU), Singapore
ABSTRACT:
With the advent of deep learning and neural methods, NLP research over the last
decade has shifted from feature engineering to model engineering, primarily focusing on
inventing new architectures for NLP problems. Two other related factors that are getting
more attention only recently are: (i) which objectives (or tasks) to optimize, and (ii) how
to better use the available data; referred to as task engineering and data engineering,
respectively. In this talk, I will present our recent work along these three dimensions. In
particular, I will first present novel neural architectures for parsing texts into hierarchical
structures and efficient parallel encoding of such structures for better language
understanding and generation. I will then present a new objective for natural language
generation (NLG) tasks that aims to mitigate the degeneration issues prevalent in neural
generation models. Finally, I will present effective data augmentation methods for
supervised and unsupervised machine translation and other cross-lingual tasks. With
empirical results, I will argue that while model engineering is crucial to the advancement
of the field, the other two factors are more important to build robust NLP systems.
BIO-DATA:
Shafiq Joty is an Asst. Prof. in the School of Computer Science and Engineering
(SCSE) at NTU, where he leads the NTU-NLP group. He is also a senior manager of
NLP research and a founding member at Salesforce AI Research Asia. His work has
primarily focused on developing language analysis tools (e.g., syntactic parsers, NER,
discourse parser, coherence models) and downstream NLP applications including
machine translation, question answering, text summarization, image/video captioning
and visual question answering. A significant part of his current research focuses on
multilingual processing and robustness of NLP models. His work has mostly relied on
deep learning for better representation of the input text and on probabilistic graphical
models and reinforcement learning for capturing dependencies in the output. He served
(or will serve) as a (senior) area chair for ACL'19-21, EMNLP'19,21 and NAACL???21,
EACL???21, and a senior program committee member for AAAI???21 and IJCAI'19. He gave
tutorials at ACL-2019 and ICDM-2018. He was an associate editor for ACM
Transactions on Asian and Low Resource Language Processing. He has published
more than 95 papers in top-tier NLP/AI conferences and journals including ACL,
EMNLP, NAACL, NeurIPS, ICML, ICLR, CVPR, ECCV, ICCV, CL and JAIR.