CS Colloquium: Robin Jia (Stanford University) - Building Robust Natural Language Processing Systems
Tue, Feb 25, 2020
11:00 AM - 12:00 PM
Location: OHE 132
Speaker: Robin Jia, Stanford University
Talk Title: Building Robust Natural Language Processing Systems
Series: CS Colloquium
Abstract: While modern NLP systems have achieved outstanding performance on static benchmarks, they often fail catastrophically when presented with inputs from different sources or inputs that have been adversarial perturbed. This lack of robustness exposes troubling gaps in current models' understanding capabilities, and poses challenges for deployment of NLP systems in high-stakes situations. In this talk, I will demonstrate that building robust NLP systems requires reexamining all aspects of the current model building paradigm. First, I will show that adversarially constructed test data reveals vulnerabilities that are left unexposed by standard evaluation methods. Second, I will demonstrate that active learning, in which data is adaptively collected based on a model's current predictions, can significantly improve the ability of models to generalize robustly, compared to the use of static training datasets. Finally, I will show how to train NLP models to produce certificates of robustness---guarantees that for a given example and combinatorially large class of possible perturbations, no perturbation can cause a misclassification.
This lecture satisfies requirements for CSCI 591: Research Colloquium
Biography: Robin Jia is a sixth-year Ph.D. student at Stanford University advised by Percy Liang. His research interests lie broadly in building natural language processing systems that can generalize to unexpected test-time inputs. Robin's work has received an Outstanding Paper Award at EMNLP 2017 and a Best Short Paper Award at ACL 2018. He has been supported by an NSF Graduate Research Fellowship.
Host: Xiang Ren