Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering

Shiquan Yang, Xinting Huang, Jey Han Lau, Sarah Erfani


Abstract
Data artifacts incentivize machine learning models to learn non-transferable generalizations by taking advantage of shortcuts in the data, andthere is growing evidence that data artifacts play a role for the strong results that deep learning models achieve in recent natural language processing benchmarks.In this paper, we focus on task-oriented dialogue and investigate whether popular datasets such as MultiWOZ contain such data artifacts.We found that by only keeping frequent phrases in the trainingexamples, state-of-the-art models perform similarly compared to the variant trained with full data, suggesting they exploit these spurious correlationsto solve the task. Motivated by this, we propose a contrastive learning based framework to encourage the model to ignore these cues and focus on learning generalisable patterns. We also experiment with adversarial filtering to remove easy training instances so that the model would focus on learning from the harder instances. We conduct a number of generalization experiments — e.g., cross-domain/dataset and adversarial tests — to assess the robustness of our approach and found that it works exceptionally well.
Anthology ID:
2022.findings-emnlp.88
Volume:
Findings of the Association for Computational Linguistics: EMNLP 2022
Month:
December
Year:
2022
Address:
Abu Dhabi, United Arab Emirates
Editors:
Yoav Goldberg, Zornitsa Kozareva, Yue Zhang
Venue:
Findings
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
1220–1234
Language:
URL:
https://aclanthology.org/2022.findings-emnlp.88
DOI:
10.18653/v1/2022.findings-emnlp.88
Bibkey:
Cite (ACL):
Shiquan Yang, Xinting Huang, Jey Han Lau, and Sarah Erfani. 2022. Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 1220–1234, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Cite (Informal):
Robust Task-Oriented Dialogue Generation with Contrastive Pre-training and Adversarial Filtering (Yang et al., Findings 2022)
Copy Citation:
PDF:
https://aclanthology.org/2022.findings-emnlp.88.pdf
Video:
 https://aclanthology.org/2022.findings-emnlp.88.mp4