Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor 2020-11 Association for Computational Linguistics Online text conference publication blackboxnlp-2020-blackboxnlp 10.18653/v1/2020.blackboxnlp-1.0 https://aclanthology.org/2020.blackboxnlp-1.0/ BERTering RAMS: What and How Much does BERT Already Know About Event Arguments? - A Study on the RAMS Dataset Varun Gangal author Eduard Hovy author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication gangal-hovy-2020-bertering 10.18653/v1/2020.blackboxnlp-1.1 https://aclanthology.org/2020.blackboxnlp-1.1/ 2020-11 1 10 Emergent Language Generalization and Acquisition Speed are not tied to Compositionality Eugene Kharitonov author Marco Baroni author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication kharitonov-baroni-2020-emergent 10.18653/v1/2020.blackboxnlp-1.2 https://aclanthology.org/2020.blackboxnlp-1.2/ 2020-11 11 15 Examining the rhetorical capacities of neural language models Zining Zhu author Chuer Pan author Mohamed Abdalla author Frank Rudzicz author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication zhu-etal-2020-examining 10.18653/v1/2020.blackboxnlp-1.3 https://aclanthology.org/2020.blackboxnlp-1.3/ 2020-11 16 32 What Happens To BERT Embeddings During Fine-tuning? Amil Merchant author Elahe Rahimtoroghi author Ellie Pavlick author Ian Tenney author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication merchant-etal-2020-happens 10.18653/v1/2020.blackboxnlp-1.4 https://aclanthology.org/2020.blackboxnlp-1.4/ 2020-11 33 44 It’s not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERT Hila Gonen author Shauli Ravfogel author Yanai Elazar author Yoav Goldberg author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication gonen-etal-2020-greek 10.18653/v1/2020.blackboxnlp-1.5 https://aclanthology.org/2020.blackboxnlp-1.5/ 2020-11 45 56 Leveraging Extracted Model Adversaries for Improved Black Box Attacks Naveen Jafer Nizar author Ari Kobren author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication nizar-kobren-2020-leveraging 10.18653/v1/2020.blackboxnlp-1.6 https://aclanthology.org/2020.blackboxnlp-1.6/ 2020-11 57 67 On the Interplay Between Fine-tuning and Sentence-Level Probing for Linguistic Knowledge in Pre-Trained Transformers Marius Mosbach author Anna Khokhlova author Michael A Hedderich author Dietrich Klakow author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication mosbach-etal-2020-interplay 10.18653/v1/2020.blackboxnlp-1.7 https://aclanthology.org/2020.blackboxnlp-1.7/ 2020-11 68 82 Unsupervised Evaluation for Question Answering with Transformers Lukas Muttenthaler author Isabelle Augenstein author Johannes Bjerva author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication muttenthaler-etal-2020-unsupervised 10.18653/v1/2020.blackboxnlp-1.8 https://aclanthology.org/2020.blackboxnlp-1.8/ 2020-11 83 90 Unsupervised Distillation of Syntactic Information from Contextualized Word Representations Shauli Ravfogel author Yanai Elazar author Jacob Goldberger author Yoav Goldberg author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication ravfogel-etal-2020-unsupervised 10.18653/v1/2020.blackboxnlp-1.9 https://aclanthology.org/2020.blackboxnlp-1.9/ 2020-11 91 106 The Explanation Game: Towards Prediction Explainability through Sparse Communication Marcos Treviso author André F T Martins author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication treviso-martins-2020-explanation 10.18653/v1/2020.blackboxnlp-1.10 https://aclanthology.org/2020.blackboxnlp-1.10/ 2020-11 107 118 Latent Tree Learning with Ordered Neurons: What Parses Does It Produce? Yian Zhang author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication zhang-2020-latent 10.18653/v1/2020.blackboxnlp-1.11 https://aclanthology.org/2020.blackboxnlp-1.11/ 2020-11 119 125 Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets Chuanrong Li author Lin Shengshuo author Zeyu Liu author Xinyi Wu author Xuhui Zhou author Shane Steinert-Threlkeld author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication li-etal-2020-linguistically 10.18653/v1/2020.blackboxnlp-1.12 https://aclanthology.org/2020.blackboxnlp-1.12/ 2020-11 126 135 Controlling the Imprint of Passivization and Negation in Contextualized Representations Hande Celikkanat author Sami Virpioja author Jörg Tiedemann author Marianna Apidianaki author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication celikkanat-etal-2020-controlling 10.18653/v1/2020.blackboxnlp-1.13 https://aclanthology.org/2020.blackboxnlp-1.13/ 2020-11 136 148 The elephant in the interpretability room: Why use attention as explanation when we have saliency methods? Jasmijn Bastings author Katja Filippova author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication bastings-filippova-2020-elephant 10.18653/v1/2020.blackboxnlp-1.14 https://aclanthology.org/2020.blackboxnlp-1.14/ 2020-11 149 155 How does BERT capture semantics? A closer look at polysemous words David Yenicelik author Florian Schmidt author Yannic Kilcher author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication yenicelik-etal-2020-bert 10.18653/v1/2020.blackboxnlp-1.15 https://aclanthology.org/2020.blackboxnlp-1.15/ 2020-11 156 162 Neural Natural Language Inference Models Partially Embed Theories of Lexical Entailment and Negation Atticus Geiger author Kyle Richardson author Christopher Potts author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication geiger-etal-2020-neural 10.18653/v1/2020.blackboxnlp-1.16 https://aclanthology.org/2020.blackboxnlp-1.16/ 2020-11 163 173 BERTnesia: Investigating the capture and forgetting of knowledge in BERT Jonas Wallat author Jaspreet Singh author Avishek Anand author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication singh-etal-2020-bertnesia 10.18653/v1/2020.blackboxnlp-1.17 https://aclanthology.org/2020.blackboxnlp-1.17/ 2020-11 174 183 Probing for Multilingual Numerical Understanding in Transformer-Based Language Models Devin Johnson author Denise Mak author Andrew Barker author Lexi Loessberg-Zahl author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication johnson-etal-2020-probing 10.18653/v1/2020.blackboxnlp-1.18 https://aclanthology.org/2020.blackboxnlp-1.18/ 2020-11 184 192 Dissecting Lottery Ticket Transformers: Structural and Behavioral Study of Sparse Neural Machine Translation Rajiv Movva author Jason Zhao author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication movva-zhao-2020-dissecting 10.18653/v1/2020.blackboxnlp-1.19 https://aclanthology.org/2020.blackboxnlp-1.19/ 2020-11 193 203 Exploring Neural Entity Representations for Semantic Information Andrew Runge author Eduard Hovy author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication runge-hovy-2020-exploring 10.18653/v1/2020.blackboxnlp-1.20 https://aclanthology.org/2020.blackboxnlp-1.20/ 2020-11 204 216 BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performance R Thomas McCoy author Junghyun Min author Tal Linzen author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication mccoy-etal-2020-berts 10.18653/v1/2020.blackboxnlp-1.21 https://aclanthology.org/2020.blackboxnlp-1.21/ 2020-11 217 227 Second-Order NLP Adversarial Examples John Morris author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication morris-2020-second 10.18653/v1/2020.blackboxnlp-1.22 https://aclanthology.org/2020.blackboxnlp-1.22/ 2020-11 228 237 Discovering the Compositional Structure of Vector Representations with Role Learning Networks Paul Soulos author R Thomas McCoy author Tal Linzen author Paul Smolensky author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication soulos-etal-2020-discovering 10.18653/v1/2020.blackboxnlp-1.23 https://aclanthology.org/2020.blackboxnlp-1.23/ 2020-11 238 254 Structured Self-Attention Weights Encode Semantics in Sentiment Analysis Zhengxuan Wu author Thanh-Son Nguyen author Desmond C Ong author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication wu-etal-2020-structured 10.18653/v1/2020.blackboxnlp-1.24 https://aclanthology.org/2020.blackboxnlp-1.24/ 2020-11 255 264 Investigating Novel Verb Learning in BERT: Selectional Preference Classes and Alternation-Based Syntactic Generalization Tristan Thrush author Ethan Wilcox author Roger Levy author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication thrush-etal-2020-investigating 10.18653/v1/2020.blackboxnlp-1.25 https://aclanthology.org/2020.blackboxnlp-1.25/ 2020-11 265 275 The EOS Decision and Length Extrapolation Benjamin Newman author John Hewitt author Percy Liang author Christopher D Manning author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication newman-etal-2020-eos 10.18653/v1/2020.blackboxnlp-1.26 https://aclanthology.org/2020.blackboxnlp-1.26/ 2020-11 276 291 Do Language Embeddings capture Scales? Xikun Zhang author Deepak Ramachandran author Ian Tenney author Yanai Elazar author Dan Roth author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication zhang-etal-2020-language 10.18653/v1/2020.blackboxnlp-1.27 https://aclanthology.org/2020.blackboxnlp-1.27/ 2020-11 292 299 Evaluating Attribution Methods using White-Box LSTMs Yiding Hao author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication hao-2020-evaluating 10.18653/v1/2020.blackboxnlp-1.28 https://aclanthology.org/2020.blackboxnlp-1.28/ 2020-11 300 313 Defining Explanation in an AI Context Tejaswani Verma author Christoph Lingenfelder author Dietrich Klakow author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication verma-etal-2020-defining 10.18653/v1/2020.blackboxnlp-1.29 https://aclanthology.org/2020.blackboxnlp-1.29/ 2020-11 314 322 Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples Jin Yong Yoo author John Morris author Eli Lifland author Yanjun Qi author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication yoo-etal-2020-searching 10.18653/v1/2020.blackboxnlp-1.30 https://aclanthology.org/2020.blackboxnlp-1.30/ 2020-11 323 332 This is a BERT. Now there are several of them. Can they generalize to novel words? Coleman Haley author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication haley-2020-bert 10.18653/v1/2020.blackboxnlp-1.31 https://aclanthology.org/2020.blackboxnlp-1.31/ 2020-11 333 341 diagNNose: A Library for Neural Activation Analysis Jaap Jumelet author 2020-11 text Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Afra Alishahi editor Yonatan Belinkov editor Grzegorz Chrupała editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Online conference publication jumelet-2020-diagnnose 10.18653/v1/2020.blackboxnlp-1.32 https://aclanthology.org/2020.blackboxnlp-1.32/ 2020-11 342 350