Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP

Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor 2021-11 Association for Computational Linguistics Punta Cana, Dominican Republic text conference publication blackboxnlp-2021-blackboxnlp https://aclanthology.org/2021.blackboxnlp-1.0/ To what extent do human explanations of model behavior align with actual model behavior? Grusha Prasad author Yixin Nie author Mohit Bansal author Robin Jia author Douwe Kiela author Adina Williams author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication prasad-etal-2021-extent 10.18653/v1/2021.blackboxnlp-1.1 https://aclanthology.org/2021.blackboxnlp-1.1/ 2021-11 1 14 Test Harder than You Train: Probing with Extrapolation Splits Jenny Kunz author Marco Kuhlmann author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication kunz-kuhlmann-2021-test 10.18653/v1/2021.blackboxnlp-1.2 https://aclanthology.org/2021.blackboxnlp-1.2/ 2021-11 15 25 Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human Ratings Hendrik Schuff author Hsiu-Yu Yang author Heike Adel author Ngoc Thang Vu author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication schuff-etal-2021-external 10.18653/v1/2021.blackboxnlp-1.3 https://aclanthology.org/2021.blackboxnlp-1.3/ 2021-11 26 41 The Language Model Understood the Prompt was Ambiguous: Probing Syntactic Uncertainty Through Generation Laura Aina author Tal Linzen author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication aina-linzen-2021-language 10.18653/v1/2021.blackboxnlp-1.4 https://aclanthology.org/2021.blackboxnlp-1.4/ 2021-11 42 57 On the Limits of Minimal Pairs in Contrastive Evaluation Jannis Vamvas author Rico Sennrich author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication vamvas-sennrich-2021-limits 10.18653/v1/2021.blackboxnlp-1.5 https://aclanthology.org/2021.blackboxnlp-1.5/ 2021-11 58 68 What Models Know About Their Attackers: Deriving Attacker Information From Latent Representations Zhouhang Xie author Jonathan Brophy author Adam Noack author Wencong You author Kalyani Asthana author Carter Perkins author Sabrina Reis author Zayd Hammoudeh author Daniel Lowd author Sameer Singh author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication xie-etal-2021-models 10.18653/v1/2021.blackboxnlp-1.6 https://aclanthology.org/2021.blackboxnlp-1.6/ 2021-11 69 78 ALL Dolphins Are Intelligent and SOME Are Friendly: Probing BERT for Nouns’ Semantic Properties and their Prototypicality Marianna Apidianaki author Aina Garí Soler author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication apidianaki-gari-soler-2021-dolphins 10.18653/v1/2021.blackboxnlp-1.7 https://aclanthology.org/2021.blackboxnlp-1.7/ 2021-11 79 94 ProSPer: Probing Human and Neural Network Language Model Understanding of Spatial Perspective Tessa Masis author Carolyn Jane Anderson author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication masis-anderson-2021-prosper 10.18653/v1/2021.blackboxnlp-1.8 https://aclanthology.org/2021.blackboxnlp-1.8/ 2021-11 95 135 Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCAN Rahma Chaabouni author Roberto Dessì author Eugene Kharitonov author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication chaabouni-etal-2021-transformers 10.18653/v1/2021.blackboxnlp-1.9 https://aclanthology.org/2021.blackboxnlp-1.9/ 2021-11 136 148 Transferring Knowledge from Vision to Language: How to Achieve it and how to Measure it? Tobias Norlund author Lovisa Hagström author Richard Johansson author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication norlund-etal-2021-transferring 10.18653/v1/2021.blackboxnlp-1.10 https://aclanthology.org/2021.blackboxnlp-1.10/ 2021-11 149 162 Discrete representations in neural models of spoken language Bertrand Higy author Lieke Gelderloos author Afra Alishahi author Grzegorz Chrupała author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication higy-etal-2021-discrete 10.18653/v1/2021.blackboxnlp-1.11 https://aclanthology.org/2021.blackboxnlp-1.11/ 2021-11 163 176 Word Equations: Inherently Interpretable Sparse Word Embeddings through Sparse Coding Adly Templeton author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication templeton-2021-word 10.18653/v1/2021.blackboxnlp-1.12 https://aclanthology.org/2021.blackboxnlp-1.12/ 2021-11 177 191 A howling success or a working sea? Testing what BERT knows about metaphors Paolo Pedinotti author Eliana Di Palma author Ludovica Cerini author Alessandro Lenci author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication pedinotti-etal-2021-howling 10.18653/v1/2021.blackboxnlp-1.13 https://aclanthology.org/2021.blackboxnlp-1.13/ 2021-11 192 204 How Length Prediction Influence the Performance of Non-Autoregressive Translation? Minghan Wang author Guo Jiaxin author Yuxia Wang author Yimeng Chen author Su Chang author Hengchao Shang author Min Zhang author Shimin Tao author Hao Yang author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication wang-etal-2021-length 10.18653/v1/2021.blackboxnlp-1.14 https://aclanthology.org/2021.blackboxnlp-1.14/ 2021-11 205 213 On the Language-specificity of Multilingual BERT and the Impact of Fine-tuning Marc Tanti author Lonneke van der Plas author Claudia Borg author Albert Gatt author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication tanti-etal-2021-language 10.18653/v1/2021.blackboxnlp-1.15 https://aclanthology.org/2021.blackboxnlp-1.15/ 2021-11 214 227 Relating Neural Text Degeneration to Exposure Bias Ting-Rui Chiang author Yun-Nung Chen author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication chiang-chen-2021-relating 10.18653/v1/2021.blackboxnlp-1.16 https://aclanthology.org/2021.blackboxnlp-1.16/ 2021-11 228 239 Efficient Explanations from Empirical Explainers Robert Schwarzenberg author Nils Feldhus author Sebastian Möller author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication schwarzenberg-etal-2021-efficient 10.18653/v1/2021.blackboxnlp-1.17 https://aclanthology.org/2021.blackboxnlp-1.17/ 2021-11 240 249 Variation and generality in encoding of syntactic anomaly information in sentence embeddings Qinxuan Wu author Allyson Ettinger author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication wu-ettinger-2021-variation 10.18653/v1/2021.blackboxnlp-1.18 https://aclanthology.org/2021.blackboxnlp-1.18/ 2021-11 250 264 Enhancing Interpretable Clauses Semantically using Pretrained Word Representation Rohan Kumar Yadav author Lei Jiao author Ole-Christoffer Granmo author Morten Goodwin author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication yadav-etal-2021-enhancing 10.18653/v1/2021.blackboxnlp-1.19 https://aclanthology.org/2021.blackboxnlp-1.19/ 2021-11 265 274 Analyzing BERT’s Knowledge of Hypernymy via Prompting Michael Hanna author David Mareček author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication hanna-marecek-2021-analyzing 10.18653/v1/2021.blackboxnlp-1.20 https://aclanthology.org/2021.blackboxnlp-1.20/ 2021-11 275 282 An in-depth look at Euclidean disk embeddings for structure preserving parsing Federico Fancellu author Lan Xiao author Allan Jepson author Afsaneh Fazly author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication fancellu-etal-2021-depth 10.18653/v1/2021.blackboxnlp-1.21 https://aclanthology.org/2021.blackboxnlp-1.21/ 2021-11 283 295 Training Dynamic based data filtering may not work for NLP datasets Arka Talukdar author Monika Dagar author Prachi Gupta author Varun Menon author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication talukdar-etal-2021-training 10.18653/v1/2021.blackboxnlp-1.22 https://aclanthology.org/2021.blackboxnlp-1.22/ 2021-11 296 302 Multi-Layer Random Perturbation Training for improving Model Generalization Efficiently Lis Kanashiro Pereira author Yuki Taya author Ichiro Kobayashi author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication kanashiro-pereira-etal-2021-multi 10.18653/v1/2021.blackboxnlp-1.23 https://aclanthology.org/2021.blackboxnlp-1.23/ 2021-11 303 310 Screening Gender Transfer in Neural Machine Translation Guillaume Wisniewski author Lichao Zhu author Nicolas Ballier author François Yvon author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication wisniewski-etal-2021-screening 10.18653/v1/2021.blackboxnlp-1.24 https://aclanthology.org/2021.blackboxnlp-1.24/ 2021-11 311 321 What BERT Based Language Model Learns in Spoken Transcripts: An Empirical Study Ayush Kumar author Mukuntha Narayanan Sundararaman author Jithendra Vepa author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication kumar-etal-2021-bert 10.18653/v1/2021.blackboxnlp-1.25 https://aclanthology.org/2021.blackboxnlp-1.25/ 2021-11 322 336 Assessing the Generalization Capacity of Pre-trained Language Models through Japanese Adversarial Natural Language Inference Hitomi Yanaka author Koji Mineshima author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication yanaka-mineshima-2021-assessing 10.18653/v1/2021.blackboxnlp-1.26 https://aclanthology.org/2021.blackboxnlp-1.26/ 2021-11 337 349 Investigating Negation in Pre-trained Vision-and-language Models Radina Dobreva author Frank Keller author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication dobreva-keller-2021-investigating 10.18653/v1/2021.blackboxnlp-1.27 https://aclanthology.org/2021.blackboxnlp-1.27/ 2021-11 350 362 Not all parameters are born equal: Attention is mostly what you need Nikolay Bogoychev author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication bogoychev-2021-parameters 10.18653/v1/2021.blackboxnlp-1.28 https://aclanthology.org/2021.blackboxnlp-1.28/ 2021-11 363 374 Not All Models Localize Linguistic Knowledge in the Same Place: A Layer-wise Probing on BERToids’ Representations Mohsen Fayyaz author Ehsan Aghazadeh author Ali Modarressi author Hosein Mohebbi author Mohammad Taher Pilehvar author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication fayyaz-etal-2021-models 10.18653/v1/2021.blackboxnlp-1.29 https://aclanthology.org/2021.blackboxnlp-1.29/ 2021-11 375 388 Learning Mathematical Properties of Integers Maria Ryskina author Kevin Knight author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication ryskina-knight-2021-learning 10.18653/v1/2021.blackboxnlp-1.30 https://aclanthology.org/2021.blackboxnlp-1.30/ 2021-11 389 395 Probing Language Models for Understanding of Temporal Expressions Shivin Thukral author Kunal Kukreja author Christian Kavouras author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication thukral-etal-2021-probing 10.18653/v1/2021.blackboxnlp-1.31 https://aclanthology.org/2021.blackboxnlp-1.31/ 2021-11 396 406 How Familiar Does That Sound? Cross-Lingual Representational Similarity Analysis of Acoustic Word Embeddings Badr Abdullah author Iuliia Zaitova author Tania Avgustinova author Bernd Möbius author Dietrich Klakow author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication abdullah-etal-2021-familiar 10.18653/v1/2021.blackboxnlp-1.32 https://aclanthology.org/2021.blackboxnlp-1.32/ 2021-11 407 419 Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processing Sanchit Sinha author Hanjie Chen author Arshdeep Sekhon author Yangfeng Ji author Yanjun Qi author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication sinha-etal-2021-perturbing 10.18653/v1/2021.blackboxnlp-1.33 https://aclanthology.org/2021.blackboxnlp-1.33/ 2021-11 420 434 An Investigation of Language Model Interpretability via Sentence Editing Samuel Stevens author Yu Su author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication stevens-su-2021-investigation 10.18653/v1/2021.blackboxnlp-1.34 https://aclanthology.org/2021.blackboxnlp-1.34/ 2021-11 435 446 Interacting Knowledge Sources, Inspection and Analysis: Case-studies on Biomedical text processing Parsa Bagherzadeh author Sabine Bergler author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication bagherzadeh-bergler-2021-interacting 10.18653/v1/2021.blackboxnlp-1.35 https://aclanthology.org/2021.blackboxnlp-1.35/ 2021-11 447 456 Attacks against Ranking Algorithms with Text Embeddings: A Case Study on Recruitment Algorithms Anahita Samadi author Debapriya Banerjee author Shirin Nilizadeh author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication samadi-etal-2021-attacks 10.18653/v1/2021.blackboxnlp-1.36 https://aclanthology.org/2021.blackboxnlp-1.36/ 2021-11 457 467 Controlled tasks for model analysis: Retrieving discrete information from sequences Ionut-Teodor Sorodoc author Gemma Boleda author Marco Baroni author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication sorodoc-etal-2021-controlled 10.18653/v1/2021.blackboxnlp-1.37 https://aclanthology.org/2021.blackboxnlp-1.37/ 2021-11 468 478 The Acceptability Delta Criterion: Testing Knowledge of Language using the Gradience of Sentence Acceptability Héctor Javier Vázquez Martínez author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication vazquez-martinez-2021-acceptability 10.18653/v1/2021.blackboxnlp-1.38 https://aclanthology.org/2021.blackboxnlp-1.38/ 2021-11 479 495 How Does BERT Rerank Passages? An Attribution Analysis with Information Bottlenecks Zhiying Jiang author Raphael Tang author Ji Xin author Jimmy Lin author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication jiang-etal-2021-bert 10.18653/v1/2021.blackboxnlp-1.39 https://aclanthology.org/2021.blackboxnlp-1.39/ 2021-11 496 509 Do Language Models Know the Way to Rome? Bastien Liétard author Mostafa Abdou author Anders Søgaard author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication lietard-etal-2021-language 10.18653/v1/2021.blackboxnlp-1.40 https://aclanthology.org/2021.blackboxnlp-1.40/ 2021-11 510 517 Exploratory Model Analysis Using Data-Driven Neuron Representations Daisuke Oba author Naoki Yoshinaga author Masashi Toyoda author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication oba-etal-2021-exploratory 10.18653/v1/2021.blackboxnlp-1.41 https://aclanthology.org/2021.blackboxnlp-1.41/ 2021-11 518 528 Fine-Tuned Transformers Show Clusters of Similar Representations Across Layers Jason Phang author Haokun Liu author Samuel R Bowman author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication phang-etal-2021-fine 10.18653/v1/2021.blackboxnlp-1.42 https://aclanthology.org/2021.blackboxnlp-1.42/ 2021-11 529 538 BERT Has Uncommon Sense: Similarity Ranking for Word Sense BERTology Luke Gessler author Nathan Schneider author 2021-11 text Proceedings of the Fourth BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP Jasmijn Bastings editor Yonatan Belinkov editor Emmanuel Dupoux editor Mario Giulianelli editor Dieuwke Hupkes editor Yuval Pinter editor Hassan Sajjad editor Association for Computational Linguistics Punta Cana, Dominican Republic conference publication gessler-schneider-2021-bert 10.18653/v1/2021.blackboxnlp-1.43 https://aclanthology.org/2021.blackboxnlp-1.43/ 2021-11 539 547