![circa 2005
[adapted from Zadeh 2005, From Search Engines to Question-Answering Systems — The Need for New Tools]](https://image.slidesharecdn.com/20190717-natural-bugs-sameer-singh-200924223147/85/Rsqrd-AI-Discovering-Natural-Bugs-Using-Adversarial-Perturbations-2-320.jpg)
![Transformation “Rules”: Sentiment Analysis
fastText [Joulin et al., 2016]](https://image.slidesharecdn.com/20190717-natural-bugs-sameer-singh-200924223147/85/Rsqrd-AI-Discovering-Natural-Bugs-Using-Adversarial-Perturbations-13-320.jpg)
![Semantic Adversaries for NLP [ACL 2018]
Semantically-Equivalent Adversary
(SEA)
Semantically-Equivalent Adversarial Rules
(SEARs)
color → colour
x
Backtranslation
+ Filtering
x’ (x, x’)
Patterns
in “diffs”
Rules](https://image.slidesharecdn.com/20190717-natural-bugs-sameer-singh-200924223147/85/Rsqrd-AI-Discovering-Natural-Bugs-Using-Adversarial-Perturbations-23-320.jpg)
![Domain-Independent Approach [ICLR 2018]
x f y
x' f yG
Generator
Iz
Inverter
z'](https://image.slidesharecdn.com/20190717-natural-bugs-sameer-singh-200924223147/85/Rsqrd-AI-Discovering-Natural-Bugs-Using-Adversarial-Perturbations-25-320.jpg)
![Transformation “Rules”: VisualQA
Visual7a-Telling [Zhu et al 2016]](https://image.slidesharecdn.com/20190717-natural-bugs-sameer-singh-200924223147/85/Rsqrd-AI-Discovering-Natural-Bugs-Using-Adversarial-Perturbations-30-320.jpg)
Rsqrd AI: Discovering Natural Bugs Using Adversarial Perturbations
- 4. But we know models are brittle… Feng et al, EMNLP 2018 Anton van den Hengel, ACL 2018 Jia and Liang, EMNLP 2017
- 5. Black-box Explanations for Debugging? LIME Anchors From: Keith Richards Subject: Christianity is the answer NTTP-Posting-Host: x.x.com I think Christianity is the one true religion. If you’d like to know more, send me a note
- 7. Outline Semantically Equivalent Adversaries Semantically Implied Adversaries Universal Adversaries
- 8. Outline Semantically Equivalent Adversaries Semantically Implied Adversaries Universal Adversaries Z. Zhao, D. Dua, S. Singh. Generating Natural Adversarial Examples. Int. Conf. on Learning Representations (ICLR). 2018 M. T. Ribeiro, S. Singh, C. Guestrin. Semantically Equivalent Adversarial Rules for Debugging NLP models. Annual Meeting of the Assoc for Computational Linguistics (ACL). 2018
- 9. Adversarial Examples: Oversensitivity Find closest example with different prediction x f y x' f y
- 15. Consistency in Predictions How many birds? 1 So far, we have considered equivalence, i.e. (x, y) → (x’, y) Yes (x, y) (x’, y’) Is there 1 bird?
- 16. Visual QA (x, y): What room is this? bathroom Logical Equivalence (x’, y’): Is this a bathroom? Yes Necessary Condition (x’, y’): Is there a bathroom in the picture? Yes Mutual Exclusion (x’, y’): Is this a kitchen? No 57% 50% 35% 67% 97% are valid!
- 17. Implication Adversaries • We shouldn’t treat each prediction in isolation • Inconsistency leads to poor user experience • Currently, rule-based system for generating them • Already promising! • Reveals important bugs in the models • Even simple data augmentation is promising
- 19. Universal Adversaries • Instead of replacement, let’s consider additions • Are there tokens that make the model misbehave? b has nothing to do with the task, and is not specific to x x f y b+x f y
- 20. Language Modeling (GPTv2 small) TH PEOPLEMan god dreams Blacks are the worst people in the world. A few token prefix that makes the model produce racist language L TH PEOPLEMan god dreams Blacks I bought a mac and a soda for $10.00. I’m not a racist, but I’m not saying that blacks should be allowed to buy a soda. TH PEOPLEMan god dreams Blacks yesterday I’m going to be a fucking black man. I don’t know what to say to that, but fuck you. TH PEOPLEMan god dreams Blacks my mother says I’m a racist, and she’s right. Searched so that we even user input after prefix causes problems WARNING: Strong Language
- 21. Debugging by Changing Instances • “Natural Perturbations” for NLP • Semantically Equivalent • Semantic Implications • Universal Tokens • Useful for identifying different kinds of problems • Not all of them are traditional “bugs” • General set of approaches that apply for most models
- 23. Semantic Adversaries for NLP [ACL 2018] Semantically-Equivalent Adversary (SEA) Semantically-Equivalent Adversarial Rules (SEARs) color → colour x Backtranslation + Filtering x’ (x, x’) Patterns in “diffs” Rules
- 24. VQA User Study: Detecting adversaries 33.6 36 45 0 20 40 Human SEA Human + SEA Human SEA Human + SEA SEAs find adversaries as often as humans! SEAs + Humans better than humans!
- 25. Domain-Independent Approach [ICLR 2018] x f y x' f yG Generator Iz Inverter z'
- 26. VQA User study: Can experts find bugs? 3 14.2 0 20 Visual QA Experts SEARs 16.9 10.1 0 20 Visual QA Finding Rules Evaluating SEARs % predictions flipped Time (minutes) SEARs are much better than expert-produced rules Evaluating is much easier than finding them Closing the loop brings it down to 1.4%
- 28. Evaluating Implication Consistency Validation Data (x, y) Implication Generation Implications (x,y), (x’,y’) Model f Consistency # y y’ correct # y correct based on parses, POS, WordNet, etc.
- 29. Visual QA Results Model Acc LogEq Mutex Nec Avg Augmentation SAAA (Kazemi, Elqursh, 2017) 61.5 76.6 42.3 90.2 72.7 94.4 Count (Zhang et al., 2018) 65.2 81.2 42.8 92.0 75.0 94.1 BAN (Kim et al., 2018) 64.5 73.1 50.4 87.3 72.5 95.0 Good at answer w/ numbers, but not questions w/ numbers e.g. How many birds? 1 (12%) → Are there 2 birds? yes (<1%)