Scribd
Upload
9 views

Information Age

The document introduces AdaVision, an interactive process for testing vision models. It retrieves images from large datasets based on natural language queries to help users identify coherent failure modes. The user labels examples to refine the search for high-error examples. AdaVision then suggests new queries to explore. User studies found major bugs in models with failure rates 2-3x higher than automatic methods. Finetuning on examples found fixes bugs without degrading original performance or improving out-of-distribution accuracy.

Uploaded by

arlynguerrero846
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
9 views

Information Age

The document introduces AdaVision, an interactive process for testing vision models. It retrieves images from large datasets based on natural language queries to help users identify coherent failure modes. The user labels examples to refine the search for high-error examples. AdaVision then suggests new queries to explore. User studies found major bugs in models with failure rates 2-3x higher than automatic methods. Finetuning on examples found fixes bugs without degrading original performance or improving out-of-distribution accuracy.

Uploaded by

arlynguerrero846
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 1

Adaptive Testing of Computer Vision

Models
Irena Gao, Gabriel Ilharco, Scott Lundberg, Marco Tulio Ribeiro; Proceedings of the IEEE/CVF International Conference
on Computer Vision (ICCV), 2023, pp. 4003-4014

Vision models often fail systematically on groups of data that share


common semantic characteristics (e.g., rare objects or unusual scenes), but
identifying these failure modes is a challenge. We introduce AdaVision, an
interactive process for testing vision models which helps users identify and
fix coherent failure modes. Given a natural language description of a
coherent group, AdaVision retrieves relevant images from LAION-5B with
CLIP. The user then labels a small amount of data for model correctness,
which is used in successive retrieval rounds to hill-climb towards high-error
regions, refining the group definition. Once a group is saturated, AdaVision
uses GPT-3 to suggest new group descriptions for the user to explore. We
demonstrate the usefulness and generality of AdaVision in user studies,
where users find major bugs in state-of-the-art classification, object
detection, and image captioning models. These user-discovered groups
have failure rates 2-3x higher than those surfaced by automatic error
clustering methods. Finally, finetuning on examples found with AdaVision
fixes the discovered bugs when evaluated on unseen examples, without
degrading in-distribution accuracy, and while also improving performance
on out-of-distribution datasets.
" Adaptive testing of computer vision models" is significant and relevant to our life in
several ways. It provides us with a better understanding of how computer vision models are
tested and how to identify and fix their failure modes.The article introduces us to a new and
innovative testing methodology. The article demonstrates the importance of testing computer
vision models on a variety of data. This is important because computer vision models can fail
systematically on groups of data that share common semantic characteristics.

The article provides a comprehensive overview of computer vision testing strategies and
highlights the importance of testing computer vision models on a variety of data distributions. We
believe that this information is valuable for students and researchers who are interested in
developing and deploying computer vision models. Adaptive testing of computer vision models
to be well-written and informative. We would recommend it to anyone who is interested in
learning more about computer vision testing and the development of more reliable and
generalizable computer vision systems.

References:https://openaccess.thecvf.com/content/ICCV2023/html/
Gao_Adaptive_Testing_of_Computer_Vision_Models_ICCV_2023_paper.html

You might also like