News from the AI & ML world
DeeperML
@learn.aisingapore.org
//
MIT researchers have uncovered a critical flaw in vision-language models (VLMs) that could have serious consequences in high-stakes environments like medical diagnosis. The study, published May 14, 2025, reveals that these AI models, widely used to analyze medical images, struggle with negation words such as "no" and "not." This deficiency causes them to misinterpret queries, leading to potentially catastrophic errors when retrieving images based on the absence of certain objects. An example provided highlights the case of a radiologist using a VLM to find reports of patients with tissue swelling but without an enlarged heart, the model incorrectly identifying reports with both conditions, leading to an inaccurate diagnosis.
Researchers tested the ability of vision-language models to identify negation in image captions and found the models often performed as well as a random guess. To address this issue, the MIT team created a dataset of images with corresponding captions that include negation words describing missing objects. Retraining a vision-language model with this dataset resulted in improved performance when retrieving images that do not contain specific objects, and also boosted accuracy on multiple choice question answering with negated captions.
Kumail Alhamoud, the lead author of the study, emphasized the significant impact of negation words and the potential for catastrophic consequences if these models are used blindly. While the researchers were able to improve model performance through retraining, they caution that more work is needed to address the root causes of this problem. They hope their findings will alert potential users to this previously unnoticed shortcoming, especially in settings where these models are used to determine patient treatments or identify product defects. Marzyeh Ghassemi, the senior author, warned against using large vision/language models without intensive evaluation if something as fundamental as negation is broken.
References :
Researchers tested the ability of vision-language models to identify negation in image captions and found the models often performed as well as a random guess. To address this issue, the MIT team created a dataset of images with corresponding captions that include negation words describing missing objects. Retraining a vision-language model with this dataset resulted in improved performance when retrieving images that do not contain specific objects, and also boosted accuracy on multiple choice question answering with negated captions.
Kumail Alhamoud, the lead author of the study, emphasized the significant impact of negation words and the potential for catastrophic consequences if these models are used blindly. While the researchers were able to improve model performance through retraining, they caution that more work is needed to address the root causes of this problem. They hope their findings will alert potential users to this previously unnoticed shortcoming, especially in settings where these models are used to determine patient treatments or identify product defects. Marzyeh Ghassemi, the senior author, warned against using large vision/language models without intensive evaluation if something as fundamental as negation is broken.
Share:
References :
- learn.aisingapore.org: Study shows vision-language models can’t handle queries with negation words | MIT News
- www.sciencedaily.com: Study shows vision-language models can't handle queries with negation words