MathAgent: Leveraging a Mixture-of-Math-Agent Framework for Real-World Multimodal Mathematical Error Detection

Yibo Yan, Shen Wang, Jiahao Huo, Philip S. Yu, Xuming Hu†, Qingsong Wen†

• A novel Mixture-of-Math-Agent framework designed specifically to address mutlimodal error detection.

63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria (ACL’25 Industry)

Oral Presentation

A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges

Yibo Yan, Jiamin Su, Jianxiang He, Fangteng Fu, Xu Zheng, Yuanhuiyi Lyu, Kun Wang, Shen Wang, Qingsong Wen, Xuming Hu†

• First comprehensive analysis of mutlimodal mathematical reasoning in the era of MLLMs.

63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria (ACL’25 Findings)

UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web

Yibo Yan, Haomin Wen, Siru Zhong, Wei Chen, Haodong Chen, Qingsong Wen, Roger Zimmermann, Yuxuan Liang†

• First-ever LLM-enhanced framework that integrates the knowledge of textual modality into urban imagery profiling.

The International World Wide Web Conference 2024, Singapore (WWW’24)

Oral Presentation

GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understanding

Yibo Yan, Joey Lee†

• A pipeline integrating linguistic and geospatial information, showcasing the advantages of an LLM-assisted workflow over conventional methods in geo-reasoning tasks.

33rd ACM International Conference on Information and Knowledge Management, Idaho, USA (CIKM’24)

Best Short Paper Award

ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection

Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, Yi-Fan Zhang, Tianlong Xu, Zhendong Chu, Aoxiao Zhong, Kun Wang, Hui Xiong, Philip S. Yu, Xuming Hu†, Qingsong Wen†

• First benchmark designed to assess MLLMs’ complex reasoning capabilities in multimodal error detection.

Workshop on Reasoning and Planning for LLMs, 13th International Conference on Learning Representations, Singapore (ICLR Workshop’25)

Position: Multimodal Large Language Models Can Significantly Advance Scientific Reasoning

Yibo Yan, Shen Wang, Jiahao Huo, Jingheng Ye, Zhendong Chu, Xuming Hu†, Philip S. Yu, Carla Gomes, Bart Selman, Qingsong Wen†

• This position paper argues that MLLMs can significantly advance scientific reasoning across disciplines such as mathematics, physics, chemistry, and biology.

Under Review

Pierce the Mists, Greet the Sky: Decipher Knowledge Overshadowing via Knowledge Circuit Analysis

Haoming Huang*, Yibo Yan*, Jiahao Huo, Xin Zou, Xinfeng Li, Kun Wang, Xuming Hu†

• A novel framework designed to comprehensively analyze and detect knowledge overshadowing, a particularly challenging variant of LLM hallucinations.

The 2025 Conference on Empirical Methods in Natural Language Processing, Soochow, China (EMNLP 2025)

MMUnlearner: Reformulating Multimodal Machine Unlearning in the Era of Multimodal Large Language Models

Jiahao Huo*, Yibo Yan*, Xu Zheng, Yuanhuiyi Lyu, Xin Zou, Zhihua Wei, Xuming Hu†

• We propose to reformulate the task of multimodal MU in the era of MLLMs, and develop a novel geometry-constrained gradient descent method MMUnlearner.

63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria (ACL’25 Findings)

EssayJudge: A Multi-Granular Benchmark for Assessing Automated Essay Scoring Capabilities of Multimodal Large Language Models

Jiamin Su*, Yibo Yan*, Fangteng Fu, Han Zhang, Jingheng Ye, Xiang Liu, Jiahao Huo, Huiyu Zhou, Xuming Hu†

• First multimodal benchmark to evaluate Automated Essay Scoring (AES) capabilities across lexical-, sentence-, and discourse-level traits.

63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria (ACL’25 Findings)

PhysicsArena: The First Multimodal Physics Reasoning Benchmark Exploring Variable, Process, and Solution Dimensions

Song Dai*, Yibo Yan*, Jiamin Su, Dongfang Zihao, Yubo Gao, Yonghua Hei, Jungang Li, Junyan Zhang, Sicheng Tao, Zhuoran Gao, Xuming Hu†

• First multimodal physics reasoning benchmark designed to holistically evaluate MLLMs across three critical dimensions: variable identification, physical process formulation, and solution derivation.

The 2025 Conference on Empirical Methods in Natural Language Processing, Soochow, China (EMNLP 2025)

Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook

Xingchen Zou*, Yibo Yan*, Xixuan Hao, Yuehong Hu, Haomin Wen, Erdong Liu, Junbo Zhang, Yong Li, Tianrui Li, Yu Zheng, Yuxuan Liang†

• First comprehensive survey that systematically reviews studies on deep learning-based multimodal and multi-source data fusion models in urban computing.

Information Fusion Journal (IF=15)

Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality

Guanyu Zhou, Yibo Yan, Xin Zou, Kun Wang, Aiwei Liu, Xuming Hu†

• A causal inference framework that applies structural causal modeling to MLLMs, treating modality priors as a confounder between attention mechanisms and output.

13th International Conference on Learning Representations, Singapore (ICLR’25)

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model

Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, Xuming Hu†

• Investigation of the distribution of domain-specific neurons and the mechanism of how MLLMs process features from diverse domains.

Conference on Empirical Methods in Natural Language Processing 2024, Florida, USA (EMNLP’24)

CAFES: A Collaborative Multi-Agent Framework for Multi-Granular Multimodal Essay Scoring

Jiamin Su, Yibo Yan, Zhuoran Gao, Han Zhang, Xiang Liu, Xuming Hu†

• First collaborative multi-agent framework specifically designed for Automated Essay Scoring.

Under Review

Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models

Xin Zou*, Yizhou Wang*, Yibo Yan, Sirui Huang, Kening Zheng, Junkai Chen, Chang Tang, Xuming Hu†

• A novel hallucination mitigation paradigm that without the need for external knowledge retrieval or additional fine-tuning.

42nd International Conference on Machine Learning, Vancouver, Canada (ICML’25)

Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models

Kening Zheng*, Junkai Chen*, Yibo Yan, Xin Zou, Xuming Hu†

• A comprehensive benchmark specifically targeting relation hallucinations, consisting of over 20k samples derived from real-world scenarios.

63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria (ACL’25 Findings)

Unveiling Instruction-Specific Neurons & Experts: An Analytical Framework for LLM’s Instruction-Following Capabilities

Junyan Zhang*, Yubo Gao*, Yibo Yan, Jungang Li, Zhaorui Hou, Sicheng Tao, Shuliang Liu, Song Dai, Yonghua Hei, Junzhuo Li, Xuming Hu†

• A novel analytical framework for identification of instruction-specific sparse components of LLMs, and evaluation of their functional generality and uniqueness.

Under Review