Eric YAN

Eric YAN

National University of Singapore

Hi, I obtained my master degree and bachelor degree from School of Computing, National University of Singapore (NUS) and Hong Kong Baptist University (HKBU), respectively. I am currently pursuing my PhD degree at Hong Kong University of Science and Technology under the supervision of Prof.Xuming HU. I am also interning at AI Research, Squirrel AI now, supervised by Dr.Qingsong WEN.

Previously, I had extensive internship experience in both industry and academia, including NLP Team, ByteDance AI Lab (supervised by Mr.Yang WANG and Dr.Hang LI, director of ByteDance Research), LLM Group, Institute for Advanced Algorithms Research (co-supervised by Dr.Zhiyu LI, Dr.Feiyu XIONG, and Prof.Weinan E), Reefknot Investment (co-supervised by Mr.Marc DRAGON, managing director of Reefknot, and Prof.Wei Ngan CHIN, vice dean of SoC, NUS), and University of California, Berkeley (supervised by Dr.Qing ZHU, research scientist at Lawrence Berkeleey National Laboratory). I also conducted research at CityMind Lab, HKUST(GZ) (led by Prof.Yuxuan LIANG) before.

My research interests include natural language processing , multimodal representation learning , data mining applications including urban computing and recommendation systems . Look forward to any academic collaboration.

đź“– Education

  • Now, PhD, Hong Kong University of Science and Technology
  • 2021 - 2023, Master, National University of Singapore
  • 2017 - 2021, Undergraduate, Hong Kong Baptist University (President’s Honour Roll )

đź“ť Selected Publications

Note: * as Co-first Author; † as Corresponding Author

WWW 2024
sym

UrbanCLIP: Learning Text-enhanced Urban Region Profiling with Contrastive Language-Image Pretraining from the Web

Yibo Yan, Haomin Wen, Siru Zhong, Wei Chen, Haodong Chen, Qingsong Wen, Roger Zimmermann, Yuxuan Liang†

  • First-ever LLM-enhanced framework that integrates the knowledge of textual modality into urban imagery profiling.

The International World Wide Web Conference 2024, Singapore (WWW’24)

Oral Presentation

arXiv 2024
sym

ErrorRadar: Benchmarking Complex Mathematical Reasoning of Multimodal Large Language Models Via Error Detection

Yibo Yan, Shen Wang, Jiahao Huo, Hang Li, Boyan Li, Jiamin Su, Xiong Gao, Yi-Fan Zhang, Tianlong Xu, Zhendong Chu, Aoxiao Zhong, Kun Wang, Hui Xiong, Philip S. Yu, Xuming Hu†, Qingsong Wen†

  • First benchmark designed to assess MLLMs’ complex reasoning capabilities in multimodal error detection.

Under Review

CIKM 2024
sym

GeoReasoner: Reasoning On Geospatially Grounded Context For Natural Language Understanding

Yibo Yan, Joey Lee†

  • A pipeline integrating linguistic and geospatial information, showcasing the advantages of an LLM-assisted workflow over conventional methods in geo-reasoning tasks.

33rd ACM International Conference on Information and Knowledge Management, Idaho, USA (CIKM’24)

Best Short Paper Award

InfoFusion 2024
sym

Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook

Xingchen Zou*, Yibo Yan*, Xixuan Hao, Yuehong Hu, Haomin Wen, Erdong Liu, Junbo Zhang, Yong Li, Tianrui Li, Yu Zheng, Yuxuan Liang†

  • First comprehensive survey that systematically reviews studies on deep learning-based multimodal and multi-source data fusion models in urban computing.

Information Fusion Journal (IF=15)

EMNLP 2024
sym

MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model

Jiahao Huo, Yibo Yan, Boren Hu, Yutao Yue, Xuming Hu†

  • Investigation of the distribution of domain-specific neurons and the mechanism of how MLLMs process features from diverse domains.

Conference on Empirical Methods in Natural Language Processing 2024, USA (EMNLP’24)

arXiv 2024
sym

Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models

Kening Zheng*, Junkai Chen*, Yibo Yan, Xin Zou, Xuming Hu†

  • A comprehensive benchmark specifically targeting relation hallucinations, consisting of over 20k samples derived from real-world scenarios.

Under Review

arXiv 2024
sym

Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models

Xin Zou*, Yizhou Wang*, Yibo Yan, Sirui Huang, Kening Zheng, Junkai Chen, Chang Tang, Xuming Hu†

  • A novel hallucination mitigation paradigm that without the need for external knowledge retrieval or additional fine-tuning.

Under Review

ACM MM 2024
sym

UrbanCross: Enhancing Satellite Image-Text Retrieval with Cross-Domain Adaptation

Siru Zhong, Xixuan Hao, Yibo Yan, Ying Zhang, Yangqiu Song, Yuxuan Liang†

  • First cross-domain framework that integrates the power of LMM and SAM into satellite image-text retrieval.

32nd ACM Multimedia Conference, Melbourne, Australia (ACM MM’24)

arXiv 2024
sym

UrbanVLP: A Multi-Granularity Vision-Language Pre-Trained Model for Urban Indicator Prediction

Xixuan Hao, Wei Chen, Yibo Yan, Siru Zhong, Kun Wang, Qingsong Wen, Yuxuan Liang†

  • First urban region representation learning framework that explores multi-granularity cross-modal alignment.

Under Review

đź’» Work Experience

  • Jun 2024 - Now, AI Research, Squirrel AI, Remote.
    • Focus: Multimodal LLM for Education
    • Supervisors: Dr.Shen WANG (Principal Researcher of AI Research) and Dr.Qingsong WEN (Head of AI Research & Chief Scientist)
    • Achievement: Ongoing Project
  • Feb 2024 - May 2024, LLM Group, Institute for Advanced Algorithms Research, Shanghai, Remote.
    • Focus: LLM Hallucination Mitigation
    • Supervisors: Dr.Zhiyu LI (Principal Researcher of LLM Team) and Dr.Feiyu XIONG (Director of LLM Group)
    • Achievement: Mitigated the entity-level hallucination in real-life news corpora from Xinhua News Agency
  • Nov 2022 - Jul 2023, AI Lab, ByteDance, Singapore.
    • Focus: NLP (esp. user intent recognition and conversation modelling) and Recommendation Systems
    • Supervisors: Mr.Yang WANG (Leader of Conversation Team) and Dr.Hang LI (Director of ByteDance Research, Fellow of ACM/ACL/IEEE)
    • Achievement: Successfully designed and deployed multiple models in real-life applications such as Tiktok Intelligence Customer Service and Douyin E-commerce Platform
  • May 2022 - Sep 2022, Reefknot Investment, Singapore.
    • A joint venture between Temasek and Kuehne+Nagel
    • Focus: Graph Analytics, NLP (esp. entity resolution), Federated Learning
    • Supervisors: Mr.Marc DRAGON (Managing Director of Reefknot) and Prof.Wei Ngan CHIN (Associate Professor and Vice Dean of SoC, NUS)
    • Achievement: Comprehensive tech analysis for target deep-tech start-ups
  • Jul 2020 - Sep 2020, UC Berkeley, Remote.
    • Focus: Casual Modelling for Earth Science
    • Supervisor: Dr.Qing ZHU (Research Scientist at Institute for Data Science)
    • Achievement: Developed a transfer entropy-based climate diagnostic tool for Pearl River Delta

🎖 Honors and Awards

  • 2023, Silver Medal , OTTO - Multi-Objective Recommender System, Kaggle Competition.
  • 2023, Silver Medal , Stable Diffusion - Image to Prompts, Kaggle Competition.
  • 2021, Best Undergraduate Thesis (Remote Sensing Track)
  • 2019-2020, 2020-2021, First-class Academic Award , HKBU
  • 2017-2018, 2018-2019, Second-class Academic Award , HKBU