Yuwei Niu

About Me

I am a third-year undergraduate student in the Hongshen Stand-Out Class of Computer Science at Chongqing University. I am incredibly fortunate to have closely collaborated with Prof. Lei Feng during my undergraduate studies and will be joining Peking University, advised by Prof. Li Yuan.

My core research focuses on exploring how to effectively align and fuse "real-world" visual representations with "human-world" language priors. In my view, language itself is a high-dimensional compression and abstract extraction of visual information. It encodes massive amounts of information into more efficient and meaningful representations through learning and refinement. My research direction is precisely to find an effective path for this compression and alignment between the language and visual modalities.

Language, as the carrier of human knowledge and experience, forms a microcosm of the "human world"; while vision directly captures and reflects the true appearance of the "real world." I firmly believe that bridging the gap between the language modality, which carries "human world" prior knowledge, and the visual modality, which directly reflects the "real world," is a crucial step towards achieving general artificial intelligence. This not only involves finding effective mapping paths between modalities but also concerns how to leverage their respective strengths to collaboratively build a unified cognitive framework.

Currently, I am dedicated to building unified multimodal models and exploring why such models are essential.

Education

• [2022-2026] B.Eng. in Computer Science, Hongshen Stand-Out Class, Chongqing University (CQU).

Selected Publications & Manuscripts

* Equal Contribution    # Project Lead

• 🔥 WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation
  Yuwei Niu*, Munan Ning*, Mengren Zheng, Weiyang Jin, Bin Lin, Peng Jin, Jiaqi Liao, Chaoran Feng, Kunpeng Ning, Bin Zhu, Li Yuan†.
  Probing Language Priors in Multimodal Unify Models. First to explore how world knowledge can benefit the understanding and generation of unified models. Widely adopted by many prominent researchers.
  Under Review   [Paper] [Code 🌟100+]

• 🔥 LangBridge: Interpreting Image as a Combination of Language Embeddings
  Jiaqi Liao*, Yuwei Niu*, Fanqing Meng*, Hao Li, Changyao Tian, Yinuo Du, Yuwen Xiong, Dianqi Li, Xizhou Zhu, Li Yuan, Jifeng Dai†, Yu Cheng†.
  Language Priors as Visual Representations. Introducing a novel multimodal alignment paradigm that competes with LLaVA.
  Under Review   [Paper] [Code]

• 🔥 UniWorld: High-Resolution Semantic Encoders for Unified Visual Understanding and Generation
  Bin Lin, Zongjian Li, Xinhua Cheng, Yuwei Niu, Yang Ye, Xianyi He, Shenghai Yuan, Wangbo Yu, Shaodong Wang, Yunyang Ge, Yatian Pang, Li Yuan†.
  A comprehensive and powerful unified model!
  Tech Report   [Paper] [Code 🌟500+]

Other Works

• Test-Time Multimodal Backdoor Detection by Contrastive Prompting
  Yuwei Niu*, Shuo He*, Qi Wei, Zongyu Wu, Feng Liu, Lei Feng†.
  ICML 2025   [Paper]

• ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Models
  Junzhe Chen, Tianshu Zhang, Shiyu Huang, Yuwei Niu, Linfeng Zhang, Lijie Wen, Xuming Hu†.
  CVPR 2025   [Paper]

• Tuning Vision-Language Models with Candidate Labels by Prompt Alignment
  Zhifang Zhang, Yuwei Niu, Xin Liu, Beibei Li†.
  DASFAA 2025   [Paper]

• LanP: Rethinking the Impact of Language Priors in Large Vision-Language Models
  Zongyu Wu*, Yuwei Niu*, Hongcheng Gao, Minhua Lin, Zhiwei Zhang, Zhifang Zhang, Qi Shi, Yilong Wang, Sike Fu, Junjie Xu, Junjie Ao, Enyan Dai, Lei Feng, Xiang Zhang, Suhang Wang†.
  Probing Language Priors in Multimodal Models.
  [Paper]

• OMNIDPO: A Preference Optimization Framework to Address Omni-Modal Hallucination
  Junzhe Chen, Tianshu Zhang, Shiyu Huang, Yuwei Niu, Rongzhou Zhang, Guanyu Zhou, Lijie Wen, Xuming Hu†.
  Under Review  

Community Contribution

• Founder of Awesome-Unified-Multimodal
  Created the most comprehensive repository for organizing papers, codes, and resources on unified multimodal models. (GitHub 🌟220+)

Personality

Beyond science, I am deeply passionate about literature, poetry, anime, films, music, and all forms of art that embody human creativity. I firmly believe these are the reasons for our existence. Currently, I am captivated by Dostoevsky's literature and R&B music.

Contact Me

I truly believe that great ideas and improvements come from open discussions and debates in academia. If you have any thoughts, disagreements with my work, or fresh ideas you’d like to share, I’d be really grateful to hear from you. I am incredibly fortunate to have met many friends who have helped me along the way, and I am always willing to chat and offer any assistance I can.

If you’ve got any questions about my research or if you’ve tried reaching out through GitHub issues and haven’t heard back, please don’t hesitate to drop me an email.

Please note that I am only interested in discussing intriguing problems and insights, not metrics. If you are inclined to discuss publication or citation numbers, rely on numerical indicators to quantify individuals, or compare me to others, please refrain from contacting me.

My preferred email: niuyuwei04@gmail.com