Qing Li

I am a research scientist and team lead at Beijing Institute for General Artificial Intelligence (BIGAI), China. I received my Ph.D. in 2022 from Department of Statistics at University of California, Los Angeles (UCLA), advised by Professor Song-Chun Zhu. During my Ph.D., I have interned at Google Research, Microsoft Azure AI and Amazon Alexa. Before UCLA, I obtained my degrees of Bachelor in 2015 and Master in 2018 from University of Science and Technology of China (USTC).

My research interests lie in the intersection of machine learning, computer vision, cognition, and robotics. My current research themes include:

Multimodal Understanding: vision & language understanding, visual reasoning, 3D scene understanding, video understanding
General Machine Learning: neural-symbolic reasoning and learning, structure learning, representation learning, generative modeling, few-shot learning
Embodied Agents: language-grounded task planning, reinforcement learning, robotics

Our team is actively recruiting full-time research scientists, engineers, and self-motivated interns. We are also seeking prospective PhD students and long-term collaborators for TongProgam (通计划). Feel free to contact me if you are interested!

News

2024-07	🔥🔥🔥 Three papers are accepted by ECCV 2024! Check out these awesome works: PQ3D, a unfied model for 3D vision-language understanding; SceneVerse, the first million-scale 3D vision-language dataset; VideoAgent, a LLM agent that understands videos by using a structured memory and 4 tools.
2024-06	Call for papers to IJCLR 2024, which will happen on 20 - 22 September 2024 in Nanjing! I will serve as an area chair on neuro-symbolic learning and reasoning. If you are willing to be a PC member, please contact me!
2024-06	INSIGHT is selected as ICML 2024 Spotlight (top 3.5%)!
2024-05	Check out our new work PQ3D, the first unfied model capable of handling a wide range of 3D-VL tasks! The code and models will be released soon. Stay tuned!
2024-05	Two papers are accepted by ICML 2024. They are about 3D embodied generalist agent (LEO) and end-to-end neural-symbolic RL for explainable decision-making (INSIGHT). Congrats Jiangyong Huang et al. and Lirui Luo et al.!

Selected Publications

* Equal contribution, ✉ Corresponding author

End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanations Spotlight (top 3.5%)

Lirui Luo , Guoxi Zhang , Hongming Xu , Yaodong Yang , Cong Fang^✉ , and Qing Li^✉

International Conference on Machine Learning (ICML), 2024

arXiv Bib Website

Spotlight (top 3.5%)

@article{luo2024insight,
  title = {End-to-End Neuro-Symbolic Reinforcement Learning with Textual Explanations},
  author = {Luo, Lirui and Zhang, Guoxi and Xu, Hongming and Yang, Yaodong and Fang, Cong and Li, Qing},
  journal = {International Conference on Machine Learning (ICML)},
  year = {2024},
}

Unifying 3D Vision-Language Understanding Via Promptable Queries

Ziyu Zhu , Zhuofan Zhang , Xiaojian Ma , Xuesong Niu , Yixin Chen , Baoxiong Jia , Zhidong Deng^✉ , Siyuan Huang^✉ , and Qing Li^✉

European Conference on Computer Vision (ECCV), 2024

arXiv Bib Website

@article{zhu2024unifying,
  title = {Unifying 3D Vision-Language Understanding Via Promptable Queries},
  author = {Zhu, Ziyu and Zhang, Zhuofan and Ma, Xiaojian and Niu, Xuesong and Chen, Yixin and Jia, Baoxiong and Deng, Zhidong and Huang, Siyuan and Li, Qing},
  journal = {European Conference on Computer Vision (ECCV)},
  year = {2024}
}

VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding

Yue Fan* , Xiaojian Ma* , Rujie Wu , Yuntao Du , Jiaqi Li , Zhi Gao , and Qing Li^✉

European Conference on Computer Vision (ECCV), 2024

arXiv Bib Website

@article{fan2024videoagent,
  title = {VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding},
  author = {Fan, Yue and Ma, Xiaojian and Wu, Rujie and Du, Yuntao and Li, Jiaqi and Gao, Zhi and Li, Qing},
  journal = {European Conference on Computer Vision (ECCV)},
  year = {2024}
}

Unifying 3D Vision-Language Understanding Via Promptable Queries

Ziyu Zhu , Zhuofan Zhang , Xiaojian Ma , Xuesong Niu , Yixin Chen , Baoxiong Jia , Zhidong Deng^✉ , Siyuan Huang^✉ , and Qing Li^✉

European Conference on Computer Vision (ECCV), 2024

arXiv Bib Website

@article{zhu2024unifyinh,
  title = {Unifying 3D Vision-Language Understanding Via Promptable Queries},
  author = {Zhu, Ziyu and Zhang, Zhuofan and Ma, Xiaojian and Niu, Xuesong and Chen, Yixin and Jia, Baoxiong and Deng, Zhidong and Huang, Siyuan and Li, Qing},
  journal = {European Conference on Computer Vision (ECCV)},
  year = {2024}
}

SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding

Baoxiong Jia* , Yixin Chen* , Huangyue Yu , Yan Wang , Xuesong Niu , Tengyu Liu , Qing Li , and Siyuan Huang

European Conference on Computer Vision (ECCV), 2024

arXiv Bib Website

@article{jia2024sceneverse,
  title = {SceneVerse: Scaling 3D Vision-Language Learning for Grounded Scene Understanding},
  author = {Jia, Baoxiong and Chen, Yixin and Yu, Huangyue and Wang, Yan and Niu, Xuesong and Liu, Tengyu and Li, Qing and Huang, Siyuan},
  journal = {European Conference on Computer Vision (ECCV)},
  year = {2024}
}

CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update

Zhi Gao , Yuntao Du , Xintong Zhang , Xiaojian Ma , Wenjuan Han , Song-Chun Zhu , and Qing Li^✉

The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024

arXiv Bib Website

@article{gao2024clova,
  title = {CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update},
  author = {Gao, Zhi and Du, Yuntao and Zhang, Xintong and Ma, Xiaojian and Han, Wenjuan and Zhu, Song-Chun and Li, Qing},
  journal = {The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year = {2024}
}

An Embodied Generalist Agent in 3D World

Jiangyong Huang* , Silong Yong* , Xiaojian Ma* , Xiongkun Linghu* , Puhao Li , Yan Wang , Qing Li , Song-Chun Zhu , Baoxiong Jia , and Siyuan Huang

International Conference on Machine Learning (ICML), 2024

arXiv Bib Website

@article{huang2024embodied,
  title = {An Embodied Generalist Agent in 3D World},
  author = {Huang, Jiangyong and Yong, Silong and Ma, Xiaojian and Linghu, Xiongkun and Li, Puhao and Wang, Yan and Li, Qing and Zhu, Song-Chun and Jia, Baoxiong and Huang, Siyuan},
  journal = {International Conference on Machine Learning (ICML)},
  year = {2024}
}

Neural-Symbolic Recursive Machine for Systematic Generalization

Qing Li , Yixin Zhu , Yitao Liang , Ying Nian Wu , Song-Chun Zhu , and Siyuan Huang

International Conference on Learning Representations (ICLR), 2024

arXiv Bib Website

@article{li2024nsr,
  title = {Neural-Symbolic Recursive Machine for Systematic Generalization},
  author = {Li, Qing and Zhu, Yixin and Liang, Yitao and Wu, Ying Nian and Zhu, Song-Chun and Huang, Siyuan},
  journal = {International Conference on Learning Representations (ICLR)},
  year = {2024}
}

Bongard-OpenWorld: Few-Shot Reasoning for Free-Form Visual Concepts in the Real World

Rujie Wu* , Xiaojian Ma* , Zhenliang Zhang , Wei Wang^✉ , Qing Li^✉ , Song-Chun Zhu , and Yizhou Wang

International Conference on Learning Representations (ICLR), 2024

arXiv Bib Website

@article{wu2024bongard,
  title = {Bongard-OpenWorld: Few-Shot Reasoning for Free-Form Visual Concepts in the Real World},
  author = {Wu, Rujie and Ma, Xiaojian and Zhang, Zhenliang and Wang, Wei and Li, Qing and Zhu, Song-Chun and Wang, Yizhou},
  journal = {International Conference on Learning Representations (ICLR)},
  year = {2024}
}

Learning Non-Markovian Decision-Making from State-Only Sequences

Aoyang Qin , Feng Gao , Qing Li , Song-Chun Zhu , and Sirui Xie

Advances in Neural Information Processing Systems (NeurIPS), 2023

arXiv Bib Code

@article{qin2023learning,
  title = {Learning Non-Markovian Decision-Making from State-Only Sequences},
  author = {Qin, Aoyang and Gao, Feng and Li, Qing and Zhu, Song-Chun and Xie, Sirui},
  journal = {Advances in Neural Information Processing Systems (NeurIPS)},
  year = {2023}
}

A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics Notable-top-25%

Qing Li , Siyuan Huang , Yining Hong , Yixin Zhu , Ying Nian Wu , and Song-Chun Zhu

International Conference on Learning Representations (ICLR), 2023

arXiv Bib Website

Notable-top-25%

@article{li2023hint,
  title = {A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics},
  author = {Li, Qing and Huang, Siyuan and Hong, Yining and Zhu, Yixin and Wu, Ying Nian and Zhu, Song-Chun},
  journal = {International Conference on Learning Representations (ICLR)},
  year = {2023}
}

3D-VisTA: Pre-Trained Transformer for 3D Vision and Text Alignment

Ziyu Zhu , Xiaojian Ma , Yixin Chen , Zhidong Deng^✉ , Siyuan Huang^✉ , and Qing Li^✉

International Conference on Computer Vision (ICCV), 2023

arXiv Bib Website

@article{zhu2023vista,
  title = {3D-VisTA: Pre-Trained Transformer for 3D Vision and Text Alignment},
  author = {Zhu, Ziyu and Ma, Xiaojian and Chen, Yixin and Deng, Zhidong and Huang, Siyuan and Li, Qing},
  journal = {International Conference on Computer Vision (ICCV)},
  year = {2023}
}

SQA3D: Situated Question Answering in 3D Scenes

Xiaojian Ma* , Silong Yong* , Zilong Zheng , Qing Li , Yitao Liang , Song-Chun Zhu , and Siyuan Huang

International Conference on Learning Representations (ICLR), 2023

arXiv Bib Website

@article{ma2023sqa3d,
  title = {SQA3D: Situated Question Answering in 3D Scenes},
  author = {Ma, Xiaojian and Yong, Silong and Zheng, Zilong and Li, Qing and Liang, Yitao and Zhu, Song-Chun and Huang, Siyuan},
  journal = {International Conference on Learning Representations (ICLR)},
  year = {2023}
}

SMART: A Situation Model for Algebra Story Problems via Attributed Grammar

Yining Hong , Qing Li , Ran Gong , Daniel Ciao , Siyuan Huang , and Song-Chun Zhu

AAAI Conference on Artificial Intelligence (AAAI), 2021

arXiv Bib

@article{hong2021smart,
  title = {SMART: A Situation Model for Algebra Story Problems via Attributed Grammar},
  author = {Hong, Yining and Li, Qing and Gong, Ran and Ciao, Daniel and Huang, Siyuan and Zhu, Song-Chun},
  journal = {AAAI Conference on Artificial Intelligence (AAAI)},
  year = {2021}
}

Learning by Fixing: Solving Math Word Problems with Weak Supervision

Yining Hong , Qing Li , Daniel Ciao , Siyuan Huang , and Song-Chun Zhu

AAAI Conference on Artificial Intelligence (AAAI), 2021

arXiv Bib

@article{hong2021learning,
  title = {Learning by Fixing: Solving Math Word Problems with Weak Supervision},
  author = {Hong, Yining and Li, Qing and Ciao, Daniel and Huang, Siyuan and Zhu, Song-Chun},
  journal = {AAAI Conference on Artificial Intelligence (AAAI)},
  year = {2021}
}

YouRefIt: Embodied Reference Understanding with Language and Gesture Oral

Yixin Chen , Qing Li , Deqian Kong , Yik Lun Kei , Song-Chun Zhu , Tao Gao , Yixin Zhu , and Siyuan Huang

International Conference on Computer Vision (ICCV), 2021

arXiv Bib

Oral

@article{chen2021yourefit,
  title = {YouRefIt: Embodied Reference Understanding with Language and Gesture},
  author = {Chen, Yixin and Li, Qing and Kong, Deqian and Kei, Yik Lun and Zhu, Song-Chun and Gao, Tao and Zhu, Yixin and Huang, Siyuan},
  journal = {International Conference on Computer Vision (ICCV)},
  year = {2021}
}

VLGrammar: Grounded Grammar Induction of Vision and Language

Yining Hong , Qing Li , Song-Chun Zhu , and Siyuan Huang

International Conference on Computer Vision (ICCV), 2021

arXiv Bib

@article{hong2021vlgrammar,
  title = {VLGrammar: Grounded Grammar Induction of Vision and Language},
  author = {Hong, Yining and Li, Qing and Zhu, Song-Chun and Huang, Siyuan},
  journal = {International Conference on Computer Vision (ICCV)},
  year = {2021}
}

A Competence-Aware Curriculum for Visual Concepts Learning Via Question Answering Oral

Qing Li , Siyuan Huang , Yining Hong , and Song-Chun Zhu

European Conference on Computer Vision (ECCV), 2020

arXiv Bib Website

Oral

@article{li2020competence,
  title = {A Competence-Aware Curriculum for Visual Concepts Learning Via Question Answering},
  author = {Li, Qing and Huang, Siyuan and Hong, Yining and Zhu, Song-Chun},
  journal = {European Conference on Computer Vision (ECCV)},
  year = {2020}
}

Closed Loop Neural-Symbolic Learning Via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning Best Paper in ICML Workshop

Qing Li , Siyuan Huang , Yining Hong , Yixin Chen , Ying Nian Wu , and Song-Chun Zhu

International Conference on Machine Learning (ICML), 2020

arXiv Bib Website

Best Paper in ICML Workshop

@article{li2020ngs,
  title = {Closed Loop Neural-Symbolic Learning Via Integrating Neural Perception, Grammar Parsing, and Symbolic Reasoning},
  author = {Li, Qing and Huang, Siyuan and Hong, Yining and Chen, Yixin and Wu, Ying Nian and Zhu, Song-Chun},
  journal = {International Conference on Machine Learning (ICML)},
  year = {2020}
}