Xihui Liu
|
Assistant Professor,
HKU-MMLab
Department of Electrical and Electronic Engineering (EEE) / Institute of Data Science (IDS)
The University of Hong Kong
xihui.liu.me@gmail.com
xihuiliu@eee.hku.hk
Google Scholar CV
|
About me
I am an Assistant Professor at the Department of Electrical and Electronic Engineering, Institute of Data Science (IDS), and Department of Computer Science (by courtesy), as well as HKU-MMLab, The University of Hong Kong.
Before joining HKU, I was a postdoc Scholar at EECS Department and BAIR at UC Berkeley, advised by Prof. Trevor Darrell.
I obtained my Ph.D. degree from Multimedia Lab (MMLab), the Chinese University of Hong Kong, supervised by Prof. Xiaogang Wang and Prof. Hongsheng Li.
I received bachelor's degree in Electronic Engineering in Tsinghua University (THU) in 2017.
My research interests cover computer vision, machine learning, and artificial intelligence, with special emphasis on generative models for image/video/3D generation and multimodal AI with applications in embodied AI.
I was awarded Adobe Research Fellowship 2020, MIT EECS Rising Stars 2021, WAIC Rising Stars Award 2022, CVPR 2021 Doctoral Consortium Award, CVPR Outstanding Reviewer, and ICLR Outstanding Reviewer.
I am looking for self-motivated Ph.D. students to join my group. Please drop me an email if you are interested.
Employment and Education
Assistant Professor, Department of Electrical and Electronic Engineering / Institute of Data Science, The University of Hong Kong. (June 2022 to now)
Postdoc Scholar at Darrell Group / BAIR / EECS Department, UC Berkeley. (August 2021 to June 2022)
Advised by Prof Trevor Darrell.
Ph.D, Department of Electronic Engineering / Multimedia Lab (MMLab), The Chinese University of Hong Kong. (August 2017 to July 2021)
Advised by Prof. Xiaogang Wang and Prof. Hongsheng Li.
Bachelor, Department of Electronic Engineering, Tsinghua University. (August 2013 to July 2017)
Awards
2022 WAIC Rising Stars.
2021 MIT EECS Rising Stars.
2019 Adobe Research Fellowship.
CVPR 2021 Doctoral Consortium, mentored by Prof. Antonio Torralba.
ICLR 2021 outstanding reviewer award.
CVPR 2019 outstanding reviewer award.
2017 Tsinghua Outstanding Graduate.
Champion of ImageNet 2016 Video Object Detection Challenge (as a member of CUVideo team).
Publications
DreamComposer: Controllable 3D Object Generation via Multi-View Conditions
Yunhan Yang, Yukun Huang, Xiaoyang Wu, Yuan-Chen Guo, Song-Hai Zhang, Hengshuang Zhao, Tong He, Xihui Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024.
HumaGaussian: Text-driven 3d Human Generation with Gaussian Splatting
Xian Liu, Xiaohang Zhan, Jiaxiang Tang, Ying Shan, Gang Zeng, Dahua Lin, Xihui Liu, Ziwei Liu
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024.
Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training
Xiaoyang Wu, Zhuotao Tian, Xin Wen, Bohao Peng, Xihui Liu, Kaicheng Yu, Hengshuang Zhao
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024.
EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI
Tai Wang, Xiaohan Mao, Chenming Zhu, Runsen Xu, Ruiyuan Lyu, Peisen Li, Xiao Chen, Wenwei Zhang, Kai Chen, Tianfan Xue, Xihui Liu, Cewu Lu, Dahua Lin, Jiangmiao Pang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024.
Point Transformer V3: Simpler, Faster, Stronger
Xiaoyang Wu, Li Jiang, Peng-Shuai Wang, Zhijian Liu, Xihui Liu, Yu Qiao, Wanli Ouyang, Tong He, Hengshuang Zhao
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2024.
HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion
Xian Liu, Jian Ren, Aliaksandr Siarohin, Ivan Skorokhodov, Yanyu Li, Dahua Lin, Xihui Liu,, bei Liu, Sergey Tulyakov
International Conference on Learning Representations (ICLR) 2024.
FiT: Flexible Vision Transformer for Diffusion Model
Zeyu Lu, Zidong Wang, Di Huang, Chengyue Wu, Xihui Liu, Wanli Ouyang, Lei Bai
2024.
Divide and Conquer: Language Models Can Plan and Self-Correct for Compositional Text-to-Image Generation
Zhenyu Wang, Enze Xie, Aoxue Li, Zhongdao Wang, Xihui Liu, Zhenguo Li
2024.
Drag-A-Video: Non-rigid Video Editing with Point-based Interaction
Yao Teng, Enze Xie, Yue Wu, Haoyu Han, Zhenguo Li, Xihui Liu
2023.
EgoPlan-Bench: Benchmarking Egocentric Embodied Planning with Multimodal Large Language Models
Yi Chen, Yuying Ge, Yixiao Ge, Mingyu Ding, Bohao Li, Rui Wang, Ruifeng Xu, Ying Shan, Xihui Liu
2023.
TVTSv2: Learning Out-of-the-box Spatiotemporal Visual Representations at Scale
Ziyun Zeng, Zhan Tong, Xihui Liu, Bin Chen, Shu-Tao Xia, Yixiao Ge
2023.
T2I-CompBench: A Comprehensive Benchmark for Open-world Compositional Text-to-image Generation
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, Xihui Liu
Conference on Neural Information Processing Systems (NeurIPS), 2023.
CorresNeRF: Image Correspondence Priors for Neural Radiance Fields
Yixing Lao, Xiaogang Xu, Zhipeng Cai, Xihui Liu, Hengshuang Zhao
Conference on Neural Information Processing Systems (NeurIPS), 2023.
OV-PARTS: Towards Open-Vocabulary Part Segmentation
Meng Wei, Xiaoyu Yue, Wenwei Zhang, Shu Kong, Xihui Liu, Jiangmiao Pang
Conference on Neural Information Processing Systems (NeurIPS), 2023.
Seeing is not always believing: A Quantitative Study on Human Perception of AI-Generated Images
Zeyu Lu*, Di Huang*, Lei Bai*, Xihui Liu, Jingjing Qu, Wanli Ouyang
Conference on Neural Information Processing Systems (NeurIPS), 2023.
SAM3D: Segment Anything in 3D Scenes
Yunhan Yang, Xiaoyang Wu, Tong He, Hengshuang Zhao, Xihui Liu
Technical report, 2023.
Shape-Guided Diffusion with Inside-Outside Attention
Dong Huk Park*, Grace Luo*, Clayton Andrew Toste, Samaneh Azadi, Xihui Liu, Makrine Karalashvili, Anna Rohrbach, Trevor Darrell
Winter Conference on Applications of Computer Vision (WACV) 2024.
Hierarchical Diffusion Autoencoders and Disentangled Image Manipulation
Zeyu Lu, Chengyue Wu, Xinyuan Chen, Yaohui Wang, Yu Qiao, Xihui Liu
Winter Conference on Applications of Computer Vision (WACV) 2024.
Ddp: Diffusion model for dense visual prediction
Yuanfeng Ji*, Zhe Chen*, Enze Xie, Lanqing Hong, Xihui Liu, Zhaoqiang Liu, Tong Lu, Zhenguo Li, Ping Luo
International Conference on Computer Vision (ICCV), 2023.
Back to the Source: Diffusion-Driven Test-Time Adaptation
Jin Gao*, Jialing Zhang*, Xihui Liu, Trevor Darrell, Evan Shelhamer*, Dequan Wang*
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023.
Learning Transferable Spatiotemporal Representations from Natural Script Knowledge
Ziyun Zeng*, Yuying Ge*, Xihui Liu, Bin Chen, Ping Luo, Shu-Tao Xia, Yixiao Ge
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023.
Masked Scene Contrast: A Scalable Framework for Unsupervised 3D Representation Learning
Xiaoyang Wu, Xin Wen, Xihui Liu, Hengshuang Zhao
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023.
RIFormer: Keep Your Vision Backbone Effective But Removing Token Mixer
Jiahao Wang, Songyang Zhang, Yong Liu, Taiqiang Wu, Yujiu Yang, Xihui Liu, Kai Chen, Ping Luo, Dahua Lin
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023.
GLeaD: Improving GANs with A Generator-Leading Task
Qingyan Bai, Ceyuan Yang, Yinghao Xu, Xihui Liu, Yujiu Yang, Yujun Shen
IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2023.
More Control for Free! Image Synthesis with Semantic Diffusion Guidance [Project Page] [Slides]
Xihui Liu, Dong Huk Park, Samaneh Azadi, Gong Zhang, Arman Chopikyan, Yuxiao Hu, Humphrey Shi, Anna Rohrbach, Trevor Darrell
Winter Conference on Applications of Computer Vision (WACV) 2023.
Point Transformer V2: Grouped Vector Attention and Partition-based Pooling
Xiaoyang Wu, Yixing Lao, Li Jiang, Xihui Liu, Hengshuang Zhao
Conference on Neural Information Processing Systems (NeurIPS), 2022.
MILES: Visual BERT Pre-training with Injected Language Semantics for Video-text Retrieval
Yuying Ge, Yixiao Ge, Xihui Liu, Alex Jinpeng Wang, Jianping Wu, Ying Shan, Xiaohu Qie, Ping Luo
European Conference on Computer Vision (ECCV), 2022.
BridgeFormer: Bridging Video-text Retrieval with Multiple Choice Questions [Project Page]
Yuying Ge, Yixiao Ge, Xihui Liu, Dian Li, Ying Shan, Xiaohu Qie, Ping Luo
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Benchmark for Compositional Text-to-Image Synthesis
Dong Huk Park, Samaneh Azadi, Xihui Liu, Trevor Darrell, Anna Rohrbach
Conference on Neural Information Processing Systems (NeurIPS) Dataset and Benchmark Track, 2021.
Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions [Video] [Slides] [Code]
Xihui Liu, Zhe Lin, Jianming Zhang, Handong Zhao, Quan Tran, Xiaogang Wang, Hongsheng Li
European Conference on Computer Vision (ECCV), 2020.
Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis [Code] [Slides]
Xihui Liu, Guojun Yin, Jing Shao, Xiaogang Wang, Hongsheng Li
Conference on Neural Information Processing Systems (NeurIPS), 2019.
Improving Referring Expression Grounding with Cross-modal Attention-guided Erasing [Code]
Xihui Liu, Zihao Wang, Jing Shao, Xiaogang Wang, Hongsheng Li
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
Show, Tell and Discriminate: Image Captioning by Self-retrieval with Partially Labeled Data
Xihui Liu, Hongsheng Li, Jing Shao, Dapeng Chen, Xiaogang Wang
European Conference on Computer Vision (ECCV), 2018.
CAMP: Cross-modal Adaptive Message Passing for Text-Image Retrieval [Code]
Zihao Wang*, Xihui Liu*, Hongsheng Li, Lu Sheng, Junjie Yan, Xiaogang Wang, Jing Shao
International Conference on Computer Vision (ICCV), 2019.
Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association
Dapeng Chen, Hongsheng Li, Xihui Liu, Yantao Shen, Jing Shao, Zejian Yuan, Xiaogang Wang
European Conference on Computer Vision (ECCV), 2018.
HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis
Xihui Liu*, Haiyu Zhao*, Maoqing Tian, Lu Sheng, Jing Shao, Shuai Yi, Junjie Yan, Xiaogang Wang
International Conference on Computer Vision (ICCV), 2017.
Localization Guided Learning for Pedestrian Attribute Recognition
Pengze Liu, Xihui Liu, Junjie Yan, Jing Shao,
British Machine Vision Conference (BMVC), 2018.
Orientation invariant feature embedding and spatial temporal regularization for vehicle re-identification
Zhongdao Wang, Luming Tang, Xihui Liu, Zhuliang Yao, Shuai Yi, Jing Shao, Junjie Yan, Shengjin Wang, Hongsheng Li, Xiaogang Wang
International Conference on Computer Vision (ICCV), 2017.
Object Detection in Videos With Tubelet Proposal Networks
Kai Kang, Hongsheng Li, Tong Xiao, Wanli Ouyang, Junjie Yan, Xihui Liu, Xiaogang Wang
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Academic Services
Area chair of CVPR 2024.
Reviewer of the following conferences and journals:
Outstanding Reviewer for CVPR 2019 and ICLR 2021.
CVPR, ICCV, ECCV, NeruIPS, ICML, ICLR, AAAI.
IJCV, TCSVT, NeuroComputing, TMM.
Research Internships
Teaching Experience
Teaching the following courses at The University of Hong Kong:
ELEC 4542, Introduction to Deep Learning for Computer Vision, Fall 2023.
DATA 8009, Advanced Deep Learning for Computer Vision, Spring 2024.
Teaching Assistant of the following courses at The Chinese University of Hong Kong:
ELEG 5491, Introduction to Deep Learning [Course website], Spring 2019.
ENGG 5202, Pattern Recognition, Fall 2017, Spring 2020.
ELEG 5760, Machine Learning for Signal Processing Applications, Fall 2018.
ENGG 2450A, Probability and Statistics for Engineers, Spring 2018, Fall 2020.
Summer Tutorial, Potential Inspiration in Electronic Engineering, Summer 2018.
ENGG 2420B, Complex Analyis and Differential Equations for Engineers, Fall 2019.
ENGG 2740A, Differential Equations for Engineers, Spring 2021.
|