Benyou Wang
Mail: wabyking@gmail.com
Currently, I am an assistant professor in the Chinese University of Hong Kong, Shenzhen (CUHKSZ). I got my phd degree from the University of Padova, Italy (very fortune to be supervised by Massimo Melucci and Emanuele Di Buiccio). See our lab on CUHKSZ LLM group.
I submit my thesis in Sep. 30th 2021 (Here is a draft version)and have it defended in March 2022.
I am recently interested in NLP. The popular NLP killer, i.e., pre-trained language models (PLMs) might be good in terms of effectiveness. However, it is still limited:
PLMs need more understanding and could therefore be improved in the right directions.
PLMs cannot solve more complicated problems e.g., involving reasoning. This will be challenging in recent 10 or more years.
PLMs are too big to deploy (time and space-consuming). Tensor networks may help for time-efficient or space-efficient PLMs.
It is assumed that "the bigger the better" (see GPT 3). PLMs are too expensive to be enlarged. Can quantum computing help to build GPT 10?
Can PLMs help for other domains like biomedical problems? In the biomedical domain, there are many types of sequential tokens (e.g., DNA, proteins, disease codes) that could be trained with PLMs.
I joined CUHKSZ as an assistant professor from June 1st 2022. Please send me emails if you are interested to work with me as a Ph.D., research associate, or post-doc. See JD here
News
- I serve as a Website Chair in EMNLP 2023.
- I serve as a Publicity Chair in NLPCC 2023.
- Our paper ( Doge Ticket ) got the Best Paper Award in NLPCC 2022, see here .
- In September 2022, we got one paper accepted in NeurIPS (named MorphTE ) and another paper in EMNLP (Hypoformer ). Both papers are about compressing transformer models (either in embedding or fully-connected layers)
- In August 2022, one paper got accepted in COLING, which extends deep prompt tuning (DPT) to dense retrieval. By using two additional strategies, DPT got comparable performance with a fine-tuning.
- A joking paper is released: Can we create a new creature?
- A new paper is accepted in ICLR 2022 which could compress 12-layer BERT encoders into 1.5 M while with slight performance drop, see "Exploring extreme parameter compression for pre-trained language models".
- Our paper titled "Word2Fun: Modelling Words as Functions for Diachronic Word Representation" got accepted in NeurIPS 2021 with my supervisors Massimo and Emanuele. This introduces a new paradigm for time-specific word embeddings (e.g., imagining that word "president" in 2018 and 2021 generally refer to different people), with both theoretical advantages and empirical success. This is a kind of work that are smaller, better, and more interpretable.
- Our paper titled "On position embeddings in BERT" got accepted in ICLR 2021. Try searching "position embeddings" in Google .
- Our paper titled "Encoding word order in complex embeddings" got accepted with a spotlight presentation in ICLR 2020 (acceptance rate 6%). Codes were already open-sourced.. This is the first work for rotation-based position embeddings while the previous is translation-based.
- We won the Best explainable NLP paper in NAACL 2019 with 1000 dollars, present our paper together with BERT authors (Best Long paper winner). here
- I got Marie Curry Fellowship to rejoin academia in 2018. Thanks for the generous funding.
- Our book <推荐系统与深度学习> has been published, buy it in JD
- We (IRGAN) won the Best Paper honorable mention award in SIGIR 2017, one of the most-cited papers in SIGIR. See here
Awards
- NAACL 2019 Best Explainable NLP Paper
- SIGIR 2017 best paper award Honorable Mention
- Selected as a Marie Curie Researcher of Quantum Information Access and Retrieval Theory (QUARTZ), a fellowship for Early-stage Researcher, funded by the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 721321
Education
- 2018.10-~: Ph.D student: information engineering, University of Padua, Italy.
- 2014.9-2017.2: Master: pattern recognition and intelligent system, Tianjin University, China.
- 2010.9-2014.6: Bachelor: software engineer, Hubei University of Automotive Technology, China.
Work and Intern
- 2018.6-~: Marie Curie Researcher, Department of Information Engineering, University of Padua, Italy.
- 2020.12: Visiting Scholar. Institute of Theoretical Physics, Chinese Academy of Science (CAS) in Dec. 2020 hosted by Pan Zhang
- 2019.11-2020-2: Visiting Student. University of Montreal from Dec. 2019 to Feb. 2020, hosted by Prof. Jian-Yun Nie.
- 2019.10: Visiting Student (one week). University of Amsterdam, hosted by Prof. Maarten de Rijke.
- 2019.9-2019.12: Visiting Student. DIKU IR Lab in University of Copenhagen, hosted by Prof. Christina Lioma.
- 2017.7-2018.6: Full-time Associate Researcher, Data Application Center, Tencent, China.
- 2017.2-2017.5: Research Assistant, Laboratory of Cognitive Computing and Application, Tianjin University.
- 2015.8--2015.9 & 2016.4-2016.5 Research Intern, Institute of Linguistics , China Academy of Social Sciences Beijing, China.
Professional Activities
After being a faculty. Phd students and Research Assistants under my supervision are underlined
- Zhongwei Wan, Che Liu, Mi Zhang, Jie Fu, Benyou Wang, Sibo Cheng, Lei Ma, César Quilodrán-Casas, Rossella Arcucci. Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias. NeurIPS 2023
- Yazhou Zhang, Yang Yu, Qing Guo, Benyou Wang , Dongming Zhao, Sagar Uprety, Dawei Song, Jing Qin, Qiuchi Li. All In One: A Chinese Multi-Modal Dataset for Multi-Affection Detection in Conversations. NeurIPS 2023 Track Datasets and Benchmarks
- Zhihong Chen, Shizhe Diao, Benyou Wang, Guanbin Li, and Xiang Wan. "Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts". ICCV 2023
- Zhongwei Wan, Xin Liu, Benyou Wang, Jiezhong Qiu, Boyu Li, Ting Guo, Guangyong Chen, Yang Wang. Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendation. Transactions on Information Systems (TOIS)
- Jianquan Li, Xiangbo Wu , Xiaokang Liu , Prayag Tiwari, Qianqian Xie, and Benyou Wang. "Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk". ACL 2023
- Yajiao LIU , Xin Jiang, Yichun Yin, Yasheng Wang, Fei Mi, Qun Liu, Xiang Wan, and Benyou Wang. One Cannot Stand for Everyone! Leveraging Multiple User Simulators to train Task-oriented Dialogue Systems. ACL 2023
- Chen Zhang , Yang Yang, Jiahao Liu, Jingang Wang, Yunsen Xian, Benyou Wang and Dawei Song. Lifting the Curse of Capacity Gap in Distilling Language Models. ACL 2023
- Zhihong Chen , Guiming Hardy Chen , Shizhe Diao, Xiang Wan, and Benyou Wang. On the Difference of BERT-style and CLIP-style Text Encoders. Findings of ACL 2023
- Xiaokang Liu , Jianquan Li , Jingjing Mu, Min Yang, Ruifeng Xu, and Benyou Wang. "Effective Open Intent Classification with K-center Contrastive Learning and Adjustable Decision Boundary". AAAI 2023
- Le Sun, Mingyang Zhang, Benyou Wang and Prayag Tiwari. Few-Shot Class-Incremental Learning for Medical Time Series Classification. IEEE Journal of Biomedical and Health Informatics . 2023
- Yaochen Liu, Qiuchi Li, Benyou Wang Yazhou Zhang, Dawei Song. "A survey of quantum-cognitively inspired sentiment analysis models". ACM Computing Surveys 2023
- Benyou Wang, Qianqian Xie, Jiahuan Pei, Prayag Tiwari, Zhao Li, and Fu Jie. "Pre-trained Language Models in Biomedical Domain: A Survey from Multiscale Perspective". ACM Computing Surveys, .
- Yi Yang, Chen Zhang, Benyou Wang and Dawei Song. "Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets". NLPCC 2022 Best Paper 2022 .
- Sunzhu Li , Peng Zhang, Guobing Gan, Xiuqing Lv, Benyou Wang Junqiu Wei, Xin Jiang. "Hypoformer: Hybrid Decomposition Transformer for Edge-friendly Neural Machine Translation". EMNLP 2022 .
- Guobing Gan, Peng Zhang, Sunzhu Li, Xiuqing Lu, Benyou Wang. "MorphTE: Injecting morphology in tensorized embeddings". NeurIPS 2022 .