Benyou Wang

Mail:   wabyking@gmail.com

Currently, I am an assistant professor in the Chinese University of Hong Kong, Shenzhen (CUHKSZ). I got my phd degree from the University of Padova, Italy (very fortune to be supervised by Massimo Melucci and Emanuele Di Buiccio). See our lab on CUHKSZ LLM group.

I submit my thesis in Sep. 30th 2021 （Here is a draft version）and have it defended in March 2022.

I joined CUHKSZ as an assistant professor from June 1st 2022. Please send me emails if you are interested to work with me as a Ph.D., research associate, or post-doc. See JD here

News

  1. I serve as a Website Chair in EMNLP 2023.
  2. I serve as a Publicity Chair in NLPCC 2023.
  3. Our paper ( Doge Ticket ) got the Best Paper Award in NLPCC 2022, see here .
  4. In September 2022, we got one paper accepted in NeurIPS (named MorphTE ) and another paper in EMNLP (Hypoformer ). Both papers are about compressing transformer models (either in embedding or fully-connected layers)
  5. In August 2022, one paper got accepted in COLING, which extends deep prompt tuning (DPT) to dense retrieval. By using two additional strategies, DPT got comparable performance with a fine-tuning.
  6. A joking paper is released: Can we create a new creature?
  7. A new paper is accepted in ICLR 2022 which could compress 12-layer BERT encoders into 1.5 M while with slight performance drop, see "Exploring extreme parameter compression for pre-trained language models".
  8. Our paper titled "Word2Fun: Modelling Words as Functions for Diachronic Word Representation" got accepted in NeurIPS 2021 with my supervisors Massimo and Emanuele. This introduces a new paradigm for time-specific word embeddings (e.g., imagining that word "president" in 2018 and 2021 generally refer to different people), with both theoretical advantages and empirical success. This is a kind of work that are smaller, better, and more interpretable.
  9. Our paper titled "On position embeddings in BERT" got accepted in ICLR 2021. Try searching "position embeddings" in Google .
  10. Our paper titled "Encoding word order in complex embeddings" got accepted with a spotlight presentation in ICLR 2020 (acceptance rate 6%). Codes were already open-sourced.. This is the first work for rotation-based position embeddings while the previous is translation-based.
  11. We won the Best explainable NLP paper in NAACL 2019 with 1000 dollars, present our paper together with BERT authors (Best Long paper winner). here
  12. I got Marie Curry Fellowship to rejoin academia in 2018. Thanks for the generous funding.
  13. Our book <推荐系统与深度学习> has been published, buy it in JD
  14. We (IRGAN) won the Best Paper honorable mention award in SIGIR 2017, one of the most-cited papers in SIGIR. See here

Awards

  1. NAACL 2019 Best Explainable NLP Paper
  2. SIGIR 2017 best paper award Honorable Mention
  3. Selected as a Marie Curie Researcher of Quantum Information Access and Retrieval Theory (QUARTZ), a fellowship for Early-stage Researcher, funded by the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 721321

Education

Work and Intern

Professional Activities

Publications @ Google Scholar 

After being a faculty. Phd students and Research Assistants under my supervision are underlined

  1. Xianghong Fang, Jian Li, Qiang Sun, Benyou Wang. Rethinking the Uniformity Metric in Self-Supervised Learning. ICLR 2024
  2. Xidong Wang , Guiming Hardy Chen, Dingjie Song, Zhiyi Zhang, Zhihong Chen, Qingying Xiao, Feng Jiang, Jianquan Li, Xiang Wan, Benyou Wang, Haizhou Li. CMB: A Comprehensive Medical Benchmark in Chinese. 2023. NAACL 2024   Online leaderboard
  3. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Juncai He, Ziche Liu, Zhiyi Zhang, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, Jinchao Xu. AceGPT, Localizing Large Language Models in Arabic. NAACL 2024. ( HuggingFace downloading: 10K per month.)
  4. Fei Yu, Anningzhe Gao, Benyou Wang. Outcome-supervised verifiers for planning in mathematical reasoning. Findings of NAACL 2024. (The work brings 7B LLMs to the era with an accuracy of 0.8 and even 0.9 in GSM8K, see the leaderboard )
  5. Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Chen, Jianquan Li, Guiming Chen, Xiangbo Wu, Zhiyi Zhang, Qingying Xiao, Xiang Wan, Benyou Wang Haizhou Li. HuatuoGPT, towards Taming Language Model to Be a Doctor. 2023. Findings of EMNLP 2023 (GitHub stars: 1K; Online access : 400K+ ) )
  6. Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, Jianquan Li, Xiang Wan, Benyou Wang , Haizhou Li. "Phoenix: Democratizing chatgpt across languages". Arxiv code (Github stars: 3K)   HuggingFace downloading: 4K per month
  7. Fei Yu, Hongbo Zhang, Prayag Tiwari, and Benyou Wang. Natural language reasoning, a survey. 2023. ACM Computing Survey.
  8. Zhongwei Wan, Che Liu, Mi Zhang, Jie Fu, Benyou Wang, Sibo Cheng, Lei Ma, César Quilodrán-Casas, Rossella Arcucci. Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias. NeurIPS 2023
  9. Yazhou Zhang, Yang Yu, Qing Guo, Benyou Wang , Dongming Zhao, Sagar Uprety, Dawei Song, Jing Qin, Qiuchi Li. All In One: A Chinese Multi-Modal Dataset for Multi-Affection Detection in Conversations. NeurIPS 2023 Track Datasets and Benchmarks
  10. Zhihong Chen, Shizhe Diao, Benyou Wang, Guanbin Li, and Xiang Wan. "Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts". ICCV 2023
  11. Zhongwei Wan, Xin Liu, Benyou Wang, Jiezhong Qiu, Boyu Li, Ting Guo, Guangyong Chen, Yang Wang. Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendation. Transactions on Information Systems (TOIS)
  12. Jianquan Li, Xiangbo Wu , Xiaokang Liu , Prayag Tiwari, Qianqian Xie, and Benyou Wang. "Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk". ACL 2023
  13. Yajiao LIU , Xin Jiang, Yichun Yin, Yasheng Wang, Fei Mi, Qun Liu, Xiang Wan, and Benyou Wang. One Cannot Stand for Everyone! Leveraging Multiple User Simulators to train Task-oriented Dialogue Systems. ACL 2023
  14. Chen Zhang , Yang Yang, Jiahao Liu, Jingang Wang, Yunsen Xian, Benyou Wang and Dawei Song. Lifting the Curse of Capacity Gap in Distilling Language Models. ACL 2023
  15. Zhihong Chen , Guiming Hardy Chen , Shizhe Diao, Xiang Wan, and Benyou Wang. On the Difference of BERT-style and CLIP-style Text Encoders. Findings of ACL 2023
  16. Xiaokang Liu , Jianquan Li , Jingjing Mu, Min Yang, Ruifeng Xu, and Benyou Wang. "Effective Open Intent Classification with K-center Contrastive Learning and Adjustable Decision Boundary". AAAI 2023
  17. Le Sun, Mingyang Zhang, Benyou Wang and Prayag Tiwari. Few-Shot Class-Incremental Learning for Medical Time Series Classification. IEEE Journal of Biomedical and Health Informatics . 2023
  18. Yaochen Liu, Qiuchi Li, Benyou Wang Yazhou Zhang, Dawei Song. "A survey of quantum-cognitively inspired sentiment analysis models". ACM Computing Surveys 2023
  19. Benyou Wang, Qianqian Xie, Jiahuan Pei, Prayag Tiwari, Zhao Li, and Fu Jie. "Pre-trained Language Models in Biomedical Domain: A Survey from Multiscale Perspective". ACM Computing Surveys, .
  20. Yi Yang, Chen Zhang, Benyou Wang and Dawei Song. "Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets". NLPCC 2022 Best Paper 2022 .
  21. Sunzhu Li , Peng Zhang, Guobing Gan, Xiuqing Lv, Benyou Wang Junqiu Wei, Xin Jiang. "Hypoformer: Hybrid Decomposition Transformer for Edge-friendly Neural Machine Translation". EMNLP 2022 .
  22. Guobing Gan, Peng Zhang, Sunzhu Li, Xiuqing Lu, Benyou Wang. "MorphTE: Injecting morphology in tensorized embeddings". NeurIPS 2022 .

Before being a faculty

  1. Benyou Wang, Yuxin Ren, Lifeng Shang, Xin Jiang, Qun Liu. "Exploring extreme parameter compression for pre-trained language models". ICLR 2022, .
  2. Peng Zhang, Wenjie Hui, Benyou Wang (corresponding), Donghao Zhao, Dawei Song, Christina Lioma, Jakob Grue Simonsen. "Complex-valued Neural Network-based Quantum Language Models". ACM Transactions on Information Systems .
  3. Benyou Wang, Emanuele Di Buiccio, Massimo Melucci. Word2fun, modeling words as functions for dynamic word embeddings. NeurIPS 2021, .
  4. Benyou Wang, Lifeng Shang, Christina Lioma, Xin Jiang, Qun Liu, Jakob Grue Simonsen. On position embeddings in BERT. ICLR 2021,
  5. Benyou Wang*, Donghao Zhao*, Christina Lioma, Qiuchi Li, Peng Zhang, Jakob Grue Simonsen. Encoding word order in complex embeddings. ICLR 2020, Spotlight paper (acceptance rate: 6%)
  6. Qiuchi Li*, Benyou Wang*, Massimo Melucci. A Complex-valued Network for Matching. NAACL 2019, Best Explainable NLP Paper
  7. Benyou Wang. Dynamic content monitoring and exploration using vector spaces. SIGIR 2019 doctoral consortium. 
  8. Benyou Wang*, Qiuchi Li*, Massimo Melucci, Dawei Song. Semantic Hilbert Space for Text Representation Learning. WWW 2019
  9. Wei Zhao*, Benyou Wang*, Min Yang, Jianbo Ye, Zhou Zhao, Xiaojun Chen, Ying Shen.. Leveraging Long and Short-term Information in Content-aware Movie Recommendation via Adversarial Training. IEEE Transactions on Cybernetics (TOC), 2019 (IF: 8.803)
  10. Peng Zhang, Zhan Su, Lipeng Zhang, Benyou Wang , Dawei Song. 2018. A Quantum Many-body Wave Function Inspired Language Modeling Approach. CIKM 2018
  11. Wei Zhao, Wang Benyou , Jianbo Ye, Yongqiang Gao, Min Yang, Xiaojun Chen, PLASTIC: Prioritize Long and Short-term Information in Top-n Recommendation using Adversarial Training, IJCAI 2018
  12. Wei Zhao, Wang Benyou , Jianbo Ye, Min Yang, Zhou Zhao, Ruotian Luo, Yu Qiao A Multi-task Learning Approach for Image Captioning, IJCAI 2018
  13. Zhang Peng, Niu Jiabing, Su Zhan, Wang Benyou et al. End-to-End Quantum-like Language Models with Application to Question Answering AAAI 2018 
  14. Wang Jun, Yu Lantao, Zhang Weinan, Gong Yu, Xu Yinghui, Wang Benyou , Zhang Peng, Zhang Dell. IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models. SIGIR 2017. Best Paper Award Honourable Mentions . Zhihu link (in Chinses)
  15. Wang Benyou, Niu Jiabing, Ma Liqun, Zhang Yuhua, Zhang Lipeng, Li Jinfei, Zhang Peng Song, D. . A Chinese Question Answering Approach Integrating Count-Based and Embedding-Based Features. ICCPOL-NLPCC . December, 2016
  16. Wang Benyou, Zhang Peng, Li Jinfei, Song Dawei, Hou Yuexian, Shang Zhenguo. Exploration of quantum interference in document relevance judgement discrepancy. Entropy , 18(4), 144. 2016. (IF : 1.821)
  17. Chen Yongqiang, Zhang Peng, Song Dawei, Wang Benyou. A Real-Time Eye Tracking Based Query Expansion Approach via Latent Topic Modeling. CIKM 2015 (pp. 1719-1722). ACM. October, 2015

Book and book chapter

  1. Huang Xin, Wei zhao, Wang Benyou, Rui Zhao. Recommendation System and Deep Learning, Tsinghua University Press, in Chinese. Focusing on the chapters related "Learn to rank" and "Generative Adversarial Nets(GAN) for Recommendation". Online purchase link: JD and Dangdang. 
  2. Wang,B. , Emanuele Di, B., & Melucci, M.. Representing words in vector space and beyond . In A. Diederik, K. Andrei, M. Massimo, & T. Bourama (Eds.),Quantum-like models forinformation retrieval and decision-making. Springer.

Talks

  1. Sequencial Modeling in Vector Spaces, the Italian Information Retreival Workshop, in Sep 2021
  2. On Position embeddings , the China Student Symposium on NLP (CSSNLP), Beijing, in December, 2020
  3. Invited lecture: Quantum theory and NLP , for bachelor students, Beijing Institute of Technolohy, Beijing, in December, 2020
  4. Invited lecture: pretrained language model and its position embeddings for bachelor students, Shandong University, Qingdao, in Dec. 2020
  5. How physics and NLP help each other? , Institute of theoretical Physics, Chinese Academy of Science (CAS) Beijing, in December, 2020
  6. On Position embeddings , Alibaba, Beijing in December, 2020
  7. , Formulizing semantic shift detection as a distance between sets EVALITA 2020. Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Diachronic Lexical Semantics online, on December 17th 2020
  8. How quantum theory contributes to NLP, First workshop of quantum computing and AI, virtually, previously in Tianjin University, Tianjin, China, 22. Nov. 2020
  9. Encoding word order in complex embeddings, Speech and Language Computing Group, Huawei Noah's Ark Lab, Shenzhen, China, 23. April 2020
  10. Dynamic Content Monitoring and Exploration using Vector Spaces, University of Bedfordshire, Luton, Lonton, UK, 12 Feb. 2020
  11. Quantum Mechanics meet Information Search and Retrieval – The QUARTZ Project, Great London Text Analytics meetup, London, UK, 12. Feb. 2020
  12. Investigating complex-valued representation in NLP, Mila, Mila, Montreal, Canada, 20. Jan 2020
  13. Beyond particles: modeling words as waves , RALI Département d’informatique et recherche opérationnelle, University of Montreal, Montreal, Canada, Dec. 7th 2019
  14. Beyond particles: modeling words as waves , DIKU machine learning section, University of Copenhagen, Copenhagen, Denmark, Nov. 25th 2019
  15. Quantum formulations for language: understand words as particles , meetup Search Engines Amsterdam , University of Amsterdam, Amsterdam. Netherlands, Oct. 25th 2019
  16. Dynamic Content Monitoring and Exploration using Vector Spaces , SIGIR Doctoral Consortium, Paris, France. July, 2019
  17. 2019 Joint Statistics Summer School by Univeristy of Bolzano, Padova and Salzburg, Brixen, Italy. July 11, 2019
  18. Quantum-inspired NLP/IR . Bytedance AI Lab, Hang Li's group, Beijing, China. June 28, 2019
  19. Quantum-inspired NLP/IR. Tencent Cloud NLP team (Zhiwen Lab), Shenzhen, China, June 21, 2019
  20. Representing and interpreting words in vector space inspired by Quantum theory. Quartz Workshop, University of Copenhagen
  21. Tensor analysis for DL. Functional Analysis, University of Padova
  22. Word embedding and the beyond.> IR group, University of Padova
  23. Deep Learning in language : offline workshop in Padova 
  24. Research discussion : Quartz project, University of Padova, Italy, 2018 Oct. 
  25. Individual Research Project for Quartz : Quartz project, University of Padova, Italy, 2018 Oct. 
  26. Interpretable Neural network driven byquantum probability theory : Quartz project, Germany. 2018 Sep. 
  27. Exploring Interpretable Quantum Representation for language understanding : Tianjin University, China. 2018 Sep. 
  28. Exploring Interpretable Quantum Representation for language understanding : Tencent, China. 2018 Sep. 
  29. Exploring Interpretable Neural Network by Quantum representation : Quartz workshop in Iatly, Padova. 2018 Sep. 
  30. Representations and their matching: an overview of my previous research : Padova, Italy. 2018.7 
  31. TextZOO, a new Benchmark to Reconsidering Text Classification : Data Center, SNG, Tencent, Shenzhen, China. 2018.3.29 
  32. Neural Network based Quantum Language Model for QA : "AAAI 2018 Spotlights Proseminar ", Tencent, Shenzhen, China. 2018.3.28 
  33. Quantum-inspired Neural Network : Quartz Winter School 2018 , Padova, Italy 2018.2.14
  34. ChatBot in DSNO : Apartment of product for instant communication, SNG, Tencent, Shenzhen, China. 
  35. Quantum Language Model for QA : Quantum Penguin, Tencent, to be appeared 
  36. Detail of our ChatBot : Apartment of product for instant communication, SNG, Tencent 
  37. Progress of our ChatBot : the only non-leader 15-min Speaker in Center of Data Application, SNG, Tencent 

Teaching

  1. CSC 6201/CIE 6021 Large Language Models.