Benyou Wang

Mail:   wabyking@gmail.com

Currently, I am an assistant professor in the Chinese University of Hong Kong, Shenzhen (CUHKSZ). I got my phd degree from the University of Padova, Italy (very fortune to be supervised by Massimo Melucci and Emanuele Di Buiccio). See our lab on CUHKSZ LLM group.

I submit my thesis in Sep. 30th 2021 （Here is a draft version）and have it defended in March 2022.

I joined CUHKSZ as an assistant professor from June 1st 2022. Please send me emails if you are interested to work with me as a Ph.D., research associate, or post-doc. See JD here

News

  1. 2024年入选腾讯犀牛鸟项目
  2. 2023年获得华为火花奖
  3. I serve as a Website Chair in EMNLP 2023.
  4. I serve as a Publicity Chair in NLPCC 2023.
  5. Our paper ( Doge Ticket ) got the Best Paper Award in NLPCC 2022, see here .
  6. In September 2022, we got one paper accepted in NeurIPS (named MorphTE ) and another paper in EMNLP (Hypoformer ). Both papers are about compressing transformer models (either in embedding or fully-connected layers)
  7. In August 2022, one paper got accepted in COLING, which extends deep prompt tuning (DPT) to dense retrieval. By using two additional strategies, DPT got comparable performance with a fine-tuning.
  8. A joking paper is released: Can we create a new creature?
  9. A new paper is accepted in ICLR 2022 which could compress 12-layer BERT encoders into 1.5 M while with slight performance drop, see "Exploring extreme parameter compression for pre-trained language models".
  10. Our paper titled "Word2Fun: Modelling Words as Functions for Diachronic Word Representation" got accepted in NeurIPS 2021 with my supervisors Massimo and Emanuele. This introduces a new paradigm for time-specific word embeddings (e.g., imagining that word "president" in 2018 and 2021 generally refer to different people), with both theoretical advantages and empirical success. This is a kind of work that are smaller, better, and more interpretable.
  11. Our paper titled "On position embeddings in BERT" got accepted in ICLR 2021. Try searching "position embeddings" in Google .
  12. Our paper titled "Encoding word order in complex embeddings" got accepted with a spotlight presentation in ICLR 2020 (acceptance rate 6%). Codes were already open-sourced.. This is the first work for rotation-based position embeddings while the previous is translation-based.
  13. We won the Best explainable NLP paper in NAACL 2019 with 1000 dollars, present our paper together with BERT authors (Best Long paper winner). here
  14. I got Marie Curry Fellowship to rejoin academia in 2018. Thanks for the generous funding.
  15. Our book <推荐系统与深度学习> has been published, buy it in JD
  16. We (IRGAN) won the Best Paper honorable mention award in SIGIR 2017, one of the most-cited papers in SIGIR. See here

Awards

  1. NAACL 2019 Best Explainable NLP Paper
  2. SIGIR 2017 best paper award Honorable Mention
  3. Selected as a Marie Curie Researcher of Quantum Information Access and Retrieval Theory (QUARTZ), a fellowship for Early-stage Researcher, funded by the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 721321

Education

Work and Intern

Professional Activities

Publications @ Google Scholar 

After being a faculty. Phd students and Research Assistants under my supervision are underlined

  1. Zhengyang Tang, Xingxing Zhang, Benyou Wang, Furu Wei. MathScale: Scaling Instruction Tuning for Mathematical Reasoning ICML 2024
  2. Xianghong Fang, Jian Li, Qiang Sun, Benyou Wang. Rethinking the Uniformity Metric in Self-Supervised Learning. ICLR 2024
  3. Xidong Wang , Guiming Hardy Chen, Dingjie Song, Zhiyi Zhang, Zhihong Chen, Qingying Xiao, Feng Jiang, Jianquan Li, Xiang Wan, Benyou Wang, Haizhou Li. CMB: A Comprehensive Medical Benchmark in Chinese. 2023. NAACL 2024   Online leaderboard
  4. Huang Huang, Fei Yu, Jianqing Zhu, Xuening Sun, Hao Cheng, Dingjie Song, Zhihong Chen, Abdulmohsen Alharthi, Bang An, Juncai He, Ziche Liu, Zhiyi Zhang, Junying Chen, Jianquan Li, Benyou Wang, Lian Zhang, Ruoyu Sun, Xiang Wan, Haizhou Li, Jinchao Xu. AceGPT, Localizing Large Language Models in Arabic. NAACL 2024. ( HuggingFace downloading: 10K per month.)
  5. Fei Yu, Anningzhe Gao, Benyou Wang. Outcome-supervised verifiers for planning in mathematical reasoning. Findings of NAACL 2024. (The work brings 7B LLMs to the era with an accuracy of 0.8 and even 0.9 in GSM8K, see the leaderboard )
  6. Hongbo Zhang, Junying Chen, Feng Jiang, Fei Yu, Zhihong Chen, Jianquan Li, Guiming Chen, Xiangbo Wu, Zhiyi Zhang, Qingying Xiao, Xiang Wan, Benyou Wang Haizhou Li. HuatuoGPT, towards Taming Language Model to Be a Doctor. 2023. Findings of EMNLP 2023 (GitHub stars: 1K; Online access : 400K+ ) )
  7. Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, Jianquan Li, Xiang Wan, Benyou Wang , Haizhou Li. "Phoenix: Democratizing chatgpt across languages". Arxiv code (Github stars: 3K)   HuggingFace downloading: 4K per month
  8. Fei Yu, Hongbo Zhang, Prayag Tiwari, and Benyou Wang. Natural language reasoning, a survey. 2023. ACM Computing Survey.
  9. Zhongwei Wan, Che Liu, Mi Zhang, Jie Fu, Benyou Wang, Sibo Cheng, Lei Ma, César Quilodrán-Casas, Rossella Arcucci. Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias. NeurIPS 2023
  10. Yazhou Zhang, Yang Yu, Qing Guo, Benyou Wang , Dongming Zhao, Sagar Uprety, Dawei Song, Jing Qin, Qiuchi Li. All In One: A Chinese Multi-Modal Dataset for Multi-Affection Detection in Conversations. NeurIPS 2023 Track Datasets and Benchmarks
  11. Zhihong Chen, Shizhe Diao, Benyou Wang, Guanbin Li, and Xiang Wan. "Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts". ICCV 2023
  12. Zhongwei Wan, Xin Liu, Benyou Wang, Jiezhong Qiu, Boyu Li, Ting Guo, Guangyong Chen, Yang Wang. Spatio-Temporal Contrastive Learning Enhanced GNNs for Session-based Recommendation. Transactions on Information Systems (TOIS)
  13. Jianquan Li, Xiangbo Wu , Xiaokang Liu , Prayag Tiwari, Qianqian Xie, and Benyou Wang. "Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk". ACL 2023
  14. Yajiao LIU , Xin Jiang, Yichun Yin, Yasheng Wang, Fei Mi, Qun Liu, Xiang Wan, and Benyou Wang. One Cannot Stand for Everyone! Leveraging Multiple User Simulators to train Task-oriented Dialogue Systems. ACL 2023
  15. Chen Zhang , Yang Yang, Jiahao Liu, Jingang Wang, Yunsen Xian, Benyou Wang and Dawei Song. Lifting the Curse of Capacity Gap in Distilling Language Models. ACL 2023
  16. Zhihong Chen , Guiming Hardy Chen , Shizhe Diao, Xiang Wan, and Benyou Wang. On the Difference of BERT-style and CLIP-style Text Encoders. Findings of ACL 2023
  17. Xiaokang Liu , Jianquan Li , Jingjing Mu, Min Yang, Ruifeng Xu, and Benyou Wang. "Effective Open Intent Classification with K-center Contrastive Learning and Adjustable Decision Boundary". AAAI 2023
  18. Le Sun, Mingyang Zhang, Benyou Wang and Prayag Tiwari. Few-Shot Class-Incremental Learning for Medical Time Series Classification. IEEE Journal of Biomedical and Health Informatics . 2023
  19. Yaochen Liu, Qiuchi Li, Benyou Wang Yazhou Zhang, Dawei Song. "A survey of quantum-cognitively inspired sentiment analysis models". ACM Computing Surveys 2023
  20. Benyou Wang, Qianqian Xie, Jiahuan Pei, Prayag Tiwari, Zhao Li, and Fu Jie. "Pre-trained Language Models in Biomedical Domain: A Survey from Multiscale Perspective". ACM Computing Surveys, .
  21. Yi Yang, Chen Zhang, Benyou Wang and Dawei Song. "Doge Tickets: Uncovering Domain-general Language Models by Playing Lottery Tickets". NLPCC 2022 Best Paper 2022 .
  22. Sunzhu Li , Peng Zhang, Guobing Gan, Xiuqing Lv, Benyou Wang Junqiu Wei, Xin Jiang. "Hypoformer: Hybrid Decomposition Transformer for Edge-friendly Neural Machine Translation". EMNLP 2022 .
  23. Guobing Gan, Peng Zhang, Sunzhu Li, Xiuqing Lu, Benyou Wang. "MorphTE: Injecting morphology in tensorized embeddings". NeurIPS 2022 .

Before being a faculty

  1. Benyou Wang, Yuxin Ren, Lifeng Shang, Xin Jiang, Qun Liu. "Exploring extreme parameter compression for pre-trained language models". ICLR 2022, .
  2. Peng Zhang, Wenjie Hui, Benyou Wang (corresponding), Donghao Zhao, Dawei Song, Christina Lioma, Jakob Grue Simonsen. "Complex-valued Neural Network-based Quantum Language Models". ACM Transactions on Information Systems .
  3. Benyou Wang, Emanuele Di Buiccio, Massimo Melucci. Word2fun, modeling words as functions for dynamic word embeddings. NeurIPS 2021, .
  4. Benyou Wang, Lifeng Shang, Christina Lioma, Xin Jiang, Qun Liu, Jakob Grue Simonsen. On position embeddings in BERT. ICLR 2021,
  5. Benyou Wang*, Donghao Zhao*, Christina Lioma, Qiuchi Li, Peng Zhang, Jakob Grue Simonsen. Encoding word order in complex embeddings. ICLR 2020, Spotlight paper (acceptance rate: 6%)
  6. Qiuchi Li*, Benyou Wang*, Massimo Melucci. A Complex-valued Network for Matching. NAACL 2019, Best Explainable NLP Paper
  7. Benyou Wang. Dynamic content monitoring and exploration using vector spaces. SIGIR 2019 doctoral consortium. 
  8. Benyou Wang*, Qiuchi Li*, Massimo Melucci, Dawei Song. Semantic Hilbert Space for Text Representation Learning. WWW 2019
  9. Wei Zhao*, Benyou Wang*, Min Yang, Jianbo Ye, Zhou Zhao, Xiaojun Chen, Ying Shen.. Leveraging Long and Short-term Information in Content-aware Movie Recommendation via Adversarial Training. IEEE Transactions on Cybernetics (TOC), 2019 (IF: 8.803)
  10. Peng Zhang, Zhan Su, Lipeng Zhang, Benyou Wang , Dawei Song. 2018. A Quantum Many-body Wave Function Inspired Language Modeling Approach. CIKM 2018
  11. Wei Zhao, Wang Benyou , Jianbo Ye, Yongqiang Gao, Min Yang, Xiaojun Chen, PLASTIC: Prioritize Long and Short-term Information in Top-n Recommendation using Adversarial Training, IJCAI 2018
  12. Wei Zhao, Wang Benyou , Jianbo Ye, Min Yang, Zhou Zhao, Ruotian Luo, Yu Qiao A Multi-task Learning Approach for Image Captioning, IJCAI 2018
  13. Zhang Peng, Niu Jiabing, Su Zhan, Wang Benyou et al. End-to-End Quantum-like Language Models with Application to Question Answering AAAI 2018 
  14. Wang Jun, Yu Lantao, Zhang Weinan, Gong Yu, Xu Yinghui, Wang Benyou , Zhang Peng, Zhang Dell. IRGAN: A Minimax Game for Unifying Generative and Discriminative Information Retrieval Models. SIGIR 2017. Best Paper Award Honourable Mentions . Zhihu link (in Chinses)
  15. Wang Benyou, Niu Jiabing, Ma Liqun, Zhang Yuhua, Zhang Lipeng, Li Jinfei, Zhang Peng Song, D. . A Chinese Question Answering Approach Integrating Count-Based and Embedding-Based Features. ICCPOL-NLPCC . December, 2016
  16. Wang Benyou, Zhang Peng, Li Jinfei, Song Dawei, Hou Yuexian, Shang Zhenguo. Exploration of quantum interference in document relevance judgement discrepancy. Entropy , 18(4), 144. 2016. (IF : 1.821)
  17. Chen Yongqiang, Zhang Peng, Song Dawei, Wang Benyou. A Real-Time Eye Tracking Based Query Expansion Approach via Latent Topic Modeling. CIKM 2015 (pp. 1719-1722). ACM. October, 2015

Book and book chapter

  1. Huang Xin, Wei zhao, Wang Benyou, Rui Zhao. Recommendation System and Deep Learning, Tsinghua University Press, in Chinese. Focusing on the chapters related "Learn to rank" and "Generative Adversarial Nets(GAN) for Recommendation". Online purchase link: JD and Dangdang. 
  2. Wang,B. , Emanuele Di, B., & Melucci, M.. Representing words in vector space and beyond . In A. Diederik, K. Andrei, M. Massimo, & T. Bourama (Eds.),Quantum-like models forinformation retrieval and decision-making. Springer.

Talks

  1. "Discussion on the border of LLMs", the Practice from HuatuoGPT, 大模型机理分析论坛, July 20th-30th 2024, Taian, Shandong, China Conference on Data Mining (CCDM 2024), invited by Prof. Chenliang Li
  2. "Large Language Models for Healthcare", the Practice from HuatuoGPT, YSSNLP, Kunming, Yunnan, June 14th 2024, invited by Prof. Chenliang Li
  3. The practice and thinking of medical LLMs, 大模型赋能智能医疗的 workshop, May 6th 2024, Chongqing, invited by Prof. Hao Chen
  4. "The progress of HuatuoGPT", Guest lecture, Soutech, May 2024. invited by Prof. Bingyi Jin
  5. "The present and the future of large language models", The campus open day. Apirl 2024
  6. "The progress of the medical LLM HuatuoGPT", April 2024, Zhejiang University, invited by Prof. Guangyong Chen
  7. "The progress of the medical LLM HuatuoGPT", April 2024, Shanghai AI Lab, invited by Dr. Huaxi Huang
  8. "The progress of the medical LLM HuatuoGPT", April 2024, Peking University, Prof. Xiaohua Zhou
  9. "The practice and thinking of large langauge models", March 2024, Nankai Univesity, invited by Prof. Jie Liu
  10. "The progress of HuatuoGPT", HKUST (Guangzhou), April 2024, invited by Prof. Li Liu
  11. "Large language models, the practice and the future", Huawei 诺亚大讲堂, hosted by Dr. Lifeng Shang, 2024
  12. "What can quantum physics bring to NLP", invited talk by Prof. Masahito Hayasi in the Japan-China Joint Workshop on Quantum Information, September 2023.
  13. "AceGPT, the SOTA Arabic LLM", KAU talk invited by Prof. Eman, September 2023.
  14. "Medical LLM, the practice from HuatuoGPT", online talk invited by Prof. Libo Qin in MLNLP, September 2023.
  15. "Medical LLM, the practice from HuatuoGPT", CIPS tutorial in Jinan, September 2023.
  16. "HuatuoGPT: taming language models to be doctors", IJCAI workshop hosted by Prof. Chen Chen, August 2023.
  17. "Medical LLMs", CCF (中国计算机协会术语委员会), hosted by Prof. Peng Zhang, August 2023.
  18. "What should we do in the LLM era", SRIBD to undergraduate students, hosted by the SRIBD Director Ping Lee, August 2023.
  19. "LLM practice", KAUST, hosted by Prof. Jinchao Xu, August 2023.
  20. "Vertical LLM", CIPS (中文信息学会讲习班), hosted by Prof. Min Yang, June 2023.
  21. "Medical ChatGPT, our practice", HK Hospital Authority (香港医管局), May 2023.
  22. "Medical ChatGPT, our practice", CCF中国计算机协会天津分会, hosted by Prof. Peng Zhang, May 2023.
  23. "Large language model, our practice", KAUST, hosted by Prof. Jinchao Xu, May 2023.
  24. "The interplay between Large language model and IR", Huawei 大讲堂, hosted by Prof. Rui Zhang, May 2023.
  25. "Introduction to ChatGPT", undergraduate students in SDS, Hosted by Prof. Hongyuan Zha, April 2023.
  26. "Introduction to ChatGPT", CIPS中文信息学会青工委, Hosted by Prof. Chenliang Ma, March 2023.
  27. "Introduction to ChatGPT", Shanghai Jiaotong University, Hosted by Prof. Weinan Zhang, March 2023.
  28. "Vertical LLM", Shenzhen Robot and AI lab, hosted by Prof. Hongyuan Zha, November 2022.
  29. "Some proposals on biomedical NLP", online seminar, hosted by Prof. Xiang Wan, SRIBD, September 2022.
  30. "Position embeddings", AI Times, hosted by Miss He Yun, July 2022.
  31. "How and whether AI replaces Humans", Weibo live streaming with more than 1M watching, hosted by Mr Qingyi Gao, June 2022.
  32. "Position embeddings", Renmin University of China, hosted by Prof. Yong Liu, April 2022.
  33. "Quantum and AI: Opportunities, Challenges, and Future Trends", Huawei, Shenzhen, hosted by Prof. Qun Liu, March 2022.
  34. "How quantum physics contributes to NLP", HIT(SZ), Shenzhen, hosted by Prof. Zenglin Xu, December 2021.
  35. "Word2Fun, modeling words as functions for diachronic word embeddings", CSIRO, Australia, online, December 2021.
  36. "Large-scale Pre-trained Language Models (PLMs): Potentials, Efficiency, and Future Trends", SUSTech, Shenzhen, hosted by Qi-Man Shao, December 2021.
  37. "Bridging Quantum physics and NLP", Microsoft Research Asia (MSRA), Beijing, online, hosted by Nan Duan, November 2021.
  38. "How physics and NLP help each other?", Institute of Theoretical Physics, Chinese Academy of Science (CAS) Beijing, December 2020.
  39. "On Position embeddings", Alibaba, Beijing, China, December 2020.
  40. "How quantum theory contributes to NLP", First workshop of quantum computing and AI, virtually, previously in Tianjin University, Tianjin, China, December 2020.
  41. "Encoding Word Order in Complex Embeddings", Speech and Language Computing Group, Huawei Noah's Ark Lab, Shenzhen, China, December 2020.
  42. "Investigating complex-valued representation in NLP", Montreal Institute for Learning Algorithms (MILA), Montreal, Canada, hosted by MILA NLP reading group (hosted by Alessandro Sordoni and Siva Reddy), January 21st, 2020.
  43. "Beyond particles: modeling words as waves", RALI Département d’informatique et recherche opérationnelle, University of Montreal, Montreal, Canada, hosted by Jian-Yun Nie, December 7th, 2019.
  44. Sequencial Modeling in Vector Spaces, the Italian Information Retreival Workshop, in Sep 2021
  45. On Position embeddings , the China Student Symposium on NLP (CSSNLP), Beijing, in December, 2020
  46. Invited lecture: Quantum theory and NLP , for bachelor students, Beijing Institute of Technolohy, Beijing, in December, 2020
  47. Invited lecture: pretrained language model and its position embeddings for bachelor students, Shandong University, Qingdao, in Dec. 2020
  48. How physics and NLP help each other? , Institute of theoretical Physics, Chinese Academy of Science (CAS) Beijing, in December, 2020
  49. On Position embeddings , Alibaba, Beijing in December, 2020
  50. , Formulizing semantic shift detection as a distance between sets EVALITA 2020. Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Diachronic Lexical Semantics online, on December 17th 2020
  51. How quantum theory contributes to NLP, First workshop of quantum computing and AI, virtually, previously in Tianjin University, Tianjin, China, 22. Nov. 2020
  52. Encoding word order in complex embeddings, Speech and Language Computing Group, Huawei Noah's Ark Lab, Shenzhen, China, 23. April 2020
  53. Dynamic Content Monitoring and Exploration using Vector Spaces, University of Bedfordshire, Luton, Lonton, UK, 12 Feb. 2020
  54. Quantum Mechanics meet Information Search and Retrieval – The QUARTZ Project, Great London Text Analytics meetup, London, UK, 12. Feb. 2020
  55. Investigating complex-valued representation in NLP, Mila, Mila, Montreal, Canada, 20. Jan 2020
  56. Beyond particles: modeling words as waves , RALI Département d’informatique et recherche opérationnelle, University of Montreal, Montreal, Canada, Dec. 7th 2019
  57. Beyond particles: modeling words as waves , DIKU machine learning section, University of Copenhagen, Copenhagen, Denmark, Nov. 25th 2019
  58. Quantum formulations for language: understand words as particles , meetup Search Engines Amsterdam , University of Amsterdam, Amsterdam. Netherlands, Oct. 25th 2019
  59. Dynamic Content Monitoring and Exploration using Vector Spaces , SIGIR Doctoral Consortium, Paris, France. July, 2019
  60. 2019 Joint Statistics Summer School by Univeristy of Bolzano, Padova and Salzburg, Brixen, Italy. July 11, 2019
  61. Quantum-inspired NLP/IR . Bytedance AI Lab, Hang Li's group, Beijing, China. June 28, 2019
  62. Quantum-inspired NLP/IR. Tencent Cloud NLP team (Zhiwen Lab), Shenzhen, China, June 21, 2019
  63. Representing and interpreting words in vector space inspired by Quantum theory. Quartz Workshop, University of Copenhagen
  64. Tensor analysis for DL. Functional Analysis, University of Padova
  65. Word embedding and the beyond.> IR group, University of Padova
  66. Deep Learning in language : offline workshop in Padova 

Teaching

  1. CSC 6201/CIE 6021 Large Language Models, Fall 2023/2024. (This might be the first course of large language models in the world.)
  2. CSC6052/DDA6307/MDS6002 Natural Language Processing, Spring 2024/2025

Open-Source code

  1. Phoenix Star
  2. Medical_NLP Star
  3. HuatuoGPT Star
  4. TextClassificationBenchmark Star
  5. InstructionZoo Star
  6. HuatuoGPT-II Star
  7. ALLaVA Star
  8. Huatuo-26M Star
  9. crosstalk-generation Star
  10. ReasoningNLP Star
  11. Apollo Star
  12. Evaluation-of-ChatGPT-on-Information-Extraction Star
  13. CMB Star
  14. OVM Star
  15. AceGPT Star
  16. MLLM-Bench Star
  17. PlatoLM Star