Benyou Wang
Mail: wabyking@gmail.com
Currently, I am an assistant professor in the Chinese University of Hong Kong, Shenzhen (CUHKSZ). I got my phd degree from the University of Padova, Italy (very fortune to be supervised by Massimo Melucci and Emanuele Di Buiccio). See our lab on CUHKSZ LLM group.
I submit my thesis in Sep. 30th 2021 (Here is a draft version)and have it defended in March 2022.
I am recently interested in NLP. The popular NLP killer, i.e., pre-trained language models (PLMs) might be good in terms of effectiveness. However, it is still limited:
PLMs need more understanding and could therefore be improved in the right directions.
PLMs cannot solve more complicated problems e.g., involving reasoning. This will be challenging in recent 10 or more years.
PLMs are too big to deploy (time and space-consuming). Tensor networks may help for time-efficient or space-efficient PLMs.
It is assumed that "the bigger the better" (see GPT 3). PLMs are too expensive to be enlarged. Can quantum computing help to build GPT 10?
Can PLMs help for other domains like biomedical problems? In the biomedical domain, there are many types of sequential tokens (e.g., DNA, proteins, disease codes) that could be trained with PLMs.
I joined CUHKSZ as an assistant professor from June 1st 2022. Please send me emails if you are interested to work with me as a Ph.D., research associate, or post-doc. See JD here
News
- I serve as a Website Chair in EMNLP 2023.
- I serve as a Publicity Chair in NLPCC 2023.
- Our paper ( Doge Ticket ) got the Best Paper Award in NLPCC 2022, see here .
- In September 2022, we got one paper accepted in NeurIPS (named MorphTE ) and another paper in EMNLP (Hypoformer ). Both papers are about compressing transformer models (either in embedding or fully-connected layers)
- In August 2022, one paper got accepted in COLING, which extends deep prompt tuning (DPT) to dense retrieval. By using two additional strategies, DPT got comparable performance with a fine-tuning.
- A joking paper is released: Can we create a new creature?
- A new paper is accepted in ICLR 2022 which could compress 12-layer BERT encoders into 1.5 M while with slight performance drop, see "Exploring extreme parameter compression for pre-trained language models".
- Our paper titled "Word2Fun: Modelling Words as Functions for Diachronic Word Representation" got accepted in NeurIPS 2021 with my supervisors Massimo and Emanuele. This introduces a new paradigm for time-specific word embeddings (e.g., imagining that word "president" in 2018 and 2021 generally refer to different people), with both theoretical advantages and empirical success. This is a kind of work that are smaller, better, and more interpretable.
- Our paper titled "On position embeddings in BERT" got accepted in ICLR 2021. Try searching "position embeddings" in Google .
- Our paper titled "Encoding word order in complex embeddings" got accepted with a spotlight presentation in ICLR 2020 (acceptance rate 6%). Codes were already open-sourced.. This is the first work for rotation-based position embeddings while the previous is translation-based.
- We won the Best explainable NLP paper in NAACL 2019 with 1000 dollars, present our paper together with BERT authors (Best Long paper winner). here
- I got Marie Curry Fellowship to rejoin academia in 2018. Thanks for the generous funding.
- Our book <推荐系统与深度学习> has been published, buy it in JD
- We (IRGAN) won the Best Paper honorable mention award in SIGIR 2017, one of the most-cited papers in SIGIR. See here
Awards
- NAACL 2019 Best Explainable NLP Paper
- SIGIR 2017 best paper award Honorable Mention
- Selected as a Marie Curie Researcher of Quantum Information Access and Retrieval Theory (QUARTZ), a fellowship for Early-stage Researcher, funded by the European Union's Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No. 721321
Education
- 2018.10-~: Ph.D student: information engineering, University of Padua, Italy.
- 2014.9-2017.2: Master: pattern recognition and intelligent system, Tianjin University, China.
- 2010.9-2014.6: Bachelor: software engineer, Hubei University of Automotive Technology, China.
Work and Intern
- 2018.6-~: Marie Curie Researcher, Department of Information Engineering, University of Padua, Italy.
- 2020.12: Visiting Scholar. Institute of Theoretical Physics, Chinese Academy of Science (CAS) in Dec. 2020 hosted by Pan Zhang
- 2019.11-2020-2: Visiting Student. University of Montreal from Dec. 2019 to Feb. 2020, hosted by Prof. Jian-Yun Nie.
- 2019.10: Visiting Student (one week). University of Amsterdam, hosted by Prof. Maarten de Rijke.
- 2019.9-2019.12: Visiting Student. DIKU IR Lab in University of Copenhagen, hosted by Prof. Christina Lioma.
- 2017.7-2018.6: Full-time Associate Researcher, Data Application Center, Tencent, China.
- 2017.2-2017.5: Research Assistant, Laboratory of Cognitive Computing and Application, Tianjin University.
- 2015.8--2015.9 & 2016.4-2016.5 Research Intern, Institute of Linguistics , China Academy of Social Sciences Beijing, China.
Professional Activities