AI-Assisted Collaborative Reading and Evaluation
As scientific publishing is accelerating, it becomes increasingly hard to ensure the quality of publications. Peer review is the traditional quality control mechanism of the scientific community. However, science is slowed down by the time spent reviewing and waiting for reviews, and the reproducibility of peer reviewing has been questioned (Bornmann, 2011). The UKP Lab develops PEER, an Artificial Intelligence assisted software that makes reading and evaluation of scientific publications easier. It introduces annotation-based reviewing and provides several helpful, AI-powered functionalities, such as the generation of a structured report from user annotations.
To make PEER as useful as possible, we aim to understand how users with various scientific backgrounds interact with digital publications by means of a survey among the CAIS community. Next, we conduct an interactive user study to collect feedback on the software. Finally, we use the insights and feedback to improve our software design and algorithms.
Lutz Bornmann. 2011. Scientific Peer Review. Annual Review of Information Science and Technology, 45(1):197—245
Main Research Topics
- Natural Language Processing
- Artificial Intelligence
- Text analysis
Curriculum Vitae
1998–2001: PhD in Computational Linguistics at the University of Duisburg-Essen, Germany, thesis title: “Analysis of Natural Language Queries in Restricted Discourse Areas”
2001–2005: PostDoc, EML Research, Heidelberg, Germany
2005–2009: Founder and head of the Ubiquitous Knowledge Processing Lab, TU Darmstadt, Germany
Since 2009: Full professor and head of the Ubiqitous Knowledge Processing Lab, TU Darmstadt, Germany
Memberships and awards: ERC Advanced Grant (2021), LOEWE-Spitzenprofessur (2021), vice president of the Association for Computational Linguistics, Co-director of the European Lab for Learning and Intelligent Systems (ELLIS)
Lectures and Publications
Yang Gao, Steffen Eger, Ilia Kuznetsov, Iryna Gurevych and Yusuke Miyao. “Does My Rebuttal Matter? Insights from a Major NLP Conference”. In: The 2019 Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, USA, June 2019, pp. 1274-1290
“Towards consent-driven, ethically sound NLP for peer reviews”, Indian Symposium on Machine Learning (IndoML), December 2021 (online)
Kuznetsov, I., Buchmann, J., Eichler, M., & Gurevych, I.. (2022). “Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review”. arXiv preprint arXiv: 2204.10805
Dycke, N., Kuznetsov, I., & Gurevych, I. (2022). “Yes-Yes-Yes: Donation-based Peer Reviewing Data Collection for ACL Rolling Review and Beyond”. arXiv preprint arXiv:2201.11443.