publications

2024

1. NAACL 2024

   Oral

   

   You don’t need a personality test to know these models are unreliable: Assessing the Reliability of Large Language Models on Psychometric Instruments

   Bangzhao Shu^*, Lechen Zhang^*, Minje Choi, Lavinia Dunagan, Lajanugen Logeswaran, Moontae Lee, Dallas Card, and David Jurgens

   In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), Jun 2024

   DOI arXiv Code Slides Twitter
2. Preprint

   Under Review

   

   SPRIG: Improving Large Language Model Performance by System Prompt Optimization

   Lechen Zhang, Tolga Ergen, Lajanugen Logeswaran, Moontae Lee, and David Jurgens

   arXiv preprint arXiv:2410.14826, Oct 2024

   arXiv Code Slides Twitter
3. Preprint

   Under Review

   

   FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation

   Farima Fatahi Bayat, Lechen Zhang, Sheza Munir, and Lu Wang

   arXiv preprint arXiv:2410.22257, Oct 2024

   arXiv Code Twitter
4. Preprint

   Under Review

   

   Real or Robotic? Assessing Whether LLMs Accurately Simulate Qualities of Human Responses in Dialogue

   Jonathan Ivey^*, Shivani Kumar^*, Jiayu Liu^*, Hua Shen^*, Sushrita Rakshit^*, Rohan Raju^*, Haotian Zhang^*, Aparna Ananthasubramaniam^*, Junghwan Kim^*, Bowen Yi^*, Dustin Wright^*, Abraham Israeli^*, Anders Giovanni Møller^*, Lechen Zhang^*, and David Jurgens

   arXiv preprint arXiv:2409.08330, Sep 2024

   arXiv Code Twitter
5. Under Review

   

   Causally Modeling the Linguistic and Social Factors that Predict Email Response

   Yinuo Xu^*, Hong Chen^*, Sushrita Rakshit^*, Aparna Ananthasubramaniam^*, Omkar Yadav^*, Mingqian Zheng^*, Michael Jiang^*, Lechen Zhang^*, Bowen Yi^*, Kenan Alkiek^*, Abraham Israeli^*, Bangzhao Shu^*, Hua Shen^*, Jiaxin Pei^*, Haotian Zhang^*, Miriam Schirmer^*, and David Jurgens

   Sep 2024