Academic Writing: A Field Experiment Testing GPT's Ability to Foster Clarity and Efficiency

Addressing global challenges such as climate change or social inequality requires research as well as effective communication. However, many in academia are confronted with a lack of time. Can AI provide relief so that researchers can focus on their core work: the research itself? With this experiment, Marc Ratkovic and Thomas Gschwend explored to what extent generative AI, in a human-supervised process, can help researchers save time when writing while maintaining a high level of quality.

Literacy: Dell’Acqua, F., McFowland, E., Mollick, E., et al. (2023). Navigating the jagged technological frontier: Field experimental evidence of the effects of AI on knowledge worker productivity and quality. Harvard Business School. Noy, S., & Zhang, W. (2023). Experimental evidence on the productivity effects of generative artificial intelligence. Science, 381(6654), 187–192. https://doi.org/10.1126/science.adh2586 Cui, Z., Demirer, M., Jaffe, S., Musolff, L., Peng, S. & Salz, T. (2024). The Effects of Generative AI on High Skilled Work: Evidence from Three Field Experiments with Software Developers. SSRN eLibrary, . doi: 10.2139/ssrn.4945566 Kumar, H., Xiao, R., Lawson, B., Musabirov, I., Shi, J., Wang, X., Luo, H., Williams, J. J., Rafferty, A. N., Stamper, J., & Liut, M. (2024). Supporting self-reflection at scale with large language models: Insights from randomized field experiments in classrooms. In Proceedings of the Eleventh ACM Conference on Learning @ Scale (pp. 86–97). Association for Computing Machinery. https://doi.org/10.1145/3657604.3662042 Pankiewicz, M., & Baker, R. S. (2023). Large language models (GPT) for automating feedback on programming assignments. In J. L. Shih et al. (Eds.), Proceedings of the 31st International Conference on Computers in Education. Asia-Pacific Society for Computers in Education. Meyer, J.G., Urbanowicz, R.J., Martin, P.C.N. et al. ChatGPT and large language models in academia: opportunities and challenges. BioData Mining 16, 20 (2023). https://doi.org/10.1186/s13040-023-00339-9 Hou, Jinghui and Wang, Lei and Wang, Gang and Wang, Harry and Yang, Shuai, The Double-Edged Roles of Generative AI in the Creative Process: Experiments on Design Work (November 30, 2024). Available at SSRN: https://ssrn.com/abstract=4739471 Bauer, Kevin and Jussupow, Ekaterina and Heigl, Rebecca and Vogt, Benjamin and Hinz, Oliver, All just in your head? Unraveling the side effects of generative AI disclosure in creative tasks (November 30, 2024). Available at SSRN: https://ssrn.com/abstract=4782554 or http://dx.doi.org/10.2139/ssrn.4782554