New Study Reveals the Limits of AI Models like ChatGPT

AI models become more powerful with increasing size and the amount of data with which they are trained (scaling). This rapid development of AI models such as ChatGPT and Co. not only influences the world of work, but also raises questions about the possible development of a general “intelligent” behavior of Large Language Models (LLMs).

A new study, led by TU Darmstadt computer science professor and hessian.AI-member Iryna Gurevych, along with Dr. Harish Tayyar Madabushi from the University of Bath, reveals the limitations of ChatGPT and similar AI models. In the study, the researchers demonstrate that despite scaling, there is no evidence for the presumed development of differentiated reasoning capabilities of AI models such as ChatGPT. Instead, the LLMs “only” follow relatively simple instructions. According to the authors of the study, the systems are still a long way from having the same abilities as humans.

 The study was presented in August at the annual conference of the renowned Association for Computational Linguistics (ACL) in Bangkok, the largest international conference on automatic language processing.

Find out more about the study.

Image created by AI using DALL-E.