OpenAI’s latest update of its large language model (LLM), ChatGPT 4.0, has shown promising results in a proof-of-concept study conducted by researchers from the University Hospital Heidelberg and the German Cancer Research Center. The study involved testing the LLMs’ performance in a neurology exam, using questions from the American Board of Psychiatry and Neurology.

ChatGPT 4.0 outperformed its predecessor, ChatGPT 3.5, as well as human participants, with an impressive 85% accuracy in answering the questions. This surpasses the average human score of 73.8%. The researchers believe that with further refinements and fine-tuning, LLMs like ChatGPT 4.0 could have significant applications in clinical neurology.

However, it is important to note that both versions of ChatGPT struggled with tasks requiring higher-order thinking, indicating that there is still room for improvement. The researchers caution that while LLMs can be useful in documentation and decision-making support systems, neurologists should exercise caution when using them in practice.

Dr. Varun Venkataramani, one of the study’s authors, emphasized that the study serves as a proof of concept for the capabilities of LLMs and highlights the need for further development and fine-tuning to make them more applicable in clinical neurology.

Overall, this study demonstrates the potential of LLMs like ChatGPT 4.0 in the field of clinical neurology, but also underscores the importance of ongoing refinement and cautious implementation.