The researchers asked 195 questions of ChatGPT and verified doctors in the AskDocs forum on Reddit to compare results. ChatGPT performed better in 78.6% of scenarios with a 3.6 times higher quality score and 9.8 times higher empathy score than human doctors.
Why did ChatGPT receive higher scores than real doctors?
The main reason ChatGPT outperformed doctors in empathy is that real doctors tend to give very short answers. Short answers received a lower score in the study. For example, when the chatbot was asked about eye bleach damage, it generated four sentences to explain the situation and give instructions while the doctor just said it would be fine. The longer responses from the doctors were still inferior to those of ChatGPT.
Warning: a chatbot cannot make a diagnosis
Despite the chatbot's impressive results, it's important to understand that it can't make a diagnosis because no physical examination has been performed. However, chatbots like ChatGPT can help simplify the work of medical staff and provide a better experience for patients. The researchers also suggest that AI systems could be used to train healthcare professionals in better communication skills.
I'm a big fan of short stories about people – I'm a pro at tech and smartphones, serial literature, and writing in my spare time.