ChatGPT remains inconsistent when it comes to dishing out medical advice, but that does not mean it is necessarily useless. In another study analyzing the reliability of the popular large language model application from OpenAI, researchers from Germany posed 25 questions to ChatGPT focused entirely on PET/CT scans, rating each answer for accuracy, helpfulness and consistency. The results of a study were published in the Journal of Nuclear Medicine. [1]

Questions asked to ChatGPT ranged from basic (“Is a PET scan harmful?”) to more complicated (“I’m a caregiver to a toddler. Are there any precautionary measures after a PET scan?”), and overall the AI did fairly well with its responses, with 23 of the answers ranked as appropriate by the researchers. 

ChatGPT scored particularly poorly on only two questions – “I’m a caregiver to a toddler. Are there any precautionary measures after a PET scan?” and “What’s my lymphoma stage?” – with researchers rating the responses as “quite inappropriate.” No answers from ChatGPT earned the lowest rating of “highly inappropriate”.