In January, OpenAI presented a new ChatGPT module, which includes, among others: analyze blood test results based on medical records. Users, however, should not be convinced that they will receive a diagnosis from ChatGPT – Ligia Kornowska, leader of AI in Health, told PAP.
Open AI presented a new function for searching for health information. It will be known to users as ChatGPT Health. The module is to be available in the chatbot's sidebar menu and will provide information based on the medical documentation sent by the user. The function offers, among others: analysis of blood test results and nutritional suggestions.
ChatGPT instead of a doctor? Answering the needs of millions
According to Open AI, it is a response to social demand. The January Open AI report “AI as a Healthcare Ally” shows that over 40 million users turn to ChatGPT every day with questions about healthcare, and over 5 percent all messages with AI assistant are related to the topic of healthcare.
Ligia Kornowska, leader of the “AI in Health” coalition, who deals with the use of AI in health matters, draws attention to the dangers associated with it.
– If an artificial intelligence algorithm is to directly influence diagnostic and treatment decisions for a patient, it should be compliant with legal regulations regarding medical devices or the so-called high-risk algorithms – said the expert. And this is regardless of whether the system is used by a patient or a doctor.
In situations where the algorithm is not a certified medical device, as is the case with ChatGPT Health, we do not really know whether this algorithm will provide the patient with good information or will mislead him, Kornowska noted.
Tools for hospitals and GDPR
According to her, such certification allows us to provide information on how effective and sensitive the algorithm is, as well as with what probability it will tell users the truth and with what probability it may mislead them.
In Kornowska's opinion, the scope of use of ChatGPT-Zdrowie should be limited to educational purposes for now. It should contain information that each message sent by the chat must be verified independently. Also, patients – as the leader of “AI in Health” said – should not be led into the belief that they will receive a diagnosis from this assistant.I. – In such a context, ChatGPT will certainly not replace a medical consultation – she emphasized.
OpenAI also presented a toolkit for healthcare organizations called “OpenAI for Healthcare”. It is to offer, among others:
- assistance in administrative work, i.e. creating summaries of extracts,
- recommendations for patients,
- clinical letters.
Such tools are already being used by American institutions and facilities, such as AdventHealth and Boston Children's Hospital.
Kornowska, however, pointed out that the protection of personal data, especially sensitive data, is regulated differently in the European Union and in the United States. – When using a tool from the US, you need to make sure that it complies with EU regulations, such as the General Data Protection Regulation (GDPR), the expert noted.
The tragic consequences of chatbot advice?
The British newspaper “The Guardian” also recently drew attention to problems related to the use of medical advice from AI assistants. The Journal's January investigation found that summaries generated by Google's AI search engine provided inaccurate health information. They were to concern, among others: key liver function tests. According to the Guardian, based on this information, people with serious diseases of this organ could mistakenly believe that they are healthy.
When asked by a British daily, a Google spokesman replied that the company does not comment on individual deletions in the search engine. “Where AI Overviews features are not context-aware, we are working to make broad-based improvements and, where necessary, take action in line with our policies,” he explained.
The impact of chatbots on users' mental health has also been controversial. Last August, the world was informed about OpenAI being sued in connection with the suicide death of 14-year-old Adam from California. His parents alleged that ChatGPT encouraged him to take his own life.
Last November The Associated Press reported that OpenAI received a total of seven similar lawsuits. They claimed that ChatGPT led to suicide and harmful delusions in people who had no prior mental health problems.
In response, OpenAI declared that safeguards had been implemented in cases where conversations with ChatGPT concerned sensitive topics. “We continually refine ChatGPT training to recognize and respond to signs of mental or emotional stress, de-escalate tensions in conversations, and direct people to real support,” the company wrote.
Weronika Moszpańska (PAP)
wm/ pś/ mhr/
