Mock Patient Told to Kill Themselves by Open AI’s GPT-3 Medical Chatbot
GPT-3 also known as Generative Pre-Trained Transformer 3 is an advanced autoregressive language model that uses deep learning to produce human-like text.
A recent experience by the medical Paris-based firm, Nabla, specialising in healthcare technology used a cloud-hosted instance of GPT-3 to analyse queries by humans and produce a suitable answer to them. The bot was designed to ease the daily workload of doctors however the unpredictable nature of the software’s responses made it inappropriate for interacting with patients in the real world; this was concluded after running a series of tests.
The tasks set out for the bot by researchers were ranked from a low to high sensitivity from a medical perspective to test its capability in:
Admin chat with patient
Medical insurance check
Medical health support
Medical questions and answers
Problems arose with the algorithm within the very first task as Nabla found out the model had no understanding of time or proper memory so an initial request by the patient for an appointment before 6pm was ignored:
This was not the only problem though, the bot failed to carry out simple arithmetic as it was unable to add up cost of items when handling people’s medical insurance questions and was inept in dispensing accurate medical advice. Furthermore, in an even more dangerous situation, recommended that a mock patient kill itself.
Further tests reveal that the medical chatbot had strange ideas as to how to relax (e.g.recycling) and struggled to prescribe the relevant medication and treatments a patient needed.
Healthcare is an area that requires careful and professional expertise where medic undergo years of training before they’re capable of diagnosing and caring for patients. By attempting to replace human touch and skills with a machine is a difficult demand and is something that cutting-edge technology like GPT-3 is not yet ready for.
Article Thumbnail: https://futurism.com/godfather-ai-trashed-gpt3/amp