FST JOURNAL

There is a huge number of advances happening in AI currently, particularly with large language models (LLMs). ChatGPT is only a year old but its impact has been quite remarkable. LLMs are also known by other names: generative AI; foundation models; and the latest is ‘large multimodal models’. Multimodal in this context means combining visual and language systems to generate descriptions of images and so on. All the big companies are working in this area. The challenge for this sector is to build these models into useful healthcare systems and to embed them in real practice.

These large language models can be used to provide all kinds of tailored health information. They can summarise complex documents, automate administrative tasks, analyse images, etc. In fact, they can be used in so many different ways that the challenge is to determine what to do first.

Over the past 20 years, researchers have spent a great deal of time building complex modular systems to understand and generate human language. In generative AI, humans can interact with these systems in a conversational way and, in addition, give them instructions to, for example, write an email to a doctor.

A European project called RESQ+ is focussed on providing conversational AI for people who have had a stroke. The system asks them questions about their condition and the answers can be used as a way of assessing their condition and recovery. The project also helps them understand their condition. An app for mobile phones has been developed that patients can either speak to or type. A typical question might be: “What is aphasia?”  If someone has had a stroke and now has aphasia, they may not remember what the consultant said so these answers could be very useful.

When the AI generates an answer, this will be based on a specific collection of documents, which it uses as a trusted source of truth. That is one way of dealing with problems of ‘hallucination’ – or wrong information – that can occur with large language models, although the problem has not been totally solved yet.

The AI can also delve into quite complex questions such as: “What are the differences between Broca’s and Wernicke's aphasia?”  Here, the AI system has to carry out multi-step reasoning in finding the relevant documents, comparing them and then generating the right answer.

A recent version of GPT4 can automatically generate a radiology report from an x-ray image. The prompt or instruction to the system is to write a radiology report based on the image. Medical professionals can then check these generated reports. Sometimes there are errors which they can correct, but sometimes these reports are of high quality. Such reports would take a medical professional some time to produce but they can still be checked by humans.

How, then, do we fold these methods into existing workflows, so that people can use them to amplify and support their professional practice, making it easier for them to do their job more effectively and making it more enjoyable? AI needs to be fun to use if we want people to adopt it. The interactive, conversational aspect is one element of making people want to use this both in their jobs and their everyday lives.

Promise and peril

Much is said about both the promise and the peril of generative AI systems. It is well known that they can hallucinate facts: they can generate text which reads well and is convincing but, in reality, contains errors. There are also known issues of bias where the models are trained on data which is discriminatory in different ways. Many people are now working on ways to improve the training data sets for such models.

Shutterstock

A very important issue is the concerns that the general public have about privacy and security. I work on projects where we put AI into hospitals. The key requirement is that no data is allowed to leave the premises. That means building rather small generative AI systems, which can be run on hospital premises without data being exported. But large amounts of data are still needed for model training. A significant effort has to be made to reassure the public that all the data we use is fully anonymised.

There is, quite rightly, a great deal of discussion politically about job displacement and how AI will alter the workplace. The challenge is to determine how it can augment people's existing jobs, helping them to become faster and more productive – and more fulfilled. AI needs to become a ‘co-pilot’ or ‘buddy’, helping to get the tasks done.

The Robotarium

The UK National Robotarium, based at Heriot Watt University, is focussed on addressing the core AI problems facing robotics. Good robotics applications require computer vision, good planning, effective interactions with humans, an understanding of language and so on. These core AI issues are what make robots possible. This is, if you like, embodied AI.

Large language model systems are now being incorporated into robots. In one application, we have been able to generate facial expressions and robot gestures along with speech. This is used as a receptionist for the building. A similar system has been deployed in Paris, in a memory clinic at the Broca Hospital. This is an EU project called Spring. It captures patients’ everyday questions, such as where the coffee machine is, where the lift is and which bus to get back home.

We have built this system to speak in French. Some large language models are multilingual and it was very easy for us to translate the system from English to French. In the hospital waiting room, it helps people to find out where to go next, where to get lunch, etc – it automates responses to these trivial but important questions.

There is a huge opportunity here, not only in humanoid robotics and human-robot interaction, but for the more mundane and boring administrative tasks – such as automatically generating patient notes and records to improve efficiency.

In trying to keep the risks balanced, a key element is to always keep a human in the loop. There can be an AI co-pilot, or an AI can be part of a team of humans but, ultimately, humans must make the final decision.

Co-design will improve these systems. Academics are talking with healthcare professionals to create systems that will be useful to them, make their jobs better and more interesting, and ultimately  improve patient outcomes.