Sign in Subscribe

Generative Artificial Intelligence and Large Language Models in Medical Imaging for Medical Professionals - Great Korean Publication about the keypoints

Daniel Lorenz

• 2 min read
Generative Artificial Intelligence and Large Language Models in Medical Imaging for Medical Professionals - Great Korean Publication about the keypoints

Okay, so we're utilizing AI for a variety of tasks in radiology - from analyzing images and accelerating image acquisition to even aiding us in crafting an impression that's bound to wow our referring orthopedic surgeons. The sheer breadth of tasks indicates the diverse nature of the AIs we're dealing with. And let me tell you, unless you're more tech-savvy than I am (which isn't too hard to be), you might also feel a bit overwhelmed by the myriad of models and technical complexity. That's why I was thrilled to come across this paper from Kim et al at the University of Ulsan. It really helped me to get a clearer picture of the AI landscape.

In essence, there are three main groups:

  1. Language Models
  2. Vision Models
  3. Vision-Language Models – a newer breed that combines the strengths of both worlds.

Each category has its subgroups, and this is where it gets super interesting (at least for me):

Variational Autoencoders (VAEs)

  • What They Do: VAEs transform images into a coded format and then back again. They're great at generating new images and tweaking features in existing ones.
  • Radiology Relevance: Handy for producing detailed MRI or CT images and refining image features for better clarity.

Generative Adversarial Networks (GANs)

  • What They Do: GANs consist of two parts: a creator of images and a critic that evaluates them, perfect for crafting ultra-realistic images.
  • Radiology Relevance: Ideal for enhancing radiological image quality, adapting images across different scanning techniques, and spotting anomalies.

Diffusion Models

  • What They Do: These models gradually add and then remove noise, refining the images in stages.
  • Radiology Relevance: Great for noise reduction in CT images, transforming images into different styles, and speeding up MRI processes.

Large Language Models (LLMs)

  • What They Do: LLMs like ChatGPT are adept at understanding and generating text, serving as sophisticated text analyzers and creators.
  • Radiology Relevance: They can automate the analysis and interpretation of radiology reports, summarize medical data, and simplify complex findings for better comprehension.

Vision-Language Models (VLMs)

  • What They Do: VLMs merge the worlds of image processing and language understanding, adept at handling both visual and textual data.
  • Radiology Relevance: These models excel in creating comprehensive reports that combine visual analysis with textual interpretation, enhancing diagnosis and research by leveraging both image and text information.

That's our current AI toolkit. But the paper goes on to discuss the emergence of foundational models in radiology, such as large-scale AI models pre-trained on massive datasets (think Google’s Med-PaLM). Some efforts in building foundational models in Radiology include:

  • Training a model on 100 million medical images, including radiographs, CT, MRI, and ultrasound.
  • Developing a self-supervised network trained on 4.8 million chest radiographs.

While there are still challenges, particularly in scalability, this approach seems incredibly promising for the future of radiology.