Lost in the radiology AI universe - RADAR can be your pathfinder
The number of radiology AI solutions released in recent years seems virtually uncountable. There is an overwhelming array of tools capable of detecting findings in CT and MRI scans, alongside an ever-increasing number of tools designed to enhance workflow efficiency.
The Elephant in the Room: Which AI Tool Can I Trust?
The key question is: How do I determine which AI tool I can trust, and how do I discern its strengths and weaknesses? The general consensus is that we need comprehensive frameworks to guide us through a standardized process. Boverhof, Redekop et al. introduced a framework known as the Radiology AI Deployment and Assessment Rubric (RADAR), which is based on the imaging efficacy framework by Fryback and Thornbury from the '90s.
What are the levels of RADAR?
RADAR consists of a seven-step hierarchy that mirrors the AI solution lifecycle from proof of concept to clinical implementation. Depending on the stage of the solution, you may start at step one for evaluation, or you might jump to an advanced stage if the solution is already well-established.
Here are the 7 steps:
1. Technical efficacy
2. Diagnostic accuracy efficacy
3. Diagnostic thinking efficacy
4. Therapeutic efficacy
5. Patient outcome efficacy
6. Cost-effectiveness efficacy
7. Local efficacy
Each step in the RADAR framework is accompanied by a set of various study methodologies for evaluation.
Bridge the gap, raise the trust, and accelerate clinical implementation
The significant advantage of such a framework is that it offers a standardized process that yields comparable data. This standardization helps you and your team quickly learn to trust the data and the process, allowing you to effectively filter out AI tools that are of supreme quality and that truly suit your use case. Within this loop, you, your team, and collaborators will more readily trust and implement AI solutions that bring value to the workflow.
My opinion: value based radiology needs comprehensive frameworks for evaluating new AI tools
In my view, the big challenges for an AI-assisted (and potentially AI-driven) workflow are:
- The development of AI tools
- The development of an IT-infrastructure and radiology workflow that allows for seamless and human-centered integration of AI tools
- The development of tools, or frameworks, to determine which AI tools are most suitable for our use cases.
The RADAR framework could be an excellent approach for addressing point number three and might become the biggest promoter of using AI in clinical medicine. Doctors need to trust AI tools before they start to rely on them. However, trust can only be built if one understands the rules and benchmarks that apply to a new tool. Just as you know what to look for when buying a new car or bicycle, the same should be true for any new product, especially one as critical as AI in healthcare.
Sources
Boverhof, BJ., Redekop, W.K., Bos, D. et al. Radiology AI Deployment and Assessment Rubric (RADAR) to bring value-based AI into radiological practice. Insights Imaging 15, 34 (2024). https://doi.org/10.1186/s13244-023-01599-z