Evaluating AI-driven analytics for radiology, part 1: Understanding the lingo and the landscape

Having the right data at the right time has always been the key to better clinical and business outcomes. And today, AI-powered analytics solutions are helping providers across the care continuum collect, analyze, and act on data with unprecedented focus.

In radiology, the pressure is on to deliver even more in even less time and prove downstream value to care teams. Fortunately, radiology organizations can now effectively harness radiology-specific analytics to support their clinical decision-making and operations—whether the technology is streamlining MIPS reporting, profiling performance, or addressing critical issues like failed follow-ups.

But for the uninitiated, knowing what data to look for, how best to collect it, and then how to interpret it can be daunting and time-consuming. Finding automated solutions that will deliver meaningful results is a logical first step. Evaluating the options best suited to your organization means understanding the technical lingo and landscape, as well as knowing what great performance looks like and the kind of business and patient outcomes you should expect.

First, let’s filter the alphabet soup.

Natural Language Processing

Natural language processing (NLP) is a branch of AI that focuses on the comprehension of human language. Some organizations refer to this as “computational linguistics.” Nuance uses the term “natural language understanding” (NLU).

Traditional, operational analytics—for example, calculating average turnaround times—don’t need NLP because the required data is already structured in set fields. However, NLP is essential if you want to identify and extract non-discrete data—for example, a follow-up recommendation—from the free-flowing narrative of a radiologist’s report.

Until recently, the NLP used to analyze such narratives has been primarily rules-based. It returns results based on keywords and their proximity to each other in the report, such as “no” and “lung nodule,” or “CT” and “three months.”

This rule-defined approach delivers rapid results but has real limitations. If the words the analytics solution is looking for aren’t the words the radiologist used, it’s easy for the NLP to return a false negative. Equally, something as commonplace as a radiologist pasting a set of follow-up guidelines into the report could fool the NLP into identifying multiple follow-up recommendations—which are false positives. The noise created by both scenarios makes it vastly more difficult to extract meaningful information.

Today’s most robust radiology analytics solutions are much more sophisticated, extending beyond rules-based algorithms. For example, a solution like Nuance mPower Clinical Analytics builds on decades of radiology-specific language understanding expertise, harnessing advanced AI capabilities like machine learning (see below) for more flexible algorithms that deliver more precise results.

Machine learning, neural nets, and deep learning

Machine learning is a different branch of AI. It focuses on using data and algorithms to imitate the way humans learn and produce gradual improvement over time. Neural nets and deep learning are exciting subsets of machine learning (but beyond the scope of this article).

The important thing to understand is that all forms of machine learning involve training AI on datasets (like many, many radiology reports) to make it better at a task. And before investing in a radiology analytics solution, you’ll want to make sure it’s been trained on as large and varied a dataset as possible. Nuance mPower, for example, has learned from over one billion reports, using a diverse dataset derived from hospitals, radiology groups, and academic institutions across the United States.

As you might expect, solutions trained on smaller, less diverse sets of radiology reports will be comparatively biased toward how those reports are structured and the language they use. That means they’re unlikely to deliver the same levels of accuracy when you feed your own data into their algorithms.

Continuous learning

Continuous learning is the ability of AI tools to keep improving. Like most things in life, if you leave it alone, an AI model will begin to decay. Models need to be continually evaluated for accuracy, precision, recall, and bias (more on this in my next blog). At Nuance, we can evaluate and optimize on a mass scale, continually annotating and tuning our analytics algorithms.

Healthcare lexicons

Your AI solution needs to know the language of healthcare inside out, even to do something as simple as auto-suggesting the right search term. To correctly extract meaning from a radiologist’s report, it will also need to understand the concepts those words relate to and represent, how they are customarily used in context, and the fundamentals of human anatomy.

What’s more, there are multiple lexicons within healthcare. You may want your AI analytics to be fluent in RadLex, SNOMED, CDE, or LOINC. Either way, you’ll certainly want your solution to speak the language your radiologists use—for example, when they talk about a hangman’s fracture as a skull fracture, even though it’s strictly a fracture of the spine.

Mastery of the many languages of healthcare is something that, once again, AI can only achieve through access to vast amounts of healthcare data and constant adjustment and optimization over long periods of time. So, it’s important to look at your vendor’s expertise, experience, and long-term investment in this complex field.

Next up: What great outcomes really look like

In the second part of this blog series, I’ll explain the performance metrics that matter when evaluating AI for radiology, and the questions you’ll want to ask solution vendors before you place confidence in their claims.

Learn more about mPower Clinical Analytics

Explore how our AI-driven analytics solution helps teams monitor, understand, and improve financial and clinical performance.

Learn more
Dr. Sheela Agarwal

About Dr. Sheela Agarwal

Dr Sheela Agarwal joins Nuance from Bayer Healthcare Radiology, where she held the position of Digital Medical Advisor, acting as medical lead for the Digital Solutions Business. Prior to that, Dr. Agarwal held other leadership roles at Bayer including Head of the Digital Solutions Business and Head of Medical Affairs for region Americas. She is an experienced leader with a history of working in academics and industry, has written multiple AI-related publications through her work with the American College of Radiology DSI and was recently involved in establishing “AI Central,” the FDA-Cleared Algorithm Catalog. Dr. Agarwal completed her undergraduate and graduate degrees in Economics, as well as her MD from Duke University, her graduate radiology training subspecializing in abdominal radiology and MRI from Massachusetts General Hospital/Harvard Medical School and her MBA from the University of Chicago Booth School of Business. She continues to practice as an abdominal radiologist at Lenox Hill Radiology in New York City