Improving Headache Diagnosis with Artificial Intelligence

A recent proof-of-concept study shows that natural language processing and machine learning can accurately classify migraine versus cluster headache.

Over the past decade, researchers have used artificial intelligence (AI) along with machine learning (ML), a subset of AI, to help physicians diagnose and treat patients with a variety of medical conditions. For instance, recent research has shown the AI/ML approach can be used to detect cancerous lesions, predict stroke outcomes, and help people living with chronic diseases like diabetes better manage their conditions. A 2019 systematic review and meta-analysis of medical imaging studies even found that the diagnostic accuracy of deep learning, a form of AI, equaled that of healthcare professionals.

Now, researchers from Ghent University Hospital in Belgium apply natural language processing (NLP), another subset of AI, to identify patterns in the language that people with migraine and cluster headache use in written descriptions of their headache attacks. Further, the group reports that ML algorithms successfully classified each individual’s condition as either migraine or cluster headache, with an accuracy rate of about 86 percent.

“It’s a very interesting study and uses quite complex methodology. It takes a patient’s own assessment of their condition, their own descriptions of their headaches, and uses those words to determine whether the headache was migraine or a cluster headache,” said Shivang Joshi, Community Neuroscience Services and University of Massachusetts, US. Joshi, who said the work is an important first step in looking at the clinical utility of AI/ML in headache diagnosis, is a neurologist and headache specialist who has used data mining techniques to validate cluster headache diagnoses but was not involved with the current study.

“While this particular algorithm is a bit limited, as technology advances we can take a model like this, improve upon it, and use it to help physicians better diagnose different headache disorders in the future,” Joshi added.

The study appeared September 30, 2022, in The Journal of Headache and Pain.

A new model for headache classification
To diagnose a headache disorder, physicians must carefully listen to patients as they describe their symptoms and experience. But coming to a diagnosis isn’t always easy; busy doctors may miss specific words or language patterns that could help them distinguish one type of headache from another.

That’s why NLP is a good choice to help with diagnosis, said Nicolas Vandenbussche, a neurologist at Ghent University Hospital and first author of the new paper.

“We are well aware, as clinicians, that diagnosing the right headache disorder is sometimes difficult because we rely solely on linguistic information to determine that diagnosis,” he said. “There are no biomarkers, no imaging evidence that can support us and point us in a certain direction. But the computational power of NLP could, perhaps, help us with this diagnostic process by analyzing narratives provided by our patients.”

Vandenbussche and colleagues, including Koen Paemeleire, also a neurologist at Ghent University Hospital and senior study author, recruited 187 Dutch-speaking patients to take part in the new research. All the participants had already received a diagnosis of migraine or cluster headache by a hospital neurologist using the third edition of the International Classification of Headache Disorders (ICHD-3).

Study participants completed a web-based survey that asked standard questions about their headaches, as well as an open field question that asked them to write a detailed description of their headache experience, with no limitation on word or sentence count, or on topic or theme. A total of 121 patients sent back answers to the open field question. However, not all of them included a description of the headache attack itself, so the researchers created a second “corpus,” or data set, that included the 74 patients diagnosed with migraine and the 38 patients diagnosed with cluster headache who talked about that particular aspect of their experience.

Analyzing the text
A lexical analysis of the full texts at the word level uncovered several specific key words that helped to differentiate whether a person had been diagnosed with migraine or cluster headache. For example, people with cluster headache were more likely to use words like “eye,” “pain,” and “to come back” than those with migraine were. Migraine, in contrast, was more likely to be described with words like “headache,” “stress,” and “nausea.”

“For the most part, the analysis of word usage aligned with what we knew about the disorders from the literature as well as from our own experience,” said Vandenbussche. “But there were some interesting aspects. For example, the patients with migraine used the word ‘headache’ more than the word ‘pain,’ and the opposite was true for those with cluster headache. This difference was very statistically significant, and we think it may be a sign that certain words may be used more often with certain groups of patients with different headache disorders. But the number of times they are used may not be noticed in the course of the conversation, so clinicians who are listening to descriptions may not pick it up without a computer that analyzes everything the patient is saying.”

The research team also did a sentiment analysis to learn whether the language people used was positive or negative. With regard to the full texts, unsurprisingly, the vast majority of responses contained more negative than positive words, regardless of whether a person had migraine or cluster headache, with a sentiment distribution of 86% and 85% negative, respectively. That sentiment was even more negative specifically in the headache attack descriptions, with a 96% and 95% negative sentiment distribution for migraine and for cluster headache, respectively.

The investigators also went through the texts manually and annotated them at the sentence level to classify content into seven predefined themes: headache attack description, burden of disease, comorbidities, technical investigations (diagnostic tests), triggers, treatment, and previous medical history. That analysis determined that a median of 29.5% of responses in the full cohort included attack descriptions; 17.5% discussed treatment, followed by burden of disease (11.9%) and patient medical history (11.3%).

Applying machine learning
The authors then did experiments to test if ML algorithms could successfully classify headache attack descriptions as belonging to a person with migraine versus one with cluster headache. Because of the small number of participants who described their headache attacks, each participant’s descriptions were used both to train and test the ML algorithm in successive rounds.

Vandenbussche and colleagues used three common machine learning algorithms – called naïve Bayes, support vector machine, and logistic regression – to make the either/or classification. The support vector machine and logistic regression analyses performed the best, being able to successfully classify migraine versus cluster headache with an accuracy rate of approximately 86%.

“The results show you can quantify language in these patients, and this study suggests it is feasible to build models with NLP to distinguish and classify patient narratives to determine whether a headache patient has migraine or cluster headache,” said Vandenbussche.

A proof of concept
Vandenbussche cautioned that while the new work is promising, it is a small proof-of-concept study. That said, even with its limitations, he believes it shows the potential value for the headache field of using NLP in the clinic.

Joshi agreed, but said the results are limited in that the patients in the study had already seen physicians and thus were familiar with different clinical terms associated with their specific diagnosed conditions.

“They already know that if they have migraine, there should be some element of nausea. They may even know the term photophobia,” Joshi said. “A more ideal study would take this kind of model and apply it to a general population of patients with headache that have not been diagnosed by a physician yet – see what words they produce and whether the model can pick up on those trends.”

Vandenbussche said he and his colleagues indeed intend to do those and other types of studies in the future. There are many instances where NLP could help physicians, including in their efforts to determine the severity of the headache or the amount of disability a patient may experience.

“We would also like to investigate whether NLP might be useful for patients who have a diagnosis but are in need of follow-up care,” he said. “We’d also like to look at how NLP might help us better determine the burden of disease.”

Despite the long road ahead, Vandenbussche says that AI/ML, including NLP, has a role to assist general practitioners and headache specialists better diagnose and treat people with chronic headache conditions.

“I believe that the doctor-patient relationship is the most critical part of helping patients, and AI should be secondary, helping the physician instead of making any decisions or taking the lead on patient care,” he said. “But with the knowledge and capabilities we now have in data science, we can use these machine learning algorithms in a very useful way to avoid misclassification and make sure we are giving the best care possible to our patients.”

Kayt Sukel is a freelance writer based outside Houston, Texas.

Reference
Using natural language processing to automatically classify written self-reported narratives by patients with migraine or cluster headache.
Vandenbussche et al.
J Headache Pain. 2022 Sep 30;23(1):129.