Researchers create system that predicts vulnerability severity from tweets

To what extent do users’ opinions about threat severity expressed online align with expert judgments and can these opinions provide an early indicator to help prioritize threats based on their severity?

A group of researchers from Ohio State University, Leidos and FireEye wanted to answer those questions, so they:

• Annotated a collection of tweets describing software vulnerabilities with opinions on threat severity
• Matched tweets to NVD records, i.e. CVEs (by using CVE numbers in the URL or web pages linked in the tweets)
• Defined a severity forecast score and a threat severity classifier to assign it (before the NVD publication date)
• Waited for the official CVSS severity score to be announced and compared it with their forecasted score, to see whether their models’ performance at identifying severe threats was precise enough.

“Having a large number of tweets beforehand is a good indicator for high severity, however our approach which analyzes the content of messages discussing software vulnerabilities achieves significantly better performance; 86% of its top 50 forecasts were indeed rated as HIGH or CRITICAL severity in the NVD,” they found.

“We [also] observe that our model can predict accurate severity level even 19 days ahead of the official published date in NVD.”

Their system could prove useful for those in charge of patching and prioritizing the patching of vulnerabilities, as they also found that reports of severe vulnerabilities online is a good predictor of real-world exploits to follow.

“As the rate of discovered vulnerabilities has increased in recent years, the need for efficient identification and prioritization has become more crucial,” they researchers noted.

“However, it is well known that a large time delay exists between the time a vulnerability is first publicly disclosed to when it is published in the NVD; a recent study found that the median delay between the time a vulnerability is first reported online and the time it is published in the NVD is seven days; also, 75% of threats are first disclosed online giving attackers time to exploit the vulnerability.”