Voice analysis technology prevents phone scams
Nagoya University and Fujitsu developed the first technology to analyze phone conversations to automatically detect situations in which one party might “over trust” the other party. Specifically, this is the kind of situation in which an individual may have a diminished capacity to objectively evaluate an explanation being given by the other party.
By combining this technology for detecting situations of overtrust from voices over the phone with the detection of characteristic keywords, Nagoya University and Fujitsu have also developed basic technology for detecting remittance-soliciting phone phishing scams.
Starting this month, a prototype of this technology deployed in mobile phones will undergo verification testing in collaboration with National Police Agency of Japan (the National Police Academy) and The Bank of Nagoya.
Background
There are limits to human powers of perception and judgment. When overwhelmed with information that may be distressing, some individuals, without knowing it, may have a diminished capacity to objectively evaluate information provided by another party—a situation known as “overtrust.”
In situations of overtrust, there is the risk of believing everything another person is saying, even in cases of remittance-soliciting phone phishing scams, for example. In order to prevent such scams, there is a need to detect such situations and provide appropriate support.
Technological issues
To be able to detect a situation of overtrust that results with distressing information from another party, speech recognition systems have been used with technology that is able to detect when special keywords are included in a conversation.
When a person is subject to psychological stress, however, the person’s voice can become indistinct, making methods that rely on keyword detection alone insufficient in terms of detection accuracy. Developing a way to detect situations of overtrust with a high degree of accuracy even under these types of circumstances has, therefore, posed a challenge.
Newly developed technology
When a person is subject to psychological stress because the other party is conveying distressing information, situations of overtrust can occur. To address this issue, Nagoya University and Fujitsu developed the world’s first technology that can infer situations of overtrust by detecting changes in voice pitch and level, as illustrated in figure 1.
Figure 1. Example of changes in voice pitch and level.
In a test using this technology to detect remittance-solicitation phone phishing scams, it was demonstrated that situations of overtrust could be detected with over 90% accuracy, resulting in the impending creation of basic technology for detecting situations of overtrust.
Use in detecting remittance-solicitation phone phishing scams
In preventing remittance-solicitation fraud, a serious social problem, effective methods are believed to entail support in detecting remittance-solicitation phone phishing scams and alerting the intended victim as well as alerting the intended victim’s family or other objective third-party member, as illustrated in figure 2.
Figure 2. Detecting remittance-solicitation phone phishing scams, offering support.
The detection process, as illustrated in figure 3, involves inferring situations of overtrust and then detecting certain keywords that indicate whether the conversation potentially involves an attempted fraud. These results are then integrated to detect remittance-solicitation phone scams.
Operation of this technology is checked by using a keyword list provided by the National Police Academy and recordings of actual remittance-solicitation phone scams.
1. Inferring situations of overtrust from the intended victim’s voice
Nagoya University and Fujitsu created a function that, by detecting changes in voice pitch and level, as shown in figure 1, is able to infer situations of overtrust on the part of the intended victim when overwhelmed by distressing information from the suspected perpetrator.
2. Detecting special keywords from the suspected perpetrator’s voice
In cases of remittance-solicitation phone phishing scams, the suspected perpetrator will often pretend to be an acquaintance of the victim or a person in a position of authority, such as a police officer or lawyer. They will then convey distressing information, such as relating to a scandal or crime supposedly committed by an acquaintance of the intended victim.
Nagoya University and Fujitsu developed a function using word spotting voice recognition technology to identify when the suspected perpetrator uses special keywords from a pre-registered list, such as “indebtedness” or “compensation.” The function using word spotting voice recognition technology, which ignores everything except the keywords on the pre-registered list, detects the number of times keywords relating to remittance-solicitation scams are spoken.
Figure 3. Technology for detecting remittance-solicitation phone scams.
Future plans
Using mobile phones containing this technology for detecting remittance-solicitation phone scams, Nagoya University and Fujitsu will move forward with verification testing of the technology, in collaboration with the National Police Academy and The Bank of Nagoya. The aim is to evaluate the practical utility of the technology in a variety of actual cases of remittance-solicitation fraud, identifying any potential problems, and devising solutions to overcome the problems.