How the EFF was pushed to rethink its Secure Messaging Scorecard
As good as the idea behind Electronic Frontier Foundation’s Secure Messaging Scorecard is, its initial version left much to be desired.
The idea was to provide a guideline for users of communication technologies like chat clients, text messaging apps, emails apps, and technologies for voice and video calls, on which of those offerings might be more secure to use than others.
Unfortunately, the seven criteria on which the EFF evaluated the solutions were simply not enough to deem a tool secure. Infosecurity researchers and experts kept pointing out the flaws in the approach, saying that the Scorecard was misleading, broken, and dangerous.
Earlier this year, researchers Daniel Hodson and Matt Jones, from Australian security research and development outfit Elttam, added their two cents to the debate by publishing the results of their (non-comprehensive) code review of RetroShare, one of the apps that scored 6 out of 7 points on EFF’s Scorecard.
“Our preliminary review identified vulnerabilities that show the impact security vulnerabilities can have on privacy applications. The design of such systems need to consider a multitude of attacks, and require a strong foundation on which the privacy functionality can then be built,” they noted at the time, and pointed out that their review was far from complete.
“These kinds of reviews illustrate the need for those in the security community to give back to open source for the benefit of its end users. To those interested, the developers would love to see additional audits, including of the underlying crypto stack and implementation code,” they added.
They, of course, reported the issues, and the RetroShare Team was grateful and quick to fix them.
This was surely one of the things that pushed the EFF to admit the faults of the Scorecard. Shortly after that they set it aside, and announced that they will be working on a “new, updated, more nuanced format for the Secure Messaging Guide.”
“Security is hard, and some aspects of it are hard to measure,” they noted. One of the complaints regarding the Scorecard was also that it was not accompanied with an explanation of which criteria are more important for evaluation than others.
We’ve yet to see the new Scorecard, but in the meantime, Hodson and Jones continued with their work, and on Thursday shared the results of their partial review of Libotr, a “library (C, Java) used by a number of clients to speak the OTR protocol, including Adium, ChatSecure and Jitsi natively and then Irssi, Miranda, and Pidgin via plug-in.”
Again, they found a number of issues and, what’s most interesting, some of these issues have been previously flagged by other researchers, but never addressed.
More details about them can be found in this blog post, along with some very good advice and recommendations for those who run open source software projects.