Unlike popular belief, short links on Twitter aren’t malicious

Twitter recently announced that it has implemented a new security system to scan all URLs posted in tweets to protect users from malicious sites. This follows a similar announcement from bit.ly in November 2009.

Twitter, and the URL shorteners it has helped to popularize, have long been blamed for leading users to malicious sites. I wanted to do a thorough investigation of the Twitter links both before the security scan and after. I have retrieved more than 1 million URLs (1,314,615 to be exact) from the public timeline over a couple of weeks before they put any protections in place. I then ran the links through the Zscaler infrastructure to find out which links lead to malicious sites.

The state of the Twitter links

As expected, URL shorteners are very popular on Twitter, and bit.ly represents 40% of all links. TinyUrl, one of the original URL shorteners, comes in 3rd with only 5% of all URLs.

I looked for malicious sites – phishing sites, malware, etc. I did not look for spam, only for pages that present a security risk to users. To my surprise, a very low number of links led to malicious pages – only 773, links, 0.06% of all links scanned, redirected to malicious content.

Bit.ly represents 40% of all links, and roughly the same proportion of malicious links. Same case for TinyUrl: 5% of all URLs and 6 % of all malicious sites. It does not look like bit.ly’s phishing and malware protection is making it any safer than other URL shorteners. Twitpic.com is used to share images, so it is unlikely to be used for malicious content. Mediafire is known for hosting malware and other viruses, even if it is not blacklisted by Google Safe Browsing. Note that these links may have been scanned up to 4 weeks after they were collected. Bad sites may already have been taken down, or cleaned up.

Can Twitter and bit.ly really protect their users?
The key to protecting end users, is real-time scanning of both the URL and the content. Twitter and bit.ly can only scan the links periodically. Malicious websites try to hide their malicious content to non-users by checking the user agent or geography and by requiring a real browser which fully understands Javascript, Flash, etc. An attacker can present harmless content to the Twitter or bit.ly scanners, but harmful content to a real user. But remember that only 0.06% of all the URLs tests represented a security risk. It is actually much safer to follow link s from Twitter that from some search results on Google.