The HTTPS interception dilemma: Pros and cons

HTTPS is the bread-and-butter of online security. Strong cryptography that works on all devices without complicating things for users. Thanks to innovative projects like Let’s Encrypt, adoption of HTTPS is rising steadily: in mid-2015 it was at 39%, now it’s at 51% of HTTPS requests.

Recent research shows however that HTTPS interception happens quite often. In fact, about 10% of connections to CloudFlare are intercepted, and the main culprits are enterprise network monitoring products. Without HTTPS interception a network monitoring tool will only ‘see’ the Internet domain names and/or the IP addresses of the two sides of the connection. Not the full URL or the content of the communication, for example.

But HTTPS interception is controversial in the IT security community. There are two sides in this debate. Much depends on the setting you are in. In this article I want to outline the benefits and drawbacks of HTTPS interception, just from an IT security perspective.

The benefits of HTTPS interception

Detect malware downloads: In most cyber-attacks the end-user is triggered to download malware or an infected file, for example via a phishing attack, a waterhole attack or a malvertisement attack. HTTPS interception would allow the network proxy to see the binaries and documents downloaded and this means they could be scanned, by comparing with known malware signatures or by opening documents in sandboxes.

Detect C&C traffic: Command & Control traffic to exotic domain names and IP addresses is the hallmark of an infected device calling back to the attacker’s infrastructure. To avoid detection, attackers have started to use legitimate websites for C&C traffic, for example a Twitter feed of a burner Twitter account. Without HTTPS connections you can only see there is an Internet connection with Twitter. So it blends in with normal user traffic. HTTPS interception would allow the Internet proxy to see also the content, for example which Twitter accounts are accessed. This could in principle be used to distinguish C&C traffic from normal user traffic.

Detect exfiltration: Attackers could use HTTPS connections to exfiltrate data. In principle HTTPS interception could be used to detect if corporate documents or files are being uploaded, for example by looking for known patterns, markers or headers in documents.

Bypass HTTPS weaknesses: You’d be forgiven for having lost patience after more than a decade of problems with HTTPS: First slow adoption by websites, then bad issuing practices by Certificate Authorities (CAs), then security breaches at CAs, then lax implementation by browsers, and finally we have trained all end-users to click blindly on “Yes” or “Can I continue now” with each warning message. Intercepting HTTPS before it reaches the browser or the end-user could in principle be used to bypass the issues with the browsers and the end-users.

Speed: HTTPS interception can be implemented relatively quickly, especially if you already have a proxy in place for outgoing Internet connections. Most products in fact offer it as a simple add-on. Of course the work of implementation does not end here (see below) but making changes to endpoints, such as installing software or hardening, could take much longer and require a lot of work.

Reasons against HTTPS interception

HTTPS interception is controversial in the IT security community for a good reason. Here are 10 good reasons for not doing HTTPS interception.

1. Are we serious? After a decade of telling everyone to implement HTTPS, educating users to check certificate warnings, preaching about how fundamentally important it is to encrypt network traffic, at the very moment that HTTPS is finally picking up, we scramble and start to intercept it, acting out the very man-in-the-middle attacks it was meant to prevent. It does not look very consistent or serious. But let’s move on.

2. Strict transport security: HTTP Strict Transport Security (HSTS) is an Internet standard allowing websites to tell the browser to never accept non-HTTPS connections. This is important when a user forgets to type the S in the URL and it prevents from stripping attacks. A similar thing is done with cookies. If the website sets the cookie with a secure flag, then the browser will not send it back without HTTPS. These protections are important to prevent man-in-the-middle attacks. It also means you cannot just remove the SSL/TLS connection without breaking things. So intercepted HTTPS connections will have to be re-encrypted by the intercepting proxy. This is a kind of impersonation/spoofing.

Part of the deal with HTTPS interception is that connections are re-encrypted with a fake certificate, a wildcard certificate (*), which is valid for all websites. This is a kind of impersonation or spoofing, except that it is done with good intentions. The wildcard certificates are in fact not sold by real CAs, but an enterprise could create one with a non-official internal CA. This wildcard certificate then needs to be installed on all the PC’s in the enterprise.

3. Whitewashing: Re-encryption with wildcard certificates effectively makes the browser and the user blind. The browser will no longer be able to warn the user about HTTPS connections and the end-user has no way to see if the certificate is valid and if the connection can be trusted. The original certificate is whitewashed.

This would not be a problem of course if the intercepting proxy is perfect and flawless, refusing all the bad connections, accepting only the good ones. But this is a tall order. A recent report actually shows that many of these interception products are very bad when it comes to accepting certificates, effectively opening the door to all sorts of attacks (decryption, downgrade or stripping attacks). This raises some liability questions.

4. Disrupts personal use: Social media, webmail providers and online banks ask their users to verify the HTTPS connection. In the case of HTTPS interception this is impossible. Maybe HTTPS interception requires some legal disclaimer about liability. But apart from the legal matters, many employees would no longer use their corporate PCs for things like social media, personal web mail or web banking. In some settings this is not a problem, but I think that for most organizations it is important to allow some form of personal use of corporate PCs, for example to allow employees during their break to make a bank transfer or buy their groceries online.

5. Certificate transparency: It is not enough to re-encrypt connections with wildcard certificates. Certificate transparency is an extra protection measure allowing browsers to check if a certificate is normally used by that website. A browser like Google Chrome, for instance, warns users when a Google page is shown with a different certificate, even if it is formally valid. It is a reaction to the continuous security problems and breaches at CAs. It is an important feature and it helped to discover the large-scale MITM attack on Iranian Internet users, aka the Diginotar breach. So HTTPS interception requires you to tweak also the browser to accept the masquerading of the original certificate without complaints. Certificate transparency needs to be turned off.

At this point it should be clear that HTTPS interception involves not only a quick intervention at the network monitoring box, but that it actually involves changing how endpoints, browsers, and ultimately the end-users, handle HTTPS connections. Choosing HTTPS interception has wider implications for IT at the organization.

6. Breaks with consumerization: IT in the workplace is driven by consumer products and services. Also endpoint and browsers and their security is driven by the consumer market. But because HTTPS interception has no place in the consumer market, this means that important protection like certificate transparency and certificate pinning do not work anymore in the enterprise. It would have to be tweaked or turned off on the browsers, and then implemented again on the intercepting proxy, for example. Also the awareness raising material, tutorials, and warning messages about HTTPS cannot be re-used anymore.

7. Disrupts BYOD: Employees are increasingly using their own personal computers in the office and for work. Sometimes devices are used side-by-side with corporate devices. To implement HTTPS interception personal BYOD devices will need to be tweaked and configured, to install and trust the wildcard certificate, and to turn off browser warnings. This not only leads to a lot of work by the IT department, which runs counter to the very idea of BYOD, but it is also likely to raise some eyebrows with users. HTTPS interception does not play well with BYOD.

8. Discourages good practices by the users: Even if we ignore BYOD, there is also the problem that employees have personal devices for personal use. Social media, online banks all ask end-users to inspect certificates and to heed browser warnings. With HTTPS interception in the enterprise and no HTTPS interception at home, the employee is dealing with two worlds: At work all HTTPS connections look strange, but they are to be trusted. At home when HTTPS connections look strange it is an attack. This is confusing for the end-user and this creates risks.

9. Limited benefits: The benefits of HTTPS interception are small and will diminish:

• Malware detection is failing. Attackers evade signature-based detection by using polymorphic malware. Sandbox-based detection is being evaded also. It is easy to see that traditional signature-based antivirus programs are losing the battle. In fact it is much more important to keep systems updated then to install extra software to detect malware. Network monitoring tools cannot do better than AV (only worse actually.
• Attackers also know how C&C traffic is detected, so they hide it. For example there are attacks in which the C&C communications are hidden in JPG images posted on Twitter timelines. In these attacks HTTPS was not even used by the attacker! The problem is not the HTTPS encryption but the fact that there is a sea of communications to hide in.
• The same applies to exfiltration. For an attacker obfuscation is more important than encryption. If needed attackers (insiders and outsiders) could always use an extra layer of cryptography.
• Perhaps the most compelling reason for HTTPS interception is to bypass the flaws in the browsers and the weakness of the end-user ignoring warnings. But also this benefit is diminishing. Browsers are implementing HTTPS better. It is harder for users to ignore HTTPS warnings.

10. Hard-shell-soft-inside: HTTPS is an extension of network monitoring and detection. Investing in network monitoring and detection now is like betting on the horse that is lagging behind and visibly tired. Not only is it easy for attackers to evade detection, it is a continuation of the traditional approach based on securing the corporate perimeter (hard-shell-soft-inside). It is known that this approach is flawed and it also clashes with the increased mobility of staff and the uptake of cloud services. Implementing HTTPS just goes further down the wrong path.

Conclusion

HTTPS interception can give some short-term benefits specifically for organizations. But these benefits are limited and diminishing. More and more the emphasis is on hardening the endpoints. HTTPS also has important drawbacks. Instead of going against the wave of HTTPS uptake, by tweaking how HTTPS works inside the enterprise, it is smarter to ride it and work with HTTPS as an assumption.

It is wiser to invest in protection measures that have a real chance against attackers, such as patching and updating, removing local admin rights from PCs, removing risky plugins, preventing users from installing software, starting detection and monitoring on the endpoint, etc. If you do decide to choose HTTPS interception then make sure it is a temporary work-around and start implementing the measures needed to phase it out.

Disclaimer: This post does not express the views of my employer, it is my personal view, from a sheer technical IT security perspective. It is not the full picture and, for example, it doesn’t cover the legal implications.