Automating web app testing to secure your environment

In this podcast recorded at RSA Conference 2018, Dave Ferguson, Director, Product Management for Web Application Security at Qualys, talks about the challenges and benefits of automating web app testing, Qualys Browser Recorder, as well as Qualys Web Application Scanning.

Here’s a transcript of the podcast for your convenience.

OK. Hi everyone. My name is Dave Ferguson, I’m the product manager for web application security here at Qualys, and in this Help Net Security podcast I’m going to be talking about web application security, and specifically about automating some of the testing of web applications, as well as APIs to help secure web applications in your environment.

So, first of all I think it’s no question that web applications and vulnerabilities and web apps are a problem. Businesses depend on web applications and more and more on APIs. Because applications can be a foothold in your organization and application developers are not really incentivized for security, you can end up having some security holes in your environment that you may not be aware of.

Developers are definitely incentivized to build functional applications, highly performing applications, but security not so much. And we already know that applications are being targeted. In the 2017 Verizon Data Breach Investigations Report web apps were the most prevalent attack category.

And so the fact is that web apps and APIs are everywhere. You have your public facing web apps, Internet facing, you have your internal production web apps, you have new applications under development. You have REST APIs to support mobile applications and machine to machine communication. You have web apps and public clouds like Amazon and Azure and Google, any of these can be a place where attackers can target you, and they all need to be considered.

So, the challenge is though are that software is being written at such a breakneck pace that developers are creating more software faster than our ability to secure it, and more and more we see a change in the development methodology, more of a move to DevOps and faster development, continuous integration, continuous deployment. That’s introduced even another level of difficulty because security testing can’t slowdown that process.

But the fact is all applications need some level of testing to, kind of the security hygiene concept, but manual testing approaches like manual pen testing is costly and not necessarily scalable. Automation is really the key to success in web application security testing. So, many of you I’m sure are aware of the OWASP Top 10, it’s probably the most well-known list in the world of application security. There is another list from OWASP that it’s really more focused for developers and it’s called the Proactive Controls, and it’s a top 10 list prioritize ranking of security techniques that should be included in every software development project.

And in fact control number one, the most important control in that list, is to verify for security early and often. The fact is that security testing should really be an integral part of your software engineering practice. Now what does security testing mean? Well it could mean a dynamic analysis of a running application. It could mean static code analysis – either source code or binary code. It can mean a variety of things. The challenge is for control number one are simply the sheer number of applications and limited resources. As I mentioned, manual tests are costly, they do require special skills.

And what we’ve found is that some of the newer architectures being deployed now such as micro services, that’s actually hindering some things like static code analysis, because with static code analysis a key part of that is data flow analysis, so you track the data from the source, the point where it enters the application, to the sink where it ends up getting used. With micro services architecture everything is simply a little HTTP call an API call, and those architectures are hindered static code analysis because the data flow can’t be tracked across those API calls.

More challenges for security testing or authentication to the application: complex workflows, new types of vulnerabilities that are being introduced and discovered. Security testing can delay software deployments, and testing of REST APIs, as I mentioned API is becoming more and more prevalent, automated crawling through browser interface is not really possible since the REST APIs are not a browser based application.

Given the fact that every web application every set of APIs should be tested at some level, how can you do that effectively? Automation is the key. So, what does that mean? First of all let’s talk about browser-based web applications. One of the quickest and easiest ways to get security testing done there is with a dynamic scan. Dynamic scanning simply needs a running instance of the application. You can tell the scanner that URL, and simply launch a scan, and the testing will begin.

Now to do deeper scanning of the application you’re going to need to set up authentication so the scanner can log into the web application and test the authenticated surface area of the web app, and you’re also going to need to train the scanner in some cases if there are any complex business workflows. For example, if there’s a Web page in a form that requires input of certain types of data, or a certain order things need to be done.

Qualys Browser Recorder

So, those workflows need to be recorded, and then the scanner can play those back during the scan so that it can reach all areas of the web app. We’ve recently introduced a Chrome extension called Qualys Browser Recorder. This is designed to help with those complex workflows and authentication to web applications. This extension is free for anyone to use. It’s not just for Qualys customers, but our Qualys Web Application Scanning product consumes scripts recorded from the Qualys browser recorder, so that the scanner can then play those back during the scan, and that’s available today in the Chrome Web Store.

Another factor that you need to consider with automation is in the DevOps environment where developers are writing code and deploying it quickly. There needs to be still some sort of testing being done. You can’t hold up a deployment to do a one week pen test. So how do you get that testing done in an automated way that doesn’t interfere with the development process? One way that you can do that is to introduce scanning as part of the continuous integration, continuous deployment environment.

A popular tool today, there’s Jenkins, there’s others as well, but Jenkins seems to be very popular out there. In that case developers are checking in code, Jenkins picks up the changes, compiles the code, builds it, pushes out to say a dev environment or QA environment. So, something that we’ve introduced to help in that CI/CD environment is a Jenkins plugin, and with the plug in you can launch a web application scan against your application automatically. No need to manually launch the scan, it’s all done as part of the normal build process.

Another type of approach that is really need to be considered with web app security is testing of the REST APIs. That’s a different sort of animal because it’s not a browser based web applications, but they can have many of the same vulnerabilities as web applications. and still have SQL injection in REST API. You can still have remote code execution.

So, how do you get testing done with REST APIs? So, what we’ve done at Qualys is in our web app scanning product is introduce support for Swagger. Swagger is a specification allowing you to describe your REST APIs, how they’re invoked, what are the parameters, what HTTP methods are supported. With that Swagger support, if you have a Swagger file with your set of REST APIs, the scanner can simply consume that Swagger file, understand how to invoke the APIs and therefore be able to test them. So, this is a way that, again, more about the automation is you can automatically test for vulnerabilities and those REST APIs simply by having a Swagger file. And typically, if developers are writing the REST APIs they’ll have that Swagger file and they can provide they can provide that to the security team or they can run it as part of their DevOps process and get those rest APIs tested.

So just to summarize Qualys we’ve introduced several new pieces to the web application security puzzle, all kind of focused around automation. These are all introduced in our Web Application Scanning 6.0. Certainly for information you can go to our website at Qualys.com. Thank you again for listening today.