jÄk: Using Dynamic Analysis to Crawl and Test Modern Web Applications

Summary

Web application scanners are popular tools to perform black box testing and are widely used to discover bugs in websites. For them to work effectively, they either rely on a set of URLs that they can test, or use their own implementation of a crawler that discovers new parts of a web application. Traditional crawlers would extract new URLs by parsing HTML documents and applying static regular expressions. While this approach can extract URLs in classic web applications, it fails to explore large parts of modern JavaScript-based applications. In this paper, we present a novel technique to explore web applications based on the dynamic analysis of the client-side JavaScript program. We use dynamic analysis to hook JavaScript APIs, which enables us to detect the registration of events, the use of network communication APIs, and dynamically-generated URLs or user forms. We then propose to use a navigation graph to perform further crawling. Based on this new crawling technique, we present j¨Ak, a web application scanner. We compare jÄk against four existing web-application scanners on 13 web applications. The experiments show that our approach can explore a surface of the web applications that is 86% larger than with existing approaches.