Discovering the Value of Verifying Web Application Security Using IBM Rational AppScan aka Hacking 101

Editor's Notes

• #9: March,2007: Sixty-six percent of vulnerabilities disclosed during this period affected Web applications. Seventy-seven percent of all easily exploitable vulnerabilities affected Web applications, and seven percent affected servers. With the advent of Windows Vista and the continued use of the Security Development Lifecycle, it is likely that Microsoft-authored code will become more difficult to exploit. As a result, attackers may turn their focus to common third-party applications that are authored by companies that have not employed the Security Development Lifecycle. These third-party applications may not use accepted best software development practices, such as secure design, secure coding practices, code reviews, or secure developer tools such as Microsoft’s Visual Studio.19 As a result, they may be less secure http://www.symantec.com/enterprise/theme.jsp?themeid=threatreport September, 2006 Report: Seventy-eight percent of easily exploitable vulnerabilities affected Web applications. Web application vulnerabilities made up 69% of all vulnerabilities this period.
• #12: .
• #14: In a typical Web application security landscape, the user on the left-hand side interacts with the server environment, on the right. The data that is exchanged back and forth between the user and server environment might be encrypted using SSL, or it may not be, but it moves across the firewalls, intrusion detection systems, intrusion prevention systems, routers, switches to the web server on the other side. Note that it is the web application that facilitates the exchange of data between the server environment and the user. It is interesting to stop and note that while the data received from the client is not to be trusted, the web application itself is implicitly trusted by the backend environment and is permitted to communicate with everything from the database to an LDAP authentication system or to the core network. Let’s take a little closer look at the network protections that might be associated with this exchange of data.
• #15: The first barrier that an HTTP request encounters while crossing the network is a firewall. Firewalls are set up to allow outsiders access to specific resources, and to prevent them from accessing other resources. For example, an outside individual wouldn't be allowed to directly connect to a database, but they can make a request to a web server. This means the firewall would be configured to deny traffic on a standard database port 1443, but allow traffic through ports 80 and 443 - web application ports. This system is clearly no protection at all against malicious attacks. The next protection an HTTP request encounters is an intrusion detection system. The IDS has been set up to look for signatures in the traffic that might indicate an attack. For example, they may look for a SQL statement embedded within a request, or they might look for a script tag for indicates a potential XSS attack. The challenge with these systems is that if the request is encoded in some alternative format (say UTF-7) or perhaps the traffic is encrypted using SSL, the intrusion detection system is often not able to interpret or understand the requests. The IDS offers little to no protection against the web application attack. The next protection that an HTTP request might encounter is an intrusion prevention system or IPS. These systems are designed to explicitly block requests that are deemed to be malicious. It is very similar to the IDS, except that it takes an active role rather than a passive one. Again, if the traffic is encoded or encrypted, the systems may not be able to block malicious requests. The IPS offers little protection against the web application attack. The lines are blurring actually, between the IDS and IPS systems. The last system that the HTTP request might encounter before the web server is probably an application firewall. These are the smartest of all the network protections and can be configured explicitly to only allow through certain traffic that it knows to be good. The problem with these systems is that it's very expensive to maintain the correct configurations or valid algorithms to recognize good traffic. If the web application firewall has been designed to fail securely (a web application security principle), that is, if you're not sure what to do, they block the user, the web application firewall may block legitimate traffic. For this reason, most application firewalls are usually designed to break one of the founding principles of security (fail securely) by allowing through traffic that they don't understand. There is one other issue with web application firewalls. They do not understand the application. As an example, they might be configured to allow a numerical value for a certain cookie value, but they do not know that my user is only allowed a value of 6014, but not 6015. A core principle in web application defense is that THE WEB APPLICATION MUST DEFEND ITSELF. Are web application firewalls valuable? Absolutely! Another implicit challenge faced by the network protections is that they throttle and slow the data traffic. Some organizations are averse to decrypting and re-encrypting, or implementing systems that have noticeable degradation of the user experience. All of these systems feed into the security incident and event management system. You may consider these devices as operational controls or real-time defense against the real-time attack. They are not used to find vulnerabilities but rather to protect the application in real time.
• #17: Security and compliance should be an important consideration during the Software Development Lifecycle. Watchfire solutions of AppScan and WebXM provide tools to ensure web security vulnerabilities are identified and addressed early in the lifecycle and also ensure that web sites are in compliance.
• #19: This security stack clearly depicts the framework of the network through to web application. What areas do each of the VA tools address? Let’s look at the network VA tools first. These do not have any permission on the operating system itself and includes products like Nessus, ISS Security Scanner, QualysGuard, and eEye Retina. These discover and search for vulnerabilities on the network (routers, switches, and firewalls) as well as web servers and known web applications. Host-based assessment products look at applications. (Remember that the operating system is really just a foundational application.) Database assessment products are specialized host-based products requiring credentials, that consider specific issues found in databases – another form of application. Black-box application scanners consider everything at the top of this stack: from the web server to web server configuration, third party components and the web application itself. This might include client-side components such as JavaScript used in AJAX and Web 2.0, web services and service oriented architectures as well as the web application itself. White-box application scanners are limited to the latter component. They can look at the source code of the web application itself, and only where the code is available. So why is it that some network scanners claim that they can find and report on SQL injection, XSS and buffer overflows? This is because there is some overlap in capabilities. Some of these issues (XSS and buffer overflows) exist in the web server itself, while all three (SQL Injection, XSS and buffer overflows) can be found in third party components. Network scanners DO NOT explore the web application or find any issues that are introduced in the application itself. Some overlap exists because both black-box application scanners and network scanners are able to find issues in the web server, web server configuration and third party components. So is a web application scanning technology really necessary? Where do the vulnerabilities exist? It is interesting to realize that 10 years ago most of the vulnerabilities lived in the network or operating system. This is no longer the case. IT administrators to become very smart about security and products have gotten very mature in the lower levels of the stack, right up to, and including, third party components used by web applications. The people who understand security are those that control the network, operating systems and applications. These issues are solved through configuration and patching. The challenge arises because they IT administrator does not understand the application – a requirement for securing the application. The web developer, who understands the application, does not understand security or have sufficient tools or knowledge to eliminate this issue. Further, while there a finite number of IPs in a network, or a finite number of ports on a device to be analyzed, a web application has a potentially infinite number of entry points, from forms, to query strings to cookies to header fields. This leaves the application with a significant number of vulnerabilities. Crackers are not unaware of this. Gartner reports that 75% of attacks now occur at the network layer and Watchfire has found that 90% of applications are vulnerable. One last thought. There is implicit trust in this stack. If the malicious individual is able to find an issue in the web application, keep in mind that the web application is typically trusted by the database, the applications, the operating system and the network. The entire system may be compromised.
• #20: This chart details more of the differences between CWVs and ASVs and ultimately points out how an organization can most effectively reduce security defect costs. Basically, an organization has very little control over the costs to find and fix CWVs and a lot of control over the costs to find and fix ASVs. CWVs are a result of 3 rd party defects and as such can only be found once the application is in production. Because they are relatively easy to identify, and have patches issued for them that are publicly available their cost to the organization is relatively low in terms of finding and fixing. On the other hand, ASVs are defects introduced during the application development lifecycle, are very difficult to identify manually, and require the entire app lifecycle process for creating a fix, therefore, the ability to control the cost is relatively high. The cost to fix a vulnerability once it reaches deployments is 100 times greater than if it were caught and fixed in design. Because an ASV can be caught throughout the application lifecycle the organization has the ability to control this cost.
• #22: The OWASP Top 10 list, includes the following 10 common security issues, which we will cover in a moment.
• #23: A Cross Site Scripting attack, attempts to echo back a malicious script in the HTML returned from a trusted site. Since the script is echoed back from a trusted site, it runs in the context of that site. The implications of XSS are: Stealing HTTP session tokens Page content may be compromised (this may include “local” site defacement, or hijacking of the browser’s session using scripting) Future pages may be contaminated as well (by hijacking the session)
• #24: Let’s take a look at the chain of events during a XSS attack The attack creates and sends the victim a link to bank.com (a trusted site). The link contains a search string (or any other string that is echoed back), which contains a malicious JavaScript code The victim, clicks on this link, since he/she trusts the bank.com web site The bank.com web application, echoes back the malicious JavaScript code inside the response page. This JavaScript is executed in the security context of bank.com, since it is echoed by from that site. This means that it has access to DOM elements belonging to this domain/session The malicious script, sends the current cookie and session information, without the victim’s consent, to the evil.org web site, where the hacker is waiting for it.
• #25: Let’s take a look at the following banking web site – this site contains a search function, that allows users to search the site for specific text. If we type the string “asdf”, the response to the search will contain that string, inside the results page, in what we call “free HTML context”. What will happen if instead of typing “asdf”, we will type some JavaScript code? Let’s try to type the following JavaScript code: <script>alert(document.cookie)</script>
• #26: As you can see – the piece of JavaScript code that we wrote, was echoed back by the site’s search function – since it was returned from the banking application, it had access to the Document Object Model (DOM), and could access the current session cookie. In this situation, I myself planted this JavaScript code in the web page, but in a XSS attack, it is the attacker who creates a link that contains the malicious JavaScript, and then sends this link to the victim. When the victim clicks on the link, the malicious JavaScript will be echoed back from the trusted site.
• #27: XSS usually occurs in pages that echo back user input, for example – search pages, error pages and forms that are returned in subsequent pages Echoed input can come from any part of the HTTP message that is used by the application, for example: parts of the path, query, cookie or other headers. Some browser technologies can help with mounting the XSS attack, for example XMLHttpRequest (used in AJAX), flash objects or IFrames. There are several different flavors and variations of XSS, for example – XSS in HTML attributes, DOM Based XSS, etc.
• #28: If a hacker can get you to run a JavaScript, he/she can: - Steal your cookies for the domain you’re browsing - Completely modify the content of any page you see on this domain - Track every action you do in that browser from now on - Redirect you to a Phishing site - Exploit browser vulnerabilities to take over machine XSS is currently one of the “hottest” security risks
• #34: Injection flaws occur when user supplied data, is sent to an interpreter as a part of a command, query or data. The main issue here is that user input is not sanitized, and is embedded in pre-existing commands. Injection flaws can occur in: SQL queries (known as SQL Injection) Server Side Includes (execute commands on the web server) LDAP queries – used to bypass authentication
• #35: SQL Injection occurs when user input is embedded as-is inside a pre-built SQL query. For example: Let’s assume that our web application receives a product ID as input, and presents that product’s page. The SQL query looks like this: “ Select * from products where id=‘” + $REQUEST[‘id’]; You should note, that the query is basically a text string, and user input is concatenated to it. In this example, the user string is surrounded by apostrophes. Let’s take a look at what will happen if we submit the product ID value of ‘ or ‘’=‘ The query will be: SELECT * from products where id=‘’ or ‘’=‘’; You should pay attention to the fact that the WHERE criteria here is basically a Boolean TRUE. Since the results of this query matches every entry in the database, all the products will be returned.
• #36: Let’s take a look at how SQL Injection can assist a hacker to bypass the login mechanism of a banking application: - First, in order to sense that SQL Injection is possible, the hacker will injection the character apostrophe (‘), as the user name
• #37: This yields a very informative SQL error message, which helps the attacker to devise the next phase of the injection
• #38: Now, the hacker attempts to send the username: ‘ or 1=1— Note: the apostrophe is used to close the string context in which our input is embedded in 1=1 is a Boolean TRUE -- is used in MS SQL to comment out everything after the – sign, so we don’t have to worry about the rest of the SQL query
• #39: After sending this SQL injection payload, we will be logged into the application, as the first user in the user's table - without having to supply actual credentials.
• #40: Comments at one level can be command at another
• #42: Let's continue to the next item on the OWASP Top 10 list - Malicious File Execution. In Malicious File Execution, the hacker attempts to trick the application into executing commands or creating files on the server. The implications of this attack are: - The hacker can execute remote commands on the server, which means a complete takeover - The hacker may deface the web site.
• #43: Let’s take a look at our banking application again – The application contains a feedback form, which allows users to send the application owner all sorts of feedback. This feedback is submitted and appended to a file on the operating system. Since the application was designed poorly, the location of the feedback is taken from a “hidden” form parameter called “cfile”. All a hacker has to do in order to “create” a new file on the operating system, is manipulate the value of the “cfile” parameter to a different filename, and submit contents to that file (submitted as feedback).
• #44: Let’s perform the attack – We change the value of the cfile parameter to myevilfile.aspx (we are creating an ASPX file, which is a server-side Microsoft ASP.NET script) Instead of a feedback, we’ll fill this file with some C# code, that will reveal the contents of the system’s hosts file
• #45: When requesting the server-side script we just created, the application will execute it for us, revealing the contents of the hosts file. Game Over!
• #46: In several scenarios, it may be possible for an attacker to manipulate the web application to disclose a resource such as a sensitive file. This can occur by either guessing a common file name and location and attempting to request it, or by manipulating a parameter value that is used to access a file, as will be seen in the next example. The implications of Insecure Direct Object Reference is usually information leakage or access to sensitive resources.
• #47: In this example, we see that a web application that uses a parameter called “content”, which points to the contents of the page to be displayed. An attacker might attempt to manipulate the parameter value, from “business_deposit.htm”, which is the valid page, to some other file – for example, the Boot.ini which is a system file.
• #48: The attempt failed, and the system disclosed that it only allows parameter value (file names) that end with either txt or htm as their file extension. Let’s try a little trick called “Poison Null Byte”, we’ll write the file we actually want to open which is Boot.ini, but append a NULL character and the extension the application is looking for (in this example .htm)
• #49: Bingo! – we managed to circumvent the file extension validation, and open a sensitive system file. Using this technique, we can manipulate the application to hand us the contents of other, more sensitive files, such as databases, customer files, etc.
• #50: Let's move on to the next item on the OWASP Top 10 list - Information Leakage and Improper Error Handling. Information leakage vulnerabilities usually do not allow a hacker to perform malicious actions, but rather enable the attacker to gather sensitive information, either about the application or about its users. This usually happens when application debugging information is not sanitized from response pages, when all sorts of errors (such as SQL error messages as shown before) occur, but it can also happen from more naive mistakes, such as leaving personal information or debugging remnants inside HTML comments. The implication of Information leakage can range between sensitive data being exposed, to web application internal logic being visible to the hacker.
• #51: Let’s take a look at two information leakage examples: The first example is a simple one – the administrator left his/her phone number inside HTML comments, assuming that users do not read them. This information can be harvested and later on used for social engineering purposes The second example is the same as our previous SQL Injection scenario – if we submit a value (in this case apostrophe), that the application does not know how to handle, it might spit back debugging information.
• #52: What you see here, is the response to the form submission, which included an apostrophe character as the user name. This error page reveals information about the type of SQL database that is used, and about the structure of the SQL query, allowing us to further devise a SQL Injection attack against the web application
• #53: One last example of information leakage, which is also very common, is verbose login error messages. Some applications, will present the user with different error messages when the login process failed due to invalid username or invalid password (as seen in this slide). While this is not a severe issue, it narrows down the amount of time it will take the hacker to guess (or brute force) his/her way into the application.
• #54: The last item on the OWASP Top 10 list, is Failure to Restrict URL Access. One of the most common security issues in web applications is the lack of proper access restrictions. Many people forget that not having a link to a resource, doesn’t mean that hackers can’t guess it. You should always limit user’s access, by putting URL access restrictions. Portions of the site that belong to administrators, should never be accessed by regular users. Failing to create proper access restrictions on web application resources might lead to: Information leakage Privilege escalation
• #55: We'll now take a look at a simple Privilege Escalation example - This example, shows how unrestricted access to the administration page, may lead to complete compromise of the web application. When we log into the application as the administrator, we are presented with a link to the user account editing page. Naturally, this link doesn't appear in regular users' screens, nor should it be accessible for them. Since the application doesn't restrict access to this page, a hacker can attempt to guess that link, and take over the web application, as will be seen in the following slide.
• #56: In this slide you see how the user authenticates as a regular user, but since the application did not have proper URL access permissions, the user guessed the link to the administration page, and was able to perform actions on behalf of the administrator, without having to log in as a high privileged user.
• #57: You should always block users from requesting and retrieving restricted resources (for example, backup files, log files, source code files, etc.) Regular users shouldn’t be allowed to access pages that belong to higher privileged users (this is referred to as vertical PE) Regular users shouldn’t have access to other regular users’ pages (this is referred to horizontal PE)
• #64: Rational has an integrated solution that allows for Testing(both manual and automated) and Change management. Watchfire’s tools AppScan and WebXM expand our exisitng solution to allow for security and compliance testing. WebXM allows organizations to ensure the complicance of their web sites and AppScan the quality and security of the websites. TEST AND CHANGE MANAGEMENT The key components of Rational’s software development and test platform are: Rational Team Unifying Platform, which contains RequisitePro for capturing and tracking software system requirements and use cases, ClearQuest for building detailed test plans executing automated and manual tests and analyzing test results and ClearQuest for managing all defects and change requests. DEVELOPER TESTING Rational PurifyPlus for run-time analysis, performance profiling and code coverage analysis, Rational Test RealTime for component test and run-time analysis for embedded applications. FUNCTIONAL AND PERFORMANCE TESTING Rational Functional Tester for automated build validation and functional/regression system testing Rational Manual Tester for building modular, reusable manual tests and automation of common error-prone tasks in manual testing Rational Performance Tester for load and scalability testing and transaction breakdown analysis to identify the source of a performance bottleneck
• #65: This proof of technology focuses on vulnerability testing using AppScan
• #67: AppScan scans for vulnerabilities by traversing an application similarly to the way a user browses a website. It starts from the home page or some other entry point, as defined by the user, and follows all the links. Each page is analyzed, and based on the characteristics of the page, AppScan sends a number of tests. The tests are sent in the form of HTTP requests. AppScan determines the presence of vulnerabilities based on the responses from the web server. The application is treated as a black box and AppScan communicates with it just like a browser does. AppScan Enterprise has thousands of built-in tests and checks for hundreds of different types of vulnerabilities.
• #76: It’s time to join the League of Extraordinary Software Development Professionals at the 11 th annual Rational Software Development Conference, which will be held on June 1 – 5, 2008 at the Walt Disney Dolphin & Swan Resort. This is the premier event where software development professionals come to learn how to heroically team with each other and IBM Rational to combat the evil forces of information isolation, data overload, poor processes, and barriers to distance and language. Winning in today’s geographically distributed world takes more than just individual heroics. It takes Collaboration! Effective Organizational Ability! Teamwork! For more information on the tracks and or to register at www.ibm.com/rational/rsdc This year added: Application Security and Compliance Track