ITS4: A Static Vulnerability Scanner for C and C++ Code

John Viega, J.T. Bloch, Yoshi Kohno, Gary McGraw

Reliable Software Technologies
Dulles, Virginia
fjviega, jtbloch, kohno, [email protected]
http://www.rstcorp.com

Abstract flow attacks are perhaps the most common security flaw in
applications today. The technical details of these attacks are
We describe ITS4, a tool for statically scanning widely discussed in the security community [6, 17].
security-critical C source code for vulnerabilities. Com- In practice, discovering gets in a program usually in-
pared to other approaches, our scanning technique stakes dicates a security problem. Nevertheless, this function
out a new middle ground between accuracy and efficiency. still resides in the standard C library, along with other
This method is efficient enough to offer real-time feedback problematic constructs. Some well-known “gotchas” in-
to developers during coding while producing few false neg- clude sprintf, strcpy and strcat. Wagner et. al.
atives. Unlike other techniques, our method is also simple discuss more subtle buffer overflow problems with com-
enough to scan C++ code despite the complexities inher- mon C functions, including the so-called “safe” alternatives
ent in the language. Using ITS4 we found new remotely- snprintf, strncpy and strncat [17].
exploitable vulnerabilities in a widely distributed software Buffer overflow vulnerabilities are not the only known
package as well as in a major piece of e-commerce soft- security problems in C and C++ programs. For example,
ware. The ITS4 source distribution is available at http: system and popen, two library calls for running pro-
//www.rstcorp.com/its4. grams through the command shell, are both notoriously
difficult to use correctly. Nonetheless, these functions are
commonly used in security-critical applications together
1. Introduction with the unsafe string operations listed above. For exam-
ple, sendmail version 8.9.3 boasts 285 individual calls to
The C and C++ programming languages and supporting strcpy alone. If these problems are so well known, why
libraries make it extremely easy for programmers to inad- are they still so prevalent?
vertently introduce security vulnerabilities into their code. There are several reasons:
For example, the standard C library defines the gets rou-
tine which takes as a parameter a pointer to a character s. 1. Well known problems are not universally recognized.
gets reads text from the standard input, placing the first Furthermore, even programmers who know about a
character in the location specified by s, and subsequent problem may not focus on the issue when employing a
data consecutively in memory. Reading continues until a questionable routine; many programmers consider se-
newline or end of file character is reached, at which point curity after writing all the code.
the buffer is terminated with a null character. The program-
mer has no way to specify the size of the buffer passed to 2. Programmers often know a particular call introduces
n
gets. As a result, when the buffer is bytes an attacker potential vulnerabilities without understanding the de-
n m
trying to write + bytes into the buffer will always suc- tails about these problems.
ceed if the data excludes newlines.
This example illustrates two significant risks. First, vari- 3. Programmers are often unaware of what corrections
ables adjacent to the buffer in memory may be overwritten. will eliminate a known problem.
If these variables store security-critical data such as an ac-
cess control list, then an attacker can modify the data. The 4. Programmers may hope that hazardous constructs are
second risk is that an attacker could overflow the stack and not exploitable or that no one will discover vulnerabil-
trick the program into running arbitrary code. Stack over- ities (the “security through obscurity” approach).
 Unfortunately, there are few good sources of information and-dirty” approaches should be available to the program-
about writing secure software. Information alleviates prob- mer as early in the development cycle as possible (prefer-
lems through education, but does not completely solve them ably as the programmer types), even if they forego a signif-
unless the developer remains security conscious. icant amount of precision. Our work falls in this space. On
Subtle, lesser known risks still join the obvious problems the other end of the spectrum, compilers (or similar tools)
in creating security vulnerabilites. For example, synchro- should be capable of performing a much higher-assurance
nization issues—race conditions—can often lead to security static security analysis at build time, even if such an analy-
vulnerabilities. The “time-of-check-time-of-use” (TOC- sis is time consuming.
TOU) category of file-based race conditions identified by
Bishop and Dilger [5] serves as an example. Many pro- 2. The problem with grep
grams using temporary, publicly writable files are suscepti-
ble to racing by a malicious process. Problems arise when ITS4 was developed to address the need for a practical,
a process checks information about a file (such as its exis- widely applicable tool to help people identify potentially
tence), then later uses the file assuming the validity of the unsafe constructs in C and C++ code. While we certainly
recently checked information. For example, a setuid text would find such a tool useful in the course of developing
editor might open a temporary file “/tmp/foo” after ver- our own security-critical software, the primary motivation
ifying that it does not already exist. If an attacker links was to save ourselves time when performing security audits
“/etc/passwd” to “/tmp/foo” between the check and of C and C++ source.
open operation, he can modify passwords. Before ITS4, we would use grep at the command line
Few programmers anticipate these attacks, and fewer as one part of a source code audit (as we believe many peo-
still know how to eliminate their causes. For example, sup- ple do). The primary goal was to identify locations at which
posed experts often address TOCTOU vulnerabilities by a program might fall prey to one of the many common se-
creating temporary files with obscure names constructed curity problems. We almost exclusively looked for call sites
from a random number generator such as rand. This so- to standard library functions with known issues. While this
lution is poor since most random number generators yield technique was indeed useful in finding actual vulnerabili-
reproducible output based on a seed value. Choosing a se- ties, it was lacking in several respects:
cure seed is itself a difficult software security problem.
We believe that in an ideal world, programmers should 1. Too much expert knowledge is required. There are
need to know nothing about security; abstractions and tools hundreds of vulnerable system calls, but many rarely
used in programming should diminish the chance that a pro- appear in the wild. It is often impossible for a security
grammer would ever write insecure code. This goal is unre- auditor to remember to check every potential problem
alistic. For example, determining whether “untrusted” data by hand.
is able to affect “trusted” data in general is a complex prob- A good tool lowers the requirement for possessing ex-
lem; current solutions require the programmer to essentially pert knowledge by keeping a database of vulnerabili-
annotate variables with a security policy [14]. Automating ties. This database would include a description of pos-
this task appears unrealistic. sible problems, hints on how to tell if there really is a
The C and C++ languages are unlikely to become inher- problem, and suggested fixes.
ently more secure anytime soon. To make up for this short-
coming, we believe that programming environments should 2. Using grep is too inflexible. It would be useful for
attempt to ease the burden of writing secure software for the the code auditor to be able to sort data intelligently.
end programmer. For example, both editors and program For example, an auditor may wish to look at vulnera-
compilers could be made to examine code for potential se- bilities in order, on a per-file basis, instead of looking
curity violations. This approach is already used to catch at all strcpys followed by all sprintfs, etc. Also,
syntactic errors in many interactive development environ- an auditor might want to look at all buffer overflow
ments problems at once, followed by all TOCTOU problems.
The main advantage for including security checks in ed- Unfortunately, grep alone does not provide this sort
itors over security checks in compilers is that the program- of functionality; a special-purpose tool is necessary.
mer receives more immediate feedback from an editor than More importantly, to help refine results, it would be
a compiler. Every flaw the editing environment catches useful to perform other forms of analysis in addition to
can potentially spare the programmer an additional compile the grep. For example, a heuristic for detecting race
when building and testing a program. conditions [5] may help keep the auditor from having
We see similar parallels in the area of static software vul- to check dozens of calls. Grep does not provide a good
nerability detection. On one end of the spectrum, “quick- framework for such analysis, since it contains no data
 structures representing the program (e.g., there are no Package Counted lines Percent passive
parse trees or token streams). wu-ftpd-2.4 6613 8.65%
net-tools-1.33 8493 9.73%
3. There tend to be too many false positives. Since sshd-1.2.26 21336 15.45%
grep is only performing simple string matching, its sendmail-8.9.3 37124 17.95%
false positive rate can be quite high. When a user has apache-1.3.9 60543 27.54%
to sift through high proportions of false positives, it
is common for the user not to examine individual in- Table 1. Code not compiled into an average
stances closely or at all.1 We call this the “get done, configuration
go home” phenomenon. We postulate that this phe-
nomenon contributes to the fact that significant vulner-
abilities are often missed during manual security au-
dits [17]. large open-source projects, using default configurations for
a Pentium-90 running Redhat 5.0. The tool counts lines of
source and blank lines, but omits comments. We did not
3. Why not more precise analysis?
count lines in packaged third-party software. All prepro-
cessor directives are ignored in our statistics. The results
3.1. Parsing strategy are shown in Table 1.
Even 8.65% of a program is quite a large portion to om-
ITS4 breaks a non-preprocessed file into a series of lex- mit during an analysis; in the testing world, 91.35% state-
ical tokens, and then matches patterns in that stream of to- ment coverage is not considered adequate.
kens. Matching code is added by hand, so non-regular pat- Although we elide per-module data for the sake of
terns can be recognized. When performing more sophis- brevity, we should note that the percentage of passive lines
ticated static analysis, it is generally easier to use a fairly in individual modules can vary greatly. This means that
complete, easy to navigate representation of a program, static analysis tools can fail to analyze mission-critical mod-
such as a parse tree generated with a context-free parser. ules accurately. For example, the net-tools package in-
cludes code to support IPv6. However, if HAVE AFINET6
3.1.1. False negatives is undefined, none of the functionality in the IPv6 portions
of net-tools will be examined by a static analysis tool.
One reason we chose not to use a “real parser” was because Of course, multiple builds can be made, but the analyst
we wanted to have a false negative rate as close to 0 as pos- has to figure out which builds to make, compile each, and
sible. Analysis tools using traditional parsing (such as the run the entire analysis algorithm repeatedly.
lint family of tools) can only analyze a single build of
a program at once, since there is currently no known tech-
nique for parsing C and C++ programs with preprocessor 3.1.2. Interactivity
directives into a single abstract syntax tree. Another reason for not using “real” context-free parsing is
As developers ourselves, we want to check every possi- that we wanted to be able to support interactive program-
ble build of our program. As people who audit the code of ming environments such as Emacs and Microsoft Visual
others, we also want to examine the entire program easily C++ in real time. We would like to see potential security
without having to specify multiple build configurations and problems highlighted by programming environments in the
keep track of uncovered code. same way that incorrectly spelled words are highlighted by
Under the assumption that people seldom analyze more Microsoft Office applications. In other words, as the pro-
than a single build, we examined several large pieces of grammer enters code, the programming environment should
open-source software to see how much source code such recognize the likelihood that a particular piece of code con-
an analysis would miss. We wrote a simplistic preprocessor tains a security problem, and act appropriately.
that counts how many lines of original source (not count- Unfortunately, traditional parsing techniques are not
ing system headers) will be included into an executable (we suitable for meeting this goal because they only work re-
call these active lines), and how many will not be (we call liably on semantically valid programs. Furthermore, highly
these passive lines). This tool is not sophisticated enough to accurate error handling in traditional parsers is notoriously
handle complex conditional expressions, so in those cases, difficult [1]. Traditional parsing considers an entire file as a
we evaluate the conditional expressions by hand and sub- unit, and thus may be inefficient in practice if an individual
stituted a constant expression. We ran this tool on several file must be parsed after every few keystrokes.
1 In fact, if a user doesn’t find a vulnerability fairly quickly, we often Heuristics based on regular languages are known to work
find people claiming that the code is secure without finishing their audit! fairly well in similar situations, even if they are not fully
precise. For example, Emacs uses regular-expression-based buffers could potentially overflow, and by how much.
matching on code in order to perform syntax highlighting, That technique ignores control flow information as
though its inferences about the syntax of an individual token well as context. Their prototype tool can process
are occasionally wrong. Similarly, the Microsoft Office in- sendmail in about 15 minutes on a Pentium III. It is
cremental spelling and grammar checker can fail to parse an believed that a version of the software could be made
English sentence properly. Emacs and Microsoft Office are, to run in significantly less time if the code were better
however, right far more often than they are wrong. Conse- tuned for performance [16]. We anticipate that an anal-
quently, despite their shortcomings, tools such as these can ysis similar to [17] that handled flow and context prop-
be extremely useful in practice. erly would be at least an order of magnitude slower.

3.2. Current limitations of advanced static analysis These problems played a significant role in our decision
for C and C++ to avoid complicated forms of analysis in ITS4. The con-
clusion we drew from our experience with static analysis is
We believe that static analysis of a quality beyond that that it would take several years of solid effort to produce
available in ITS4 can have a tremendous impact on C and a robust, precise, portable, and, most importantly, practical
C++ software security. We identify several problems, how- tool that does an excellent job of statically analyzing source
ever, which make a practical tool involving such technology for security vulnerabilities.
difficult.
4. It’s The Software, Stupid! (Security
1. C’s liberal nature makes the language poorly suited ¯ canner)
¯ ¯ ¯ ¯
for static analysis. The general laxness of the C lan- S
¯
guage (e.g., arbitrary pointer arithmetic and gotos)
makes many types of static analysis intractable in the This section discusses version 1.0.2 of ITS4. The cur-
worst case [11]. In the average case, C’s heavy re- rent version of the tool supports a command-line interface
liance upon pointers makes any sophisticated analysis to the scanning engine and integration with Gnu Emacs.
very difficult.
4.1. Initial scanning and assessment
2. The added complexities of C++ make it very dif-
ficult to analyze. Though recent research on static ITS4 takes one or more C or C++ source files as input,
analysis has made some headway into performing use- breaking each into a stream of tokens. After scanning a
ful analyses on object-oriented languages in general, file ITS4 examines the resultant token stream, comparing
C++ suffers because it is both object-oriented and de- identifiers against a database of “suspects.” This database is
rived from C. Currently, object-oriented analysis tech- discussed in Section 4.2.
niques are cutting-edge research; performing an accu- Checking each identifier is a heuristic that is not com-
rate analysis in an environment with classes, dynamic pletely accurate: security-neutral identifiers may be flagged.
dispatch and templates is a large challenge. The most obvious example is variable names. Consider the
3. Static analysis in a multi-threaded environment is following C code:
difficult. Multi-threaded applications are quite popu- int main()
lar on Windows platforms and are becoming ever-more {
popular on Unix-based systems. Unfortunately, the po- int strcpy;
tential for interaction of data between threads must be return 0;
considered by any analysis tool that wishes to be cor- }
rect.
Obviously, we would like to avoid a false positive in
4. Better static analysis is less efficient. ITS4, which the case of the declaration of strcpy. However, without
performs a very simple analysis (described in Section “real” parsing we cannot accurately determine all identifiers
4), analyzes about 9000 lines of code per second on a that are lexically used as variables. The largest problem is
Pentium-90. For sendmail-8.9.3, it took an aver- that the preprocessor can arbitrarily modify our identifiers.
age of 5.9 seconds of CPU time to scan the entire pack- In the program above, both the int specifier and the vari-
age, and never more than 7.5 seconds of wall-clock able strcpy could be replaced with arbitrary code. We
time (more detailed performance information is given could make a “closed-world” assumption that our scanner
in Section 4.6). gets to examine all code used to build an application. How-
Wagner et. al. [17] present a static analysis technique ever, to handle the general case correctly, we would have to
that uses constraint solving to try to determine which implement a full preprocessor, as the programmer might do
arbitrarily complex things. The problem is made worse in 2. The descriptions and recommendations should be ex-
that the preprocessor can have arbitrarily complex expres- panded.
sions in conditionals where the resulting value of each con-
3. Several additional fields would be desirable, such as a
ditional can change from build to build by passing in flags
detailed code example on how to mitigate the problem.
at the compile line.
Our current solution to this problem is to restrict checks 4. The database currently contains primarily Unix vulner-
to those identifiers followed by a left parenthesis, as pro- abilities; Windows vulnerabilities should be added.
grammers don’t generally pervert the preprocessor in this
way— a simpler analysis almost always suffices for prac- We hope each of these issues can be addressed in the near
tical applications. E.g., programmers don’t generally use future with the help of the community.
strcpy as a variable name. There is an option to flag all The location of the vulnerability database can be spec-
suspect identifiers if the auditor is worried about potential ified at the command line. As a result, it is very easy to
false negatives. use databases that have been modified, such as a trimmed
database that contains only buffer overflow information.
4.2. The vulnerability database The programmer can also specify functions for which ITS4
should check at the command line, even if they are not in the
database.
ITS4 reads a vulnerability database from a text file at
startup, keeping the entire contents resident in memory for
4.3. ITS4 commands
the lifetime of the tool. Vulnerabilities can be added to the
database, removed, and changed with ease.
ITS4 can ignore individual occurrences of a particular
The ITS4 vulnerability database currently contains 131
function. While such a feature can be detrimental (as mis-
calls culled from many sources [4, 5, 8] including the Bug-
use can cause the tool to ignore actual vulnerabilities), it
traq archives [12] and our own personal experience. The
is useful for pruning the output as individual vulnerabilities
largest single class of problems in our database are race
are manually audited and eliminated.
conditions involving file accesses. Functions susceptible to
For example, a developer may add a strcpy to a work-
buffer overflows also account for many entries. Several dif-
in-progress. After running ITS4, they learn about the po-
ferent pseudo-random number routines are flagged because
tential problem and fix it by adding an explicit bounds check
they are often used, albeit incorrectly, to provide entropy
before the call. ITS4 cannot currently perform a sophis-
in security-critical applications. For example, developers
ticated enough analysis to determine that such a check is
may use these functions to shuffle cards or generate crypto-
present. As a result, it will always flag this instance of str-
graphic keys in situations where security is important [3, 9].
cpy. It would be desirable for there to be a way to suppress
For each call, we store the following information:
this error.
 A brief description of the problem. ITS4 commands can ameliorate this problem in two
ways. First, the developer can insert in-place comments
 A high-level description of how to code around the with embedded commands to the scanner. For example,
problem.
strcpy(buf, dst); // ITS4: ignore
 A relative assessment of the severity of the problem, on
will be ignored. The comment usually occurs on the same
the following scale: NO RISK, LOW RISK, MODER-
line as the code it effects. However, if there is no code on
ATE RISK, RISKY, VERY RISKY, MOST RISKY.
the same line, it affects the subsequent line.
 An indication of what type of analysis to perform The case-insensitive text “ITS4:” must appear in the
whenever the function is found in the token stream. comment, followed by an optional list of function calls. The
list may be comma separated. Nothing else besides whites-
 Whether or not the function can retrieve input from paces may appear in the comment. If no calls are specified,
an external source such as a file or socket. ITS4 has ITS4 will ignore any call on the affected line.
a mode that finds all points at which input can come When modifying the source code is not an option, the
in to the program, because we often found ourselves user can keep a list of ITS4 commands in a file, along with
wanting that sort of functionality in our manual audits. the file name and line number to which the command ap-
Unfortunately, the database currently has several limita- plies. The user specifies the location of this file on the com-
tions: mand line.
To allow auditing of code that already has embedded
1. Measures of severity should be refined based on feed- ITS4 commands, the tool provides a command line option
back from the security community. to ignore all commands.
 ITS4 provides other ways to reduce the amount of out- in practice. When performing code inspections with grep
put and to present the output in a more useful way. For ex- we would often note in frustration the things that could eas-
ample, there are several different sorting methods available, ily be ignored with some code that wrapped the command.
and vulnerabilities can be filtered based on severity. The most common examples we saw were strcpys that
only copied a fixed string into a buffer and sprintfs with
4.4. Analysis techniques no string specifier (i.e., no %s) in the format string. ITS4
is able to identify these obvious cases through its handler
When ITS4 first flags a function name, it looks up a mechanism.
“handler” for that function in the vulnerability database. One handler that comes with ITS4 is the “strcpy” han-
The handler is responsible for reporting the problem flagged dler. This handler is currently used not only by strcpy,
by the scanner. If no handler is found in the database, the but also by strcat and strncpy. In each of these func-
default handler is used, which merely adds the problem to tions, the first argument is the target buffer and the second
the results database. Handlers can also be used to perform is the source string. If the source string is a constant, then
more sophisticated analyses on a program. we should reduce the severity of this vulnerability. For ex-
ITS4 performs several tricks in an attempt to reduce the ample, the following call should not be flagged as severe
number of false positives produced by the tool. However, because the second argument is a fixed string:
the notion of “false positive” is slightly fuzzy in this dis-
strcpy(dst, "\n");
cussion because our tool will never throw away information
about a vulnerability. In practice, we expect that users will Our handler has a pointer to the current token, which is
often consider only a percentage of the output, and then only the left parenthesis immediately after the strcpy. If the
the output ranked as most severe. Consider the following C handler finds anything other than a parenthesis, it gives up.
code: Next, it tries to find the second argument, by scanning for-
ward in the token stream, looking for commas at the right
strcpy(buf, "\n");
nesting level. If the first argument consists of nested func-
ITS4 will reduce the severity of the above use of strcpy tion calls, the algorithm will work properly. For example,
from VERY RISKY to the lowest available. Since the scan- ITS4 has no problem with the following:
ner only outputs vulnerabilities of MODERATE RISK or
strcat(a(b("h(i",e(x,y,z))),"the end");
above by default, the end user will never see the warning
generated by the tool unless she specifically asks to see all If a second argument is not found or is not a string, ITS4
warnings. gives up. Otherwise, it matches the pattern, and awards the
In our experience with the tool, we have found that even problem the lowest possible severity level.
the most patient programmers will give up fairly quickly Similar checking is performed for the sprintf fam-
when the severity of all problems is RISKY or below. We ily of functions. First, the format string is found. Then,
believe the RISKY designation is approximately where the the format string is scanned for a percent sign, followed
false positive rate starts to approach 100% rapidly. 2 There- immediately by an ‘s’. If no such pattern is found, ITS4
fore, even in our own security audits, we may only look at assumes that either the format string only contains format-
such items if time permits, depending on the situation. This ting for numbers, or that all strings have a precision speci-
problem is discussed further in Section 6. fication. Either way, the chances of exploit are greatly re-
Currently, there are two types of analysis that ITS4 can duced. More checking could easily distinguish between the
perform to refine the initial assessment it produces. The two possibilities.
first is checking parameters of string constants in argument In both of these cases, we are recognizing patterns that
parameters in unsafe string operations. The second is per- are not regular, due to the parenthesis matching that must be
forming a heuristic check for race conditions, using a mod- performed; grep-style tools cannot recognize a pattern that
ification of an algorithm presented in [5]. allows arbitrary nesting. 3 Since the programmer writing a
Both analyses can be turned off at the command line. handler can make use of the full power of the C++ language,
ITS4 is capable in the general case of performing a more
4.4.1. Sanity checking arguments sophisticated analysis.
These two checks were added as a proof of concept. Sev-
As mentioned in Section 2, grep unfortunately reports eral other checks that would be possible to add (and at least
many hazards that are “obviously” unlikely to be problems somewhat effective) are discussed in Section 4.7.
2 Unfortunately,
measuring accuracy rates is very difficult to do because A comparison of our technique vs. grep and a more
we would have to examine a large number of programs to get significant sophisticated static analysis tool is presented in Section 6.
numbers, and because the manual work involved to obtain such numbers is
enormous. 3 Unless they have context-free extensions.
4.4.2. Race condition analysis ITS4 would not increase the severity. Approaches for im-
proving the “false negatives” in this analysis are discussed
Our analysis also addresses race conditions in file ac- in Section 4.7.
cesses, so-called “Time-Of-Check, Time-Of-Use” (TOC- Note also that there is still plenty of room for false pos-
TOU) problems. As mentioned in Section 1, Bishop and itives. Having two variables with the same name is indis-
Dilger discuss this type of problem extensively [5]. tinguishable from a single variable, as far as our analysis
We scan for these problems in a simple way. First, TOC- is concerned. Also, our approach fails to take control flow
TOU functions are classified based on their handler into into account, and so if the check happens after the use, they
functions that can be checks and functions that can be uses are both promoted in severity, when they should not be.
(several can be both). Every time we see a function, we look Currently, there is no similar tool available that performs
at the identifier that holds the file name. We store a mapping a better static analysis for us to compare ourselves against.
of variables to the list of TOCTOU functions that use that However, in Section 7 we do discuss our tool in relation to
variable. the prototype discussed by Bishop and Dilger [5].
FILE *f;
4.5. Environment integration
int main(){
char *fname = argv[1]; ITS4 is designed so that the front-end to the tool and the
if(!access(fname, W_OK)){ back-end for the tool are both easily removed. We did this
f = fopen(fname, "w+"); because we hope to see ITS4 integrated into popular pro-
} else{ gramming environments, such as Microsoft’s Visual Studio.
// Do error handling. In such an environment, code should be analyzed in the
} background while the user types. The current line can be
// Write stdin to f then exit. scanned continually and the entire file can be scanned fre-
} quently to see if there are any new constructs to flag. When
such a construct is identified, it should be highlighted.
In the example above, our mapping would contain a sin- Mousing over the problem could give a detailed description
gle key “fname” which would have an array of two elements of the issues, and so on.
as a value. The array’s values would be the instance of ac- ITS4 commands would be a poor user interface for such
cess on line 5 and the instance of fopen on line 6. The an environment. For example, Microsoft Office allows the
mapping has a lifetime beyond that of the handler. user to right-click on a misspelled word to ignore it; a much
At this point, scanning continues. After scanning all to- better user interface, in our opinion.
kens, ITS4 calls the handler module to perform any final Currently, the only environment with which we have in-
analysis of the data before reporting the results. We iter- tegrated ITS4 is Gnu Emacs. The user can either run the
ate over our mapping. For any keys where there is at least scan all at once, much like one would compile a program
one check on a variable and one use, we combine the nota- from within Emacs. Alternatively, we have bindings avail-
tions into a single result, which is reported with an increased able that will scan the current file every time the user hits en-
severity. ter or moves off the current line. Problems are highlighted
This strategy works well, but there are currently several and output from the scanner is placed in another buffer. This
limitations that result in ITS4 failing to promote the sever- integration is only a prototype and is still fairly inefficient.
ity of conditions that should probably be reported. For The biggest problem is that we invoke the ITS4 command
example, we do not address the aliasing problem. If we every time. It would be easy to add a new front-end to the
change the above code to: scanner that enables it to be a persistent server communi-
cating with Emacs, which would make it far more usable.
FILE *f;
4.6. Performance
int main(){
char *f1 = argv[1]; We performed preliminary tests on the performance of
char *f2 = f1; ITS4. We measured performance on a Pentium-90 with
if(!access(f1, W_OK)){ 32M of RAM running Redhat 5.0. Generally, the machine
f = fopen(f2, "w+"); is 93.7% idle, with under 2M of real memory free. We mea-
} sured the sum of user and system time using the time com-
... mand. In this environment, we ran our scanner ten times on
} each of the tools mentioned in Section 3.1.1.
 Computed over all 50 runs, ITS4 scans an average of job. Such work should be integrated with any sort of
about 8800 lines per second, with a standard deviation of alias analysis performed.
approximately 800. During testing we noticed that adding
analyses such as TOCTOU scanning did not impact the run  Approximate flow information. Even our proposed
heuristic static analysis techniques could be improved
time of our tool whatsoever, suggesting that our tool is cur-
rently bound by I/O rather than analysis. in accuracy if we can extract a reasonable model of the
program’s control flow from the data stream alone.
4.7. Future Directions for ITS4
5. Practical experience with ITS4
There are several practical improvements that can easily
be made to ITS4. Among them: To date, we have applied ITS4 as a tool to assist in our
auditing of two large pieces of software. The first was I-Pay,
 Integrate with new programming environments. a reference version of an electronic payment system used
We discuss this option in Section 4.5. by many Dutch banks. Our tool helped us find a definitive
break in one of the network applications that comes with
 Downgrade buffer overflow severity if the destina- this package. The second was Jitterbug, a web-based bug
tion is not stack allocated. Overflows of dynamically tracking system, which has been extensively audited for se-
allocated and static memory are generally more dif- curity in the past [15]. ITS4 helped find a small number of
ficult to exploit than are overflows of stack allocated exploitable flaws, though they are unlikely to affect many
memory. ITS4 can look for patterns that look like ar- users of the software.
ray declarations. For each such pattern, ITS4 can ac- We have some initial conclusions based on our experi-
tually parse the declaration to determine whether it is ences using ITS4:
stack allocated. If not, the variable may be an alias for
a stack allocated buffer. Therefore, the scanner would 1. ITS4 still requires a significant level of expert
also need to check for allocation statements and static knowledge. While our tool does encode a vast amount
declarations before it could rule out stack allocation. of knowledge on vulnerabilities that the developer no
With our general philosophy of conservatism, items longer needs to keep in his head, we have found that
would not be downgraded unless such an allocation an expert still does a much better job than a novice at
could be found. 4 taking a potential vulnerability location and manually
performing the static analysis necessary to determine
 Perform alias analysis. More accurate TOCTOU whether an exploit is possible. We find experts tend
scanning can be performed if we obtain pointer alias- to be far more efficient and far more accurate at this
ing information with any degree of accuracy, even if it process.
is not fully precise. One way to go about this is to scan
2. Even for experts, analysis is still time-consuming.
through all tokens, looking for assignments and func-
While we have not used the tool enough to give more
tion calls, noting any aliases we see. Then, aliases can
than anecdotal evidence, we would say that the tool
be considered in a flow-insensitive, context-insensitive
only eliminates one quarter to one third of the time it
light. Since we will ignore the lack of flow informa-
takes to perform such an analysis because the manual
tion and other contextual clues, we certainly will not
analysis is so time consuming.
be capable of a precise analysis. The results should
be much better than no such analysis, however, assum- 3. Every little bit helps. We feel that ITS4 helps sig-
ing that it is uncommon for such an approach to decide nificantly with fighting the “get done, go home” effect.
something not helpful, such as “all variables can alias We noticed that in the case where ITS4 prioritizes one
all variables.” instance of a function call over another, we tend to be
more careful about analysis of the more severe prob-
 Perform range analysis. The biggest hurdle to ITS4 lem.
performing the sort of static analysis presented by
Wagner [17] and briefly described in Section 3.2 is that 4. It can help find real bugs. Using ITS4, we have
the constraint generation step is difficult given our ap- found security problems in two real applications. In
proach to parsing the input. While we would have a both cases, we found problems in the first 10 minutes
very difficult time generating the same constraint sets of analysis that we would not have otherwise found as
as they do, a heuristic parse could potentially do a good quickly.
4 We rarely see references to heap allocated memory later being used to Note that although we ran our tool on several large ap-
alias the stack, so we feel comfortable downgrading this type of situation. plications such as sendmail and apache, we did
 not hand-audit those tools. We only spent enough time that Jitterbug has previously been extensively analyzed.
with them to gather data for purposes such as timing ITS4 immeadiately flagged five popen calls. Three of
tests and comparative analyses with other tools. these calls take input from the web. One sufficiently scru-
tinizes its arguments; we were unable to exploit it. We
5.1. I-Pay developed exploits to leverage the other two popen calls.
These vulnerabilities are only exploitable if one of two un-
We used ITS4 to audit the source code for I-Pay, “the documented features are enabled (by default, they are not).
Internet payment infrastructure for the combined Dutch Therefore, very few fielded implementations are affected by
banks”[10]. We were most interested in remote exploits, this vulnerability. Apparently, the features were added for
since the I-Pay software utilities typically run on organiza- a single high-profile user who no longer uses the software,
tional web servers and other protected machines. and, in light of the vulnerabilities found, they will be re-
Our first step was to use ITS4 in locating all sites where moved in the next version of the software [15].
network or file data was read. ITS4 flagged a single call In addition, ITS4’s race condition analysis identified
to recv. We saw that this call was made from a function five potential TOCTOU vulnerabilities. An audit using
called netread. We asked ITS4 to find netread, and grep reported nearly 80 different call sites where the called
nothing else. There were several instances found, but we function could be involved in a TOCTOU condition. With-
followed the first, which was made from a function called out ITS4, we would have manually examined each of the
multiread. We asked ITS4 to find uses of this function. 80 calls in the context of the entire program; instead we
It found us one, in a function called saferead, which was considered just five locations.
itself used only three times. The first of these three calls
Thus ITS4 greatly reduces time spent examining code is
turned out to be a major vulnerability.
therefore expected to be large. However, our analysis does
I-Pay includes a utility called checkkey used after in-
not handle aliasing.
stallation to check the firewall settings of the host machine
and confirm that the Triple-DES library included with I-
Pay is correctly configured for encryption and decryption.
When checkkey executes, it constructs a text message, 6. Comparing ITS4 to other solutions
which is encrypted and sent to a server specified in a config-
uration file. The checkkey program waits for a response
from this server, decrypts the response upon receipt, and 6.1. grep
displays it along with status information. Unfortunately,
the buffer which receives the response message is a stack-
allocated 256 byte buffer, while the reading function will In this subsection, we compare grep to ITS4, each us-
accept up to 32766 bytes. This programming mistake will ing a database that only scans the 13 functions for which
allow a malicious server, or a machine masquerading as the there are buffer overflow handlers. We limit the scope of
server, to introduce and execute arbitrary code on the client our comparison in this way so that we can compare the per-
machine. An hour of subsequent experimentation yielded a formance of the handlers. Relative severities are ignored;
remote exploit. either the tool reported a problem, or it did not. In the case
The other potential problems identified by ITS4 were of ITS4 with analysis, if the analysis downgraded a prob-
less serious. Several calls to strcpy and sprintf were lem to the lowest possible setting, we considered that a fail-
flagged as risky, but were deemed harmless upon inspec- ure to report.
tion. We did locate three other buffer overflow vulnerabili- Table 3 shows the number of vulnerabilities found by
ties using the tool, but they require local access. As long as grep, ITS4 with analysis turned off, and ITS4 with anal-
the I-Pay utilities run with low privileges on non-interactive ysis on. The next to last column of this table shows the per-
machines, these flaws are likely have little or no impact. cent reduction of results reported compared to grep when
Finally, I-Pay employs a rudimentary temporary file name smarter parsing is applied (i.e., lexing instead of grep).
selection algorithm susceptible to race conditions. The last column shows the percent reduction of results re-
ported that are due to our analysis. Note that, except in the
5.2. Jitterbug case of apache, which is a vast outlier, our analysis seems
slightly more effective than better parsing.
Jitterbug is free C software for tracking bug reports over We believe that ITS4 users can expect results around
the web. We were interested in auditing Jitterbug because 25% better than grep, perhaps more. This number will
we use it, and we are skeptical of any C code we run, espe- probably vary widely by application and may also vary
cially if it has network access. We learned after our analysis based on the programming style of the developer.
 Package grep ITS4 ITS4 Lex Anl. comparison, there are some functions our tool consid-
-anl. red.(%) red.(%) ers that it will never report because their risk classifi-
wu-ftpd 146 138 112 5.5 17.8 cations are too low. The assumption is that such calls
net-tools 160 142 103 11.3 24.4 are very unlikely to show up in their analysis.
sshd 265 238 206 10.2 12.1
4. We assume that the vulnerabilities a particular tool will
sendmail 480 418 342 12.9 15.8
flag are uniformly distributed throughout the source
apache 623 168 113 73.0 8.8
code.
Table 2. Effectiveness of grep compared to Their analysis of sendmail yielded 44 “probable” vul-
ITS4, without and with analysis. nerabilities. Our analysis yielded 79. Adjusting their num-
ber for the 17.95% of the code they missed based on our
uniform distribution assumption, their modified number of
Package grep ITS4 Reduction (%) vulnerabilities for the sake of comparison would be 53.6.
wu-ftpd 146 112 23.3 With this set of simplified assumptions, their results give a
net-tools 160 103 35.6 32.15% reduction in false positives. In practice, we would
sshd 265 206 22.6 expect to see results from their tool that give up to a 50%
sendmail 480 342 28.8 reduction.
apache 623 113 81.9

Table 3. Total reduction compared to grep. 7. Related work

Regular “lint” tools such as LCLint [7] perform similar

functions, but in the context of general robustness; security
6.2. Buffer overflow detection via range analysis features generally are not included. Also, such tools tend to
work on a per-build basis, and use context-free parsing.
The only other tool we are aware of that we can compare Security experts have long proposed building simple
our work to is presented by Wagner et. al. in [17]. Unfortu- scanners that operate on source code, looking for simple
nately, this comparison proves quite difficult: patterns that can potentially be exploited. To date, we
1. Their work is not limited to picking out function calls know of three limited prototypes of such systems (other
as ours currently is. Therefore, they may flag some than ours), all of which process C, and possibly C++.
problems that we do not. The first is slint [13], a general-purpose security scan-
ner developed by Mudge, formerly of the l0pht. While
2. Their work fails to analyze approximately 18% of the there is a public web page for this product, no technical in-
program that ITS4 does not fail to analyze. (See Sec- formation is public.
tion 3.11) The second is the Bishop and Dilger race condition scan-
3. Their output is based on different metrics than ours. ner. In [5], they detail a fairly accurate static analysis for
While theirs is based solely on the results of their anal- TOCTOU problems. Their prototype is similar in function-
ysis, ours is largely based on human experience, with ality and power to our race condition scanning. For exam-
only a small analysis component. ple, it uses regular expressions for token recognition instead
of context-free parsing.
Nonetheless, we make some simplifying assumptions in an The primary difference between the two tools is that the
attempt to compare how these tools would compare “in Bishop and Dilger scanner considers variable names on a
practice”: per-function basis, whereas ITS4 does not. If two func-
tions each have a variable with the same name, ITS4 will
1. Since we do not know the configuration used to test
treat all variables with the same name as the same variable,
sendmail, we made the assumption that it was tested
even if across separate files. We believe the ITS4 behav-
under the default configuration.
ior to be slightly more useful because most programmers
2. We assume that our tool will report everything their name parameters and local variables consistently across
tool reports, and probably more. functions. For example, consider the following code:

3. They present results for how many “probable” results void do_it(char *fname) {
their tool gives. We assume that reporting our “very FILE *f = fopen(fname, "w");
risky” and “most risky” classifications has the same se- }
mantic meaning. This means that, for the sake of our int main(int argc, char **argv) {
 char *fname = argv[1]; [5] M. Bishop and M. Dilger. Checking for race condi-
if(access(fname, W_OK)) tions in file accesses. Computing Systems, 9(2):131–
do_it(fname); 152, Spring 1996.
}
[6] C. Cowan et. al. Stackguard: Automatic adaptive de-
The Bishop and Dilger scanner will miss the above race tection and prevention of buffer-overflow attacks. In
condition because it does not support interprocedural anal- Proceedings of the Seventh USENIX Security Sympo-
ysis. sium, pages 63–77, San Antonio, TX, 1998.
Although the Bishop and Dilger’s tool has never been
distributed, a third party reimplementation has recently be- [7] D. Evans, J. Guttag, J. Horning, and Y. Meng Tan.
come available [2]. Lclint: A tool for using specifications to check code.
The only other tool we know about that statically scans In In proceedings of the SIGSOFT Symposium on
for security vulnerabilities is presented in [17]. We dis- the Foundations of Software Engineering, December
cussed this tool (primarily in Sections 3 and 6), as well as 1994.
its relative advantages and disadvantages compared to the [8] S. Garfinkel and G. Spafford. Practical Unix and In-
ITS4 approach. ternet Security. O’Reilly and Associates, Inc., 1996.
Other forms of static analysis are possible. For example,
we discussed locating the places in the code where input [9] I. Goldberg and D. Wagner. Randomness and the
to the program is possible. From there, the usual goal is netscape browser: How secure is the world wide web?
to follow program flow to see what damage untrusted in- Communications of the ACM, January 1996.
put can do. Static language support for such an analysis
is now available for a subset of the Java programming lan- [10] InterPay. I-pay product web site. http://www.
ipay.com.
guage [14].
[11] W. Landi and B. Ryder. A safe approximation al-
8. Conclusion gorithm for interprocedural pointer aliasing. In Pro-
ceedings of Programming Language Design and Im-
We have presented ITS4, a static analysis tool for C and plementation, 1992.
C++. While its parsing model makes it poorly suited for [12] E. Levy. The Bugtraq mailing list. http://www.
highly accurate static analysis, the same model makes the securityfocus.com.
tool very practical for real-world use; even with some fa-
cility for a heuristic-driven static analysis of the program, [13] mudge. The slint web page. http://www.l0pht.
ITS4 can scan large programs efficiently, while still achiev- com/slint.html.
ing adequate results. The tool is also appropriate for inte-
gration into programming environments with little modifi- [14] A. Myers. Practical mostly-static information flow
cation. control. In Proceedings of ACM SIGPLAN-SIGACT
Symposium on Principles of Programming Lan-
guages, San Antonio, TX, January 1999.
References [15] A. Tridgell. Personal Communication.

[1] A. Aho, R. Sethi, and J. Ullman. Compilers: Princi- [16] D. Wagner. Personal Communication.
ples, Techniques and Tools. Addison Wesley, 1986.
[17] D. Wagner, J. Foster, E. Brewer, and A. Aiken. A
[2] Antonomasia. scancode.plx. http://www. first step towards automated detection of buffer over-
notatla.demon.co.uk/SOFTWARE. run vulnerabilities. In Proceedings of the Year 2000
Network and Distributed System Security Symposium
[3] B. Arkin, F. Hill, S. Marks, M. Schmidt, T. Walls, and (NDSS), pages 3–17, San Diego, CA, 2000.
G. McGraw. How we learned to cheat at online poker:
A study in software security. The developer.com Jour-
nal, September 1999. http://www.developer.
com/journal.

[4] M. Bishop. Writing safe setuid programs,

1998. http://seclab.cs.ucdavis.edu/
˜bishop/secprog.html.