College of Computers and information technology

Department of Computer Science

Secure Software Engineering

Course Coordinator
Associate Prof. Dr. M.Tabrez Quasim
[email protected]
 Chapter 1

Why Is Security a Software Issue?

1.1 Introduction
Software is everywhere. It runs your car. It controls your cell phone. It’s how you access your
bank’s financial services; how you receive electricity, water, and natural gas; and how you fly
from coast to coast [McGraw 2006]. Whether we recognize it or not, we all rely on complex,
interconnected, software-intensive information systems that use the Internet as their means for
communicating and transporting information.
Building, deploying, operating, and using software that has not been developed with security in
mind can be high risk—like walking a high wire without a net (Figure 1–1). The degree of risk can
be compared to the distance you can fall and the potential impact (no pun intended).
This chapter discusses why security is increasingly a software problem. It defines the dimensions of
software assurance and software security. It identifies threats that target most software and the
shortcomings of the software development process that can render software vulnerable to those
threats. It closes by introducing some pragmatic solutions that are expanded in the chapters to
follow. This entire chapter is relevant for executives (E), project managers (M), and technical leaders
(L).
 WHY IS SECURITY A SOf TwARE ISSUE?

Figure 1–1: Developing software without security in mind is like walking a high wire without a net
1.2 The Problem
Organizations increasingly store, process, and transmit their most sensitive information
using software-intensive systems that are directly connected to the Internet. Private
citizens’ financial transactions are exposed via the Internet by software used to shop,
bank, pay taxes, buy insurance, invest, register children for school, and join various
organizations and social networks. The increased exposure that comes with global
connectivity has made sensitive information and the software systems that handle it
more vulnerable to unintentional and unauthorized use. In short, software-intensive
systems and other software-enabled capabilities have provided more open, widespread
access to sensitive information—including personal identities—than ever before.
 WHY IS SECURITY A SOf TwARE ISSUE?
9000
8064
8000
Total vulnerabilities reported
(1997–2006): 30,264
7000

5990
6000

5000
4129
4000 3784 3780

3000
2437
2000

1090
1000
262 417
311
0
1997 1998 1999 2000 2001 2002 2003 2004 2005 2006

Figure 1–2: Vulnerabilities reported to CERT

In Deloitte’s 2007 Global Security Survey, 87 percent of survey respon- dents cited poor software
development quality as a top threat in the next 12 months. “Application security means ensuring that there
is secure code, integrated at the development stage, to prevent potential vulnerabilities and that steps such
as vulnerability testing, application scanning, and penetration testing are part of an organization’s soft-
ware development life cycle [SDLC]” [Deloitte 2007].
The growing Internet connectivity of computers and networks and the corresponding user dependence on
network-enabled services (such as email and Web-based transactions) have increased the number and
sophistication of attack methods, as well as the ease with which an attack can be launched. This trend
puts software at greater risk. Another risk area affecting software security is the degree to which systems
accept updates and extensions for evolving capabilities. Extensible systems are attractive because they
provide for the addi- tion of new features and services, but each new extension adds new capabilities, new
interfaces, and thus new risks. A final software security risk area is the unbridled growth in the size and
complexity of software systems (such as the Microsoft Windows operating system). The unfortunate reality
is that in general more lines of code produce more bugs and vulnerabilities [McGraw 2006].
 1.2 THE PROBLEM 5

1.2.1 System Complexity: The Context within Which Software Lives

Building a trustworthy software system can no longer be predicated on constructing and assembling
discrete, isolated pieces that address static requirements within planned cost and schedule. With the
expanding scope and scale of systems, project managers need to reconsider a number of development
assumptions that are generally applied to software security:
• Instead of centralized control, which was the norm for large stand-alone systems, project managers
have to consider multiple and often independent control points for systems and systems of systems.

• Increased integration among systems has reduced the capability to make wide-scale changes quickly. In
addition, for independently managed systems, upgrades are not necessarily synchronized. Project
managers need to maintain operational capabilities with appropriate security as services are upgraded
and new services are added.

• With the integration among independently developed and oper- ated systems, project managers have to
contend with a heteroge- neous collection of components, multiple implementations of common
interfaces, and inconsistencies among security policies.

• With the mismatches and errors introduced by independently developed and managed systems,
failure in some form is more likely to be the norm than the exception and so further complicates meeting
security requirements.

There are no known solutions for ensuring a specified level or degree of software security for complex
systems and systems of systems, assuming these could even be defined.
 WHY IS SECURITY A SOf TwARE ISSUE?
1.3 Software Assurance and Software Security

Software assurance has become critical because dramatic increases in business and mission
risks are now known to be attributable to exploitable software [DHS 2003]. The growing
extent of the resulting risk exposure is rarely understood, as evidenced by these facts:
•Software is the weakest link in the successful execution of interdependent systems and
software applications.
•Software size and complexity obscure intent and preclude exhaustive testing.
•Outsourcing and the use of unvetted software supply-chain components increase risk
exposure.
•The sophistication and increasingly more stealthy nature of attacks facilitates exploitation.
•Reuse of legacy software with other applications introduces unintended consequences,
increasing the number of vulnerable targets.
•Business leaders are unwilling to make risk-appropriate investments in software security.
According to the U.S. Committee on National Security Systems’ “National Information
Assurance (IA) Glossary” [CNSS 2006], software assurance is the level of confidence that
software is free from vulnerabilities, either intentionally designed into the software or
accidentally inserted at any time during its life cycle, and that the software functions in the
intended manner.
 1.3 SOf TwARE ASSURANCE ANd SOf TwARE SECURITY 7

Software security matters because so many critical functions are completely dependent on software. This
makes the software a very high-value target for attackers, whose motives may be malicious, criminal,
adversarial, competitive, or terrorist in nature. What makes it so easy for attackers to target software is
the virtually guaranteed presence of known vulnerabilities with known attack methods, which can be
exploited to violate one or more of the software’s security properties or to force the software into an
insecure state. Secure software remains dependable (i.e., correct and predictable) despite intentional
efforts to compromise that dependability.
The objective of software security is to field software-based systems that satisfy the following criteria:
• The system is as vulnerability and defect-free as possible.
• The system limits the damage resulting from any failures caused by attack-triggered faults, ensuring
that the effects of any attack are not propagated, and it recovers as quickly as possible from those
failures.
• The system continues operating correctly in the presence of most attacks by either resisting the
exploitation of weaknesses in the software by the attacker or tolerating the failures that result from
such exploits.

2. Software reliability means the probability of failure-free (or otherwise satisfactory) software operation
for a specified/expected period/interval of time, or for a specified/expected number of operations, in a
specified/expected environment under specified/expected operating condi- tions. Sources for this
definition can be found in [Goertzel 2006], appendix A.1.
3. Software safety means the persistence of dependability in the face of accidents or mishaps— that is,
unplanned events that result in death, injury, illness, damage to or loss of property, or environmental
harm. Sources for this definition can be found in [Goertzel 2006]
 WHY IS SECURITY A SOf TwARE ISSUE?

Software that has been developed with security in mind generally reflects the following
properties throughout its development life cycle:

•Predictable execution. There is justifiable confidence that the software, when executed,
functions as intended. The ability of malicious input to alter the execution or outcome in
a way favorable to the attacker is significantly reduced or eliminated.
•Trustworthiness. The number of exploitable vulnerabilities is intentionally minimized to
the greatest extent possible. The goal is no exploitable vulnerabilities.
•Conformance. Planned, systematic, and multidisciplinary activities ensure that software
components, products, and systems conform to requirements and applicable standards
and procedures for specified uses.
 1.4 THREATS TO SOf TwARE SECURITY 9

1.3.1 The Role of Processes and Practices in Software Security

A number of factors influence how likely software is to be secure. For instance, software vulnerabilities
can originate in the processes and practices used in its creation. These sources include the decisions
made by software engineers, the flaws they introduce in specification and design, and the faults and
other defects they include in developed code, inadvertently or intentionally.
Other factors may include the choice of programming languages and development tools used to
develop the software, and the configuration and behavior of software components in their
development and operational environments. It is increasingly observed, however, that the most critical
difference between secure software and insecure software lies like the processes and practices used to specify,
design, and develop the software [Goertzel 2006].
The return on investment when security analysis and secure engineering practices are introduced early
in the development cycle ranges from 12 percent to 21 percent, with the highest rate of return occurring
when the analysis is performed during application design [Berinato 2002; Soo Hoo 2001]. This return
on investment occurs because there are fewer security defects in the released product and hence
reduced labor costs for fixing defects that are discovered later.
A project that adopts a security-enhanced software development process is adopting a set of practices
(such as those described in this book’s chap- ters) that initially should reduce the number of
exploitable faults and weaknesses. Over time, as these practices become more codified, they should
decrease the likelihood that such vulnerabilities are introduced into the software in the first place. More
and more, research results and real-world experiences indicate that correcting potential vulnerabilities as
early as possible in the software development life cycle, mainly through the adop- tion of security-enhanced processes
and practices, is far more cost-effective than the currently pervasive approach of developing and releasing
frequent patches to operational software [Goertzel 2006].
 WHY IS SECURITY A SOf TwARE ISSUE?

1.4 Threats to Software Security

In information security, the threat—the source of danger—is often a person intending to do
harm, using one or more malicious software agents. Software is subject to two general
categories of threats:
•Threats during development (mainly insider threats). A software engineer can sabotage the
software at any point in its development life cycle through intentional exclusions from,
inclusions in, or modifications of the requirements specification, the threat models, the
design documents, the source code, the assembly and integration framework, the test cases
and test results, or the installation and configuration instructions and tools. The secure
development practices described in this book are, in part, designed to help reduce the
exposure of software to insider threats during its devel- opment process. For more
information on this aspect, see “Insider Threats in the SDLC” [Cappelli 2006].

•Threats during operation (both insider and external threats). Any software system that runs
on a network-connected platform is likely to have its vulnerabilities exposed to attackers
during its operation. Attacks may take advantage of publicly known but unpatched
vulnerabilities, leading to memory corruption, execution of arbitrary exploit scripts, remote
code execution, and buffer overflows. Software flaws can be exploited to install spyware,
adware, and other malware on users’ systems that can lie dormant until it is triggered to
execute.4
1.5 Sources of Software Insecurity

Most commercial and open-source applications, middleware systems, and

operating systems are extremely large and complex. In normal execution, these
systems can transition through a vast number of different states. These
characteristics make it particularly difficult to develop and operate software that
is consistently correct, let alone consistently secure. The unavoidable presence of
security threats and risks means that project managers and software engineers
need to pay attention to software security even if explicit requirements for it have
not been captured in the software’s specification.

A large percentage of security weaknesses in software could be avoided if

project managers and software engineers were routinely trained in how to address
those weaknesses systematically and consistently. Unfortunately, these personnel
are seldom taught how to design and develop secure applications and conduct
quality assurance
 WHY IS SECURITY A SOf TwARE ISSUE?
Software—especially networked, application-level software—is most often compromised by
exploiting weaknesses that result from the fol- lowing sources:
• Complexities, inadequacies, and/or changes in the software’s processing model (e.g., a Web- or
service-oriented architecture model).
• Incorrect assumptions by the engineer, including assumptions about the capabilities, outputs, and
behavioral states of the software’s execution environment or about expected inputs from external
entities (users, software processes).
• Flawed specification or design or defective implementation of
– The software’s interfaces with external entities. Development mistakes of this type include
inadequate (or nonexistent) input validation, error handling, and exception handling.
– The components of the software’s execution environment (from middleware-level and
operating-system-level to firm- ware- and hardware-level components).
• Unintended interactions between software components, including those provided by a third party.

Mistakes are unavoidable. Even if they are avoided during requirements engineering and design
(e.g., through the use of formal methods) and development (e.g., through comprehensive code
reviews and extensive testing), vulnerabilities may still be introduced into software during its
assembly, integration, deployment, and operation. No matter how faithfully a security-enhanced life
cycle is followed, as long as software continues to grow in size and complexity, some number of
exploitable faults and other weaknesses are sure to exist.
1.6 The Benefits of Detecting Software Security Defects Early
Limited data is available that discusses the return on investment (ROI) of reducing
security flaws in source code (refer to Section 1.6.1 for more on this subject).
Nevertheless, a number of studies have shown that significant cost benefits are
realized through improvements to reduce software defects (including security
flaws) throughout the SDLC [Goldenson 2003]. The general software quality case is
made in this section, including reasonable arguments for extending this case to
include software security defects.
Proactively tackling software security is often under-budgeted and dismissed as a
luxury. In an attempt to shorten development schedules or decrease costs, software
project managers often reduce the time spent on secure software practices during
requirements analysis and design. In addition, they often try to compress the testing
schedule or reduce the level of effort. Skimping on software quality6 is one of the
worst decisions an organization that wants to maximize development speed can
make; higher quality (in the form of lower defect rates) and reduced development
time go hand in hand. Figure 1–3 illustrates the relationship between defect rate and
development time.
Projects that achieve lower defect rates typically have shorter schedules. But only
some organizations currently develop software with defect levels that result in
longer schedules than necessary.
 WHY IS SECURITY A SOf TwARE ISSUE?

Most organizations
are somewhere
around this point

Development
Time Fastest schedule
(“best” schedule)

~ 95% 100%
Percentage of Defects Removed Before Release
Figure 1–3: Relationship between defect rate and development time

In the 1970s, studies performed by IBM demonstrated that software products with lower defect counts
also had shorter development schedules [Jones 1991]. After surveying more than 4000 software projects,
Capers Jones [1994] reported that poor quality was one of the most common reasons for schedule
overruns. He also reported that poor quality was a significant factor in approximately 50 percent of all
canceled projects. A Software Engineering Institute survey found that more than 60 percent of
organizations assessed suffered from inadequate quality assurance [Kitson 1993]. On the curve in Figure 1–
3, the orga- nizations that experienced higher numbers of defects are to the left of the “95 percent defect
removal” line.
The “95 percent defect removal” line is significant because that level of prerelease defect removal appears to
be the point at which projects achieve the shortest schedules for the least effort and with the highest levels
of user satisfaction [Jones 1991]. If more than 5 percent of defects are found after a product has been
released, then the product is vul- nerable to the problems associated with low quality, and the organiza- tion
takes longer to develop its software than necessary. Projects that are completed with undue haste are
particularly vulnerable to short- changing quality assurance at the individual developer level. Any
developer who has been pushed to satisfy a specific deadline or ship a product quickly knows how much
pressure there can be to cut corners because “we’re only three weeks from the deadline.
 1.6 THE BENEf ITS Of D ETECTINg SOf TwARE SECURITY DEf ECTS EARLY 15

One aspect of quality assurance that is particularly relevant during rapid development is the presence of error-
prone modules—that is, modules that are responsible for a disproportionate number of defects. Barry Boehm
reported that 20 percent of the modules in a program are typically responsible for 80 percent of the errors [Boehm
1987]. On its IMS project, IBM found that 57 percent of the errors occurred in 7 per- cent of the modules [Jones
1991]. Modules with such high defect rates are more expensive and time-consuming to deliver than less error-
prone modules. Normal modules cost about $500 to $1000 per function point to develop, whereas error-prone
modules cost about $2000 to $4000 per function point to develop [Jones 1994]. Error-prone modules tend to be
more complex, less structured, and significantly larger than other modules. They often are developed under
excessive schedule pressure and are not fully tested. If development speed is important, then identification and
redesign of error-prone modules should be a high priority.
If an organization can prevent defects or detect and remove them early, it can realize significant cost and
schedule benefits. Studies have found that reworking defective requirements, design, and code typically
accounts for 40 to 50 percent of the total cost of software development [Jones 1986b]. As a rule of thumb, every
hour an organization spends on defect prevention reduces repair time for a system in production by three to ten
hours. In the worst case, reworking a software requirements problem once the software is in operation typically
costs 50 to 200 times what it would take to rework the same problem during the requirements phase [Boehm
1988]. It is easy to understand why this phenomenon occurs. For example, a one-sentence requirement could
expand into 5 pages of design diagrams, then into 500 lines of code, then into 15 pages of user documentation
and a few dozen test cases. It is cheaper to correct an error in that one-sentence requirement at the time
requirements are specified (assuming the error can be iden- tified and corrected) than it is after design, code, user
documentation, and test cases have been written. Figure 1–4 illustrates that the longer defects persist, the more
expensive they are to correct.
The savings potential from early defect detection is significant: Approximately 60 percent of all defects usually exist by design time
16 CHApTER 1 WHY IS SECURITY A SOf TwARE ISSUE?

Cost to
Correct

Phase in Which a
Defect Is Created

Requirements
Engineering

Architecture

Design

Implementation
and Testing

Requirements Architecture Design Implementation Deployment

Engineering and Testing and Operations
Phase in Which a Defect Is Corrected
Figure 1–4: Cost of correcting defects by life-cycle phase

[A decision early in a project to

exclude defect detection amounts to a
decision to postpone defect detection
and correction until later in the project,
when defects become much more
expensive and time-consuming to
address. That is not a rational decision
when time and development dollars
are at a premium. According to
software qual- ity assurance empirical
research, $1 required to resolve an
issue during the design phase grows
into $60 to $100 required to resolve the
same issue after the application has
shipped [Soo Hoo 2001].
1.6 THE BENEf ITS Of D ETECTINg SOf TwARE SECURITY DEf ECTS EARLY 17

1.6.1 Making the Business Case for Software Security: Current State7
As software project managers and developers, we know that when we want to introduce new approaches in our
development processes, we have to make a cost–benefit argument to executive management to convince them that
this move offers a business or strategic return on investment. Executives are not interested in investing in new
technical approaches simply because they are innovative or exciting. For profit- making organizations, we need to
make a case that demonstrates we will improve market share, profit, or other business elements. For other types
of organizations, we need to show that we will improve our software in a way that is important—in a way that
adds to the organization’s prestige, that ensures the safety of troops in the battle- field, and so on.
In the area of software security, we have started to see some evidence of successful ROI or economic arguments for
security administrative operations, such as maintaining current levels of patches, establishing organizational
entities such as computer security incident response teams (CSIRTs) to support security investment, and so on
[Blum 2006, Gordon 2006, Huang 2006, Nagaratnam 2005]. In their article “Tangi- ble ROI through Secure Software
Engineering,” Kevin Soo Hoo and his colleagues at @stake state the following:

Findings indicate that significant cost savings and other advantages are achieved when security analysis
and secure engineering practices are introduced early in the develop- ment cycle. The return on
investment ranges from 12 percent to 21 percent, with the highest rate of return occurring when analysis is
performed during application design.
Since nearly three-quarters of security-related defects are design issues that could be resolved
inexpensively during the early stages, a significant opportunity for cost savings exists when secure
software engineering principles are applied during design.

However, except for a few studies [Berinato 2002; Soo Hoo 2001], we have seen little evidence presented to support
the idea that investment during software development in software security will result in com- mensurate benefits
across the entire life cycle.
 18 CHApTER 1 WHY IS SECURITY A SOf TwARE ISSUE?

7. Managing Secure Software Development

The previous section put forth useful arguments and identified emerg- ing evidence for the value of detecting
software security defects as early in the SDLC as possible. We now turn our attention to some of the key
project management and software engineering practices to aid in accomplishing this goal. These are introduced
here and covered in greater detail in subsequent chapters of this book.

1. Which Security Strategy Questions Should I Ask?

Achieving an adequate level of software security means more than complying with regulations or
implementing commonly accepted best practices. You and your organization must determine your own defini-
tion of “adequate.” The range of actions you must take to reduce soft- ware security risk to an acceptable level
depends on what the product,
 WHY IS SECURITY A SOf TwARE ISSUE?
A Risk Management Framework for Software Security
A necessary part of any approach to ensuring adequate software security is the definition and use of a
continuous risk management process. Software security risk includes risks found in the outputs and results
produced by each life-cycle phase during assurance activities, risks introduced by insufficient processes, and
personnel-related risks. The risk management framework (RMF) introduced here and expanded in Chapter 7
can be used to implement a high-level, consistent, iterative risk analysis that is deeply integrated throughout
the SDLC.
Figure 1–5 shows the RMF as a closed-loop process with five activity stages. Throughout the application of
the RMF, measurement and reporting activities occur. These activities focus on tracking, displaying, and
understanding progress regarding software risk.

3. Software Security Practices in the Development Life Cycle

Managers and software engineers should treat all software faults and weaknesses as potentially exploitable.
Reducing exploitable weaknesses begins with the specification of software security requirements, along with
considering requirements that may have been overlooked (see Chapter 3, Requirements Engineering for
Secure Software). Software that includes security requirements (such as security constraints on process
behaviors and the handling of inputs, and resistance to and tolerance of intentional failures) is more likely to
be engineered to remain dependable and secure in the face of an attack. In addition, exercising misuse/abuse
cases that anticipate abnormal and unex- pected behavior can aid in gaining a better understanding of how
to create secure and reliable software (see Section 3.2).
 1.7 MANAgINg SECURE SOf TwARE DEVELOpMENT 21

Measurement and Reporting

1 2 3 4
Understand Identify and Synthesize Define the Risk
the Business Link the and Rank Mitigation
Context Business and the Risks Strategy
Technical
Risks

Artifact
Analysis

Business Context 5
Carry Out
Fixes and
Validate

Figure 1–5: A software security risk management framework

Developing software from the beginning with security in mind is more effective by orders of magnitude than trying
to validate, through test- ing and verification, that the software is secure. For example, attempt- ing to demonstrate
that an implemented system will never accept an unsafe input (that is, proving a negative) is impossible. You can
prove, however, using approaches such as formal methods and function abstraction, that the software you are
designing will never accept an unsafe input. In addition, it is easier to design and implement the system so that
input validation routines check every input that the soft- ware receives against a set of predefined constraints.
Testing the input validation function to demonstrate that it is consistently invoked and correctly performed every
time input enters the system is then included in the system’s functional testing.
Analysis and modeling can serve to better protect your software against the more subtle, complex attack
patterns involving externally forced sequences of interactions among components or processes that were never
intended to interact during normal software execution. Analysis and modeling can help you determine how to
strengthen the security of the software’s interfaces with external entities and increase its tolerance of all faults.
Methods in support of analysis and modeling during each life-cycle phase such as attack patterns, misuse and abuse
cases, and architectural risk analysis are described in subsequent chapters of this book.
 WHY IS SECURITY A SOf TwARE ISSUE?
If your development organization’s time and resource constraints prevent secure development practices from being applied to
the entire software system, you can use the results of a business-driven risk assessment (as introduced earlier in this chapter
and further detailed in Section 7.4.2) to determine which software components should be given highest priority.
A security-enhanced life-cycle process should (at least to some extent) compensate for security inadequacies in the
software’s requirements by adding risk-driven practices and checks for the adequacy of those practices during all software
life-cycle phases. Figure 1–6 depicts one example of how to incorporate security into the SDLC using the concept of
touchpoints [McGraw 2006; Taylor 2005]. Software security best practices (touchpoints shown as arrows) are applied to a
set of software artifacts (the boxes) that are created during the software development process. The intent of this particular
approach is that it is process neutral and, therefore, can be used with a wide range of software development processes
(e.g., waterfall, agile, spiral, Capability Maturity Model Integration [CMMI]).
Security controls in the software’s life cycle should not be limited to the requirements, design, code, and test phases. It is
important to continue performing code reviews, security tests, strict configuration control, and quality assurance during
deployment and operations to ensure that updates and patches do not add security weaknesses or malicious logic to
production software.10 Additional considerations for project managers, including the effect of software security
requirements on project scope, project plans, estimating resources, and product and process measures, are detailed in
Chapter 7.
External
Security Review Code Review Penetration
Requirements (Tools) Testing
Abuse Risk Risk-Based Risk Security
Cases Analysis Security Tests Analysis Operations

Requirements Architecture Test Plans Code Tests and Feedback

and and Design Test Results from the
Use Cases Field

Figure 1–6: Software development life cycle with defined security

touchpoints [McGraw 2006]
1.8 Summary
It is a fact of life that software faults, defects, and other weaknesses affect the ability of software to
function securely. These vulnerabilities can be exploited to violate software’s security properties and
force the software into an insecure, exploitable state. Dealing with this possibility is a particularly
daunting challenge given the ubiquitous connectivity and explosive growth and complexity of software-
based systems.
Adopting a security-enhanced software development process that includes secure development
practices will reduce the number of exploitable faults and weaknesses in the deployed software.
Correcting potential vulnerabilities as early as possible in the SDLC, mainly through the adoption of
security-enhanced processes and practices, is far more cost-effective than attempting to diagnose and
correct such problems after the system goes into production. It just makes good sense.
Thus, the goals of using secure software practices are as follows:
• Exploitable faults and other weaknesses are eliminated to the greatest extent possible by well-
intentioned engineers.
• The likelihood is greatly reduced or eliminated that malicious engineers can intentionally implant
exploitable faults and weaknesses, malicious logic, or backdoors into the software.