Software archiecture lecture10

The ATAM
A Comprehensive method for
Architecture Evaluation

Introduction
• We will introduce the Architecture Tradeoff
Analysis Method (ATAM), a thorough and
comprehensive way to evaluate a software
architecture.
• The ATAM is so named because
▫ it reveals how well an architecture satisfies
particular quality goals
▫ it provides insight into how quality goals interact—
that is, how they trade off.

Introduction
• Evaluating an architecture for a large system is a
complicated undertaking.
• First, a large system will have a comparably large
architecture that will be difficult to understand in a
limited amount of time.
• Second, according to the Architecture Business Cycle
(ABC), a computer system is intended to support
business goals and the evaluation will need to make
connections between those goals and the technical
decisions.
• Finally, a large system usually has multiple stakeholders
and acquiring their different perspectives in a limited
amount of time requires careful management of an
evaluation process.
• Managing the limited time for an architecture evaluation
is a central problem.

Introduction
• The ATAM is designed to
▫ elicit the business goals for the system as well as
for the architecture
▫ those goals and stakeholder participation to focus
the attention of the evaluators on the portion of
the architecture that is central to the achievement
of the goals.
• This chapter will introduce the steps of the
ATAM and discuss them in light of their intended
purpose.

Participants in the ATAM
• The ATAM requires the participation of three groups
• The evaluation team
▫ This group is external to the project.
▫ They need to be recognized as competent, unbiased
outsiders with no hidden agendas or axes to grind.
• Project decision makers
▫ These people are empowered to speak for the
development project or have the authority to mandate
changes to it
• Architecture stakeholders
▫ Their job during an evaluation is to articulate the
specific quality attribute goals that the architecture
should meet in order for the system to be considered
a success.

ATAM Evaluation Team Roles
Role Responsibilities Desirable characteristics

Team Leader Sets up the evaluation; coordinates with Well-organized, with managerial skills;
client, making sure client's needs are good at interacting with client; able to
met; establishes evaluation contract; meet deadlines
forms evaluation team; sees that final
report is produced and delivered
(although the writing may be delegated)
Evaluation Leader Runs evaluation; facilitates elicitation of Comfortable in front of audience;
scenarios; administers scenario excellent facilitation skills; good
selection/prioritization process; facilitates understanding of architectural issues;
evaluation of scenarios against practiced in architecture evaluations;
architecture; facilitates onsite analysis able to tell when protracted discussion
is leading to a valuable discovery or
when it is pointless and should be re-
directed


Scenario Scribe Writes scenarios on flipchart or whiteboard Good handwriting; stickler about
during scenario elicitation; captures agreed- not moving on before an idea
on wording of each scenario, halting (scenario) is captured; can
discussion until exact wording is captured absorb and distill the essence of
technical discussions
Proceedings Scribe Captures proceedings in electronic form on Good, fast typist; well organized
laptop or workstation, raw scenarios, issue(s) for rapid recall of information;
that motivate each scenario (often lost in the good understanding of
wording of the scenario itself), and resolution architectural issues; able to
of each scenario when applied to assimilate technical issues
architecture(s); also generates a printed list quickly; unafraid to interrupt the
of adopted scenarios for handout to all flow of discussion (at opportune
participants times) to test understanding of
an issue so that appropriate
information is captured
Timekeeper Helps evaluation leader stay on schedule; Willing to interrupt discussion to
helps control amount of time devoted to each call time
scenario during the evaluation phase


Process Observer Keeps notes on how evaluation process could Thoughtful observer;
be improved or deviated from; usually keeps knowledgeable in the evaluation
silent but may make discreet process-based process; should have previous
suggestions to the evaluation leader during experience in the architecture
the evaluation; after evaluation, reports on evaluation method
how the process went and lessons learned
for future improvement; also responsible for
reporting experience to architecture
evaluation team at large
Process Enforcer Helps evaluation leader remember and carry Fluent in the steps of the
out the steps of the evaluation method method, and willing and able to
provide discreet guidance to the
evaluation leader
Questioner Raise issues of architectural interest that Good architectural insights; good
stakeholders may not have thought of insights into needs of
stakeholders; experience with
systems in similar domains;
unafraid to bring up contentious
issues and pursue them; familiar
with attributes of concern

Outputs of the ATAM
• An ATAM-based evaluation will produce at least
the following outputs
▫ A concise presentation of the architecture.
▫ Articulation of the business goals.
▫ Quality requirements in terms of a collection of
scenarios.
▫ Mapping of architectural decisions to quality
requirements.
▫ A set of identified sensitivity and tradeoff points.
▫ A set of risks and nonrisks.
▫ A set of risk themes.

Outputs of the ATAM
• The outputs are used to build a final written
report that recaps the method, summarizes the
proceedings, captures the scenarios and their
analysis, and catalogs the findings.
• There are intangible results
▫ a palpable sense of community on the part of the
stakeholders
▫ open communication channels between the
architect and the stakeholders
▫ a better overall understanding on the part of all
participants of the architecture
▫ its strengths and weaknesses

ATAM Phases and Their Characteristics
Phase Activity Participants Typical Duration
0 Partnership and Evaluation team leadership and Proceeds
preparation key project decision makers informally as
required, perhaps
over a few weeks
1 Evaluation Evaluation team and project 1 day followed by
decision makers a hiatus of 2 to 3
weeks
2 Evaluation Evaluation team, project decision 2 days
(continued) makers, and stakeholders
3 Follow-up Evaluation team and evaluation 1 week
client

Phases of the ATAM
• Activities in an ATAM-based evaluation are
spread out over four phases
▫ phase 0, "Partnership and Preparation," the
evaluation team leadership and the key project
decision makers informally meet to work out the
details of the exercise.
▫ phase 1, the evaluation team meets with the
project decision makers to begin information
gathering and analysis.
▫ phase 2, the architecture's stakeholders join the
proceeding and analysis continues.
▫ phase 3 is follow-up in which the evaluation team
produces and delivers a written final report.

Steps of the evaluation phases
• Step 1—Present the ATAM
• Step 2—Present Business Drivers
• Step 3—Present Architecture
• Step 4—Identify Architectural Approaches
• Step 5—Generate Quality Attribute Utility Tree
• Step 6—Analyze Architectural Approaches
• Hiatus and Start of Phase 2
• Step 7—Brainstorm and Prioritize Scenarios
• Step 8—Analyze Architectural Approaches
• Step 9—Present Results

1. Present the ATAM
• The evaluation leader present the ATAM
• The leader will describe the ATAM steps in brief
and the outputs of the evaluation.
• Explain the process that everyone will be
following, to answer questions, and to set the
context and expectations for the remainder of
the activities.

2. Present Business Drivers
• Let everyone to understand the context for the
system and the primary business drivers motivating
its development
• A project decision maker (project manager)
presents a system overview from a business
perspective
▫ The system's most important functions
▫ Any relevant technical, managerial, economic, or
political constraints
▫ The business goals and context as they relate to the
project
▫ The major stakeholders
▫ The architectural drivers (that is, the major quality
attribute goals that shape the architecture)

3. Present Architecture
• The lead architect (or architecture team) makes a
presentation describing the architecture at an appropriate
level of detail
• Presentation covers technical constraints such as
▫ Operating system
▫ Hardware
▫ Middleware prescribed for use
▫ Other systems with which the system must interact.
• Most important, the architect describes the architectural
approaches used to meet the requirements.
• It should convey the essence of the architecture and not stray
into ancillary areas or delve too deeply into the details of just
a few aspects.
• During presentation, The evaluation team asks for clarification
based on their phase 0 examination of the architecture
documentation and their knowledge of the business drivers
from the previous step

Example of a template for the
architecture presentation
• Architecture Presentation (~20 slides; 60 minutes)

• Driving architectural requirements, the measurable quantities you
associate with these requirements, and any existing
standards/models/approaches for meeting these (2–3 slides)
• Important Architectural Information (4–8 slides)
▫ Context diagram—the system within the context in which it will exist.
Humans or other systems with which the system will interact.
▫ Module or layer view—the modules (which may be subsystems or layers)
that describe the system's decomposition of functionality, along with the
objects, procedures, functions that populate these, and the relations
among them (e.g., procedure call, method invoca-tion, callback,
containment).
▫ Component-and-connector view—processes, threads along with the
synchronization, data flow, and events that connect them.
▫ Deployment view—CPUs, storage, external devices/sensors along with
the networks and communication devices that connect them.Also shown
are the processes that execute on the various processors.

Example of a template for the
architecture presentation
• Architectural approaches, patterns, or tactics employed, including what
quality attributes they address and a description of how the approaches
address those attributes (3–6 slides)
▫ Use of commercial off-the-shelf (COTS) products and how they are
chosen/integrated (1–2 slides)
▫ Trace of 1 to 3 of the most important use case scenarios. If possible,
include the runtime resources consumed for each scenario (1–3 slides)
▫ Trace of 1 to 3 of the most important change scenarios. If possible,
describe the change impact (estimated size/difficulty of the change) in
terms of the changed modules or interfaces (1–3 slides)
▫ Architectural issues/risks with respect to meeting the driving
architectural requirements (2–3 slides)
▫ Glossary (1 slide)

4. Identify Architectural Approaches
• By now, the evaluation team will have a good
idea of what patterns and approaches the
architect used in designing the system.
• In this short step, the evaluation team simply
catalogs the patterns and approaches that are
evident.
• The list is publicly captured by the scribe for all
to see and will serve as the basis for later
analysis.

5. Generate Quality Attribute Utility
Tree
• An architecture is either suitable or unsuitable with
respect to its ability to deliver particular quality
attributes to the system(s) built from it.
• In this step, the quality attribute goals are
articulated in detail via a mechanism known as the
utility tree
• the evaluation team works with the project decision
makers to identify, prioritize, and refine the system's
most important quality attribute goals, which are
expressed as scenarios
• The utility tree serves to make the requirements
concrete, forcing the architect and customer
representatives to define precisely the relevant
quality requirements that they were working to
provide.

Tree
• A utility tree begins with utility as the root node.
• Second level is set of quality attributes, business drivers
presentation in step 2 make up the initial set of this
second level.
• Third level, under each quality attribute are specific
quality attribute refinements
▫ For example, performance might be decomposed into "data
latency" and "transaction throughput”
• Fourth level, Scenarios : ATAM scenarios consist of three
parts:
▫ stimulus (what condition arrives at a system, who
generated it, and what system artifact it stimulates)
▫ environment (what is going on at the time)
▫ response (system's reaction to the stimulus expressed in a
measurable way).

Tree
• Some scenarios might express more than one
quality attribute and so might appear in more
than one place in the tree.
• The evaluation leader should guard against
scenarios that try to cover too much diverse
territory because they will be difficult to analyze.
• A utility tree can easily contain fifty scenarios at
its leaves, and there will not be time during the
evaluation meeting to analyze them all. Hence,
utility tree generation also includes a
prioritization step.

Tree : prioritization step
• By consensus, the decision makers assign a priority
to each scenario.
• This prioritization may be on a 0 to 10 scale or use
relative rankings such as high, medium, and low.
• After that, scenarios are prioritized a second time,
by ask the architect to rank each scenario by how
difficult he or she believes it will be for the
architecture to satisfy.
• Now each scenario has an associated ordered pair:
(H,H), (H,M), (H,L), and so forth. The scenarios that
are the most important and the most difficult will be
the ones where precious analysis time will be spent
and the remainder will be kept as part of the record.

5. Example in Tabular Form of the
Utility Tree
Quality Attribute Attribute
Refinement Scenarios
Performance Transaction A user updates a patient's account in response to a change-of-
response time address notification while the system is under peak load, and
the transaction completes in less than 0.75 second. (H,M)
A user updates a patient's account in response to a change-of-
address notification while the system is under twice the
current peak load, and the transaction completes in less than
4 seconds. (L,M)
Throughput At peak load, the system is able to complete 150 normalized
transactions per second. (M,M)
Generating reports No scenarios suggested.

Usability Proficiency training A new hire with two or more years experience in the business
becomes proficient in Nightingale's core functions in less than
1 week. (M,L)
A user in a particular context asks for help, and the system
provides help for that context. (H,L)
Normal operations A hospital payment officer initiates a payment plan for a
patient while interacting with that patient and completes the
process without the system introducing delays. (M,M)

6. Analyze Architectural Approaches
• In fact, the analysis steps of the ATAM consist of
choosing one scenario at a time and seeing how well the
architecture responds to, or achieves, it.
• The architect is asked to explain how the architecture
supports each scenario.
• Questioners probe for the architectural approaches that
the architect used to carry out the scenario.
• The team documents the relevant architectural decisions
and identifies and catalogs their risks, nonrisks,
sensitivity points, and tradeoffs.
• The key of analysis is to elicit sufficient architectural
information to establish some link between the
architectural decisions that have been made and the
quality attribute requirements that need to be satisfied.

• For example of analysis
• The number of simultaneous database clients
will affect the number of transactions that a
database can process per second.
• Thus, the assignment of clients to the server is a
sensitivity point with respect to the response as
measured in transactions per second.
• Some assignments will result in unacceptable
values of this response—these are risks.

Example of architectural approach
analysis

Hiatus and Start of Phase 2
• At this point, phase 1 is concluded.
• The evaluation team retreats to summarize what it has
learned and interacts informally (usually by phone) with the
architect during a hiatus of a week or two.
• More scenarios might be analyzed during this period, if
desired, or questions of clarification can be resolved.
• When the project's decision makers are ready to resume and
the stakeholders are assembled, phase 2 commences.
• This phase is enacted by an expanded list of participants with
additional stakeholders attending.
▫ First, step 1 is repeated so that the stakeholders understand the
method and the role they are to play.
▫ Then the evaluation leader recaps the results of steps 2 through
6, and shares the current list of risks, nonrisks, sensitivity points,
and tradeoff points.
▫ Now the stake holders are up to speed with the evaluation results
so far, and the remaining three steps can be carried out.

7. Brainstorm and Prioritize Scenarios
• The evaluation team asks the stakeholders to brainstorm
scenarios that are operationally meaningful with respect
to the stakeholders' individual roles.
• Stakeholders are free to put them into the brainstorm
pool, which gives them the opportunity to revisit
scenarios from step 5 and step 6 that they might feel
received too little attention.
• Once the scenarios have been collected, they must be
prioritized.
• Each stakeholder is allocated a number of votes equal to
30% of the number of collected scenarios.
• Each stakeholder casts his or her votes publicly.
• The evaluation leader orders the scenarios by vote total
and looks for a sharp drop-off in the number of votes

7. Brainstorm and Prioritize Scenarios
• The prioritized list of brainstormed scenarios is
compared with those from the utility tree
exercise.
• If they agree, it indicates good alignment
between what the architect had in mind and
what the stakeholders actually wanted.
• If additional driving scenarios are discovered,
this may itself be a risk showing that there was
some disagreement in goals between the
stakeholders and the architect.

• The architect explains how relevant architectural
decisions contribute to realizing each scenario
from step 7.
• Ideally this activity will be dominated by the
architect's explanation of scenarios in terms of
previously discussed architectural approaches.
• In this step the evaluation team performs the
same activities as in step 6, mapping the
highest-ranked, newly generated scenarios onto
the architectural artifacts uncovered thus far.

9. Present Results
• Finally, the collected information from the ATAM
needs to be summarized and presented once
again to stakeholders.
• In this presentation the evaluation leader
recapitulates the steps of the ATAM and all the
information collected in the steps of the method,
including the business context, driving
requirements, constraints, and architecture.

9. Present Results
• Then the following outputs are presented:
▫ The architectural approaches documented
▫ The set of scenarios and their prioritization from
the brainstorming
▫ The utility tree
▫ The risks discovered
▫ The nonrisks documented
▫ The sensitivity points and tradeoff points found

9. Present Results
• The evaluation team adds value by grouping
risks into risk themes, based on some common
underlying concern or systemic deficiency.
▫ Ex: a group of risks about inadequate or out-of-
date documentation might be grouped into a risk
theme stating that documentation is given
insufficient consideration.
• For each risk theme, the evaluation team
identifies which of the business drivers listed in
step 2 are affected.

Using the Limited Time of Evaluation
Effectively
• We identified limited time as one of the main
problems in conducting an architectural evaluation.
• ATAM show how can it solves the problem
• The business goals are used as motivation for the
collection of scenarios that represent the utility tree.
• Scenarios are prioritized, essentially, as a bottom-up
check on the top-down scenario generation of the
utility tree.
• Only the high-priority and difficult scenarios are
analyzed.
• These are the areas that will yield the most
important results.

PHASE 3: Follow-up
• The tangible output of the ATAM is a final report that contains a list
of risks, nonrisks, sensitivity points, and tradeoff points.
• It also contains a catalog of architectural approaches used, the
utility tree and brainstormed scenarios, and the record of analysis of
each selected scenario.
• Finally, the final report contains the set of risk themes identified by
the evaluation team and an indication of which business drivers are
jeopardized by each one.
• Like the presentation of results, we use a boilerplate template that
has many of the standard sections (such as a description of the
ATAM) completed and templates for other sections ready to be filled
in.
• We also write some of the final report—for instance, the utility tree
and step 6 analysis—during the hiatus between phases 1 and 2.
• Preparation pays off; whereas it used to take about two weeks to
produce a final report for an ATAM client, we can now produce a
high-quality comprehensive report in about two days.

Summary
• The ATAM is a robust method for evaluating software
architectures.
• It works by having project decision makers and
stakeholders articulate a precise list of quality attribute
requirements (in the form of scenarios) and by
illuminating the architectural decisions relevant to
carrying out each high-priority scenario.
• The decisions can be cast as risks or nonrisks to find any
trouble spots in the architecture.
• In addition to understanding what the ATAM is, it is also
important to understand what it is not.
▫ The ATAM is not an evaluation of requirements.
▫ The ATAM is not a code evaluation.
▫ The ATAM does not include actual system testing.
▫ The ATAM is not a precise instrument, but identifies
possible areas of risk within the architecture.

Software archiecture lecture10

Recommended

More Related Content

What's hot (20)

Viewers also liked (10)

Similar to Software archiecture lecture10 (20)

Recently uploaded (20)

Software archiecture lecture10