Algorithm Analysis

Objectives
To
understand
why
algorithm analysis is important.
To be able to use BigO to describe execution time.
To understand the BigO execution time of common
operations on Python lists and
dictionaries.
To understand how the
implementation of Python data
impacts algorithm analysis.
To understand how to
benchmark
simple
Python
programs.
What Is Algorithm Analysis?
an algorithm is a generic, stepby-step list of instructions for
solving a problem. It is a method
for solving any instance of the
problem such that given a
particular input, the algorithm
produces the desired result. A
program, on the other hand, is
an algorithm that has been
encoded
into
some
programming language. There
may be many programs for the
same algorithm, depending on
the
programmer
and
the
programming language being
used.
Algorithm
analysis
is
concerned
with
comparing
algorithms based upon the
amount of computing resources
that each algorithm uses. We
want to be able to consider two
algorithms and say that one is
better than the other because it
is more efficient in its use of
those resources or perhaps
because it simply uses fewer.
From this perspective, the two
functions above seem very
similar.
They
both
use
essentially the same algorithm
to
solve
the
summation
problem.
Algorithmic
complexity
is
concerned about how fast or
slow
particular
algorithm
performs. We define complexity
as a numerical function T(n) time versus the input size n. We
want to define time taken by an
algorithm without depending on
the implementation details. But

you agree that T(n) does depend

on the implementation! A given
algorithm will take different
amounts of time on the same
inputs
depending
on
such
factors as: processor speed;
instruction set, disk speed,
brand of compiler and etc. The
way around is to estimate
efficiency
of
each
algorithm asymptotically.
We
will measure time T(n) as the
number of elementary "steps"
(defined in any way), provided
each such step takes constant
time.
Let us consider two classical
examples:
addition
of two
integers. We will add two
integers digit by digit (or bit by
bit), and this will define a "step"
in our computational model.
Therefore, we say that addition
of two n-bit integers takes n
steps. Consequently, the total
computational time is T(n) = c *
n, where c is time taken by
addition of two bits. On different
computers, additon of two bits
might take different time, say
c1 and c2, thus the additon of
two n-bit integers takes T(n) =
c1 *
n and T(n)
=
c2 *
n respectively. This shows that
different machines result in
different
slopes,
but
time T(n) grows linearly as input
size increases.
The process of abstracting away
details and determining the rate
of resource usage in terms of
the input size is one of the
fundamental ideas in computer
science.

The Problem Solving process

consists of a sequence of
sections
that
fit
together
depending on the type of
problem to be solved. These are:

Problem Definition.

Problem Analysis.

Generating
possible
Solutions.

Analyzing
the
Solutions.

Selecting
the
best
Solution(s).

Planning
the
next
course of action (Next
Steps)
The process is only a guide for
problem solving. It is useful to
have a structure to follow to
make sure that nothing is
overlooked. Nothing here is
likely to be brand new to
anyone, but it is the pure
acknowledgement
and
reminding of the process that
can help the problems to be
solved.
1. Problem Definition
The normal process for solving a
problem will initially involve
defining the problem you want
to solve. You need to decide
what you want achieve and
write it down. Often people keep
the problem in their head as a
vague idea and can so often get
lost in what they are trying to
solve that no solution seems to
fit. Merely writing down the
problem forces you to think
about what you are actually
trying to solve and how much
you want to achieve. The first
part of the process not only
involves
writing
down
the
problem to solve, but also
checking that you are answering
the right problem. It is a checkstep to ensure that you do not
answer a side issue or only
solve the part of the problem
that is most easy to solve.
People often use the most
immediate solution to the first
problem definition that they find
without spending time checking
the problem is the right one to
answer.
2. Problem Analysis
The next step in the process is
often to check where we are,
what is the current situation and
what is involved in making it a
problem. For example, what are
the benefits of the current
product/service/process?
And
why did we decide to make it
like that? Understanding where

the problem is coming from,

how it fits in with current
developments and what the
current environment is, is crucial
when working out whether a
solution will actually work or
not. Similarly you must have a
set of criteria by which to
evaluate any new solutions or
you will not know whether the
idea is workable or not. This
section of the problem solving
process ensures that time is
spent in stepping back and
assessing the current situation
and what actually needs to be
changed.
After this investigation, it is
often good to go back one step
to reconfirm that your problem
definition
is
still
valid.
Frequently
after
the
investigation people discover
that the problem they really
want to answer is very different
from their original interpretation
of it.
3.
Generating
possible
Solutions
When you have discovered the
real problem that you want to
solve and have investigated the
climate into which the solution
must fit, the next stage is to
generate a number of possible
solutions. At this stage you
should
concentrate
on
generating many solutions and
should not evaluate them at all.
Very often an idea, which would
have
been
discarded
immediately, when evaluated
properly, can be developed into
a superb solution. At this stage,
you should not pre-judge any
potential solutions but should
treat each idea as a new idea in
its own right and worthy of
consideration.
4. Analyzing the Solutions
This section of the problem
solving process is where you
investigate the various factors
about each of the potential
solutions. You note down the
good and bad points and other
things which are relevant to
each solution. Even at this stage
you are not evaluating the
solution because if you do so
then you could decide not to
write down the valid good points
about it because overall you
think it will not work. However
you might discover that by
writing down its advantages that
it
has
a
totally
unique
advantage. Only by discovering
this might you choose to put the

effort in to develop the idea so

that it will work.
5.
Selecting
the
best
Solution(s)
This is the section where you
look
through
the
various
influencing factors for each
possible solution and decide
which solutions to keep and
which to disregard. You look at
the solution as a whole and use
your judgement as to whether
to use the solution or not. In
Innovation Toolbox, you can vote
using either a Yes/No/Interesting
process or on a sliding scale
depending on how good the idea
is. Sometimes pure facts and
figures dictate which ideas will
work and which will not. In other
situations, it will be purely
feelings
and
intuition
that
decides.
Remember
that
intuition is really a lifetimes
experience
and
judgement
compressed
into
a
single
decision.
By voting for the solutions you
will end up with a shortlist of
potential solutions. You may
want to increase the depth in
the analysis of each idea and
vote again on that shortlist to
further refine your shortlist.
You will then end up with one,
many or no viable solutions. In
the case where you have no
solutions that work, you will
need to repeat the generation of
solutions section to discover
more
potential
solutions.
Alternatively you might consider
re-evaluating the problem again
as sometimes you may not find
a solution because the problem
definition is not well defined or
self-contradictory.
6. Planning the next course
of action (Next Steps)
This section of the process is
where you write down what you
are going to do next. Now that
you have a potential solution or
solutions you need to decide
how you will make the solution
happen. This will involve people
doing various things at various
times in the future and then
confirming that they have been
carried out as planned. This
stage ensures that the valuable
thinking that has gone into
solving the problem becomes
reality. This series of Next Steps
is the logical step to physically
solving the problem.
In programming, classification of
a particular type of information.

It is easy for humans to

distinguish between different
types of data. We can usually
tell at a glance whether a
number is a percentage, a time,
or an amount of money. We do
this through special symbols -%, :, and $ -- that indicate the
data's type. Similarly,
a computeruses special internal
codes to keep track of the
different types of data it
processes.
Most programming
languages require
the programmer to declarethe
data type of every data object,
and most database
systemsrequire the user to
specify the type of each
data field. The available data
types vary from one
programming language to
another, and from one database
application to another, but the
following usually exist in one
form or another:

integer : In more common

parlance, whole number; a
number that has no fractional
part.
floating-point :A number with
a decimal point. For example, 3
is an integer, but 3.5 is a
floating-point number.
character (text ): Readable
text

An implementation of ADT
consists
of
storage
structures to store the data
items and algorithms for
basic operation

1.
2.

Abstract Data Type (ADT): a

collection of data items together
with the operations on the data
The word abstract
refers to the fact that
the data and the basic
operations defined on it
are
being
studied
independently of how
they are implemented

We think about what

can be done with the
data, not how it is
done

ADT consists of
The collection of data
items
Basic operation that
must be performed on
them
In the example, a
collection of data is a
list of seats
The basic operations
are (1) Scan the list to
determine which seats
are
occupied
(2)
change seats status