Foundations of

Information Processing

Algorithmic problem solving

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 1

 Problem solving
Problem
=>
General problem solving
=>
Algorithmic problem solving
=>
Implementation

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 2

 General problem solving:
Different kinds of problems

Problem 1:
Connect nine points

The problem is limited which can/must be defined exactly.

How to define the points and their positions,

what does “connect” mean, etc.?

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 3

 Different kinds of problems

Problem 2:
Fix the internet connection
which does not work

“Clear” technical steps.

A classic engineering problem.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 4

 Different kinds of problems

Problem 3:
Cope with the climate change

A very wide task, including

many opinions,
many measurements and observations,
statistical estimation,
uncertainties, etc.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 5

 Problem solving in general

In studying and in many tasks at work skills for

problem solving and groupwork are needed.

What is needed for solving problems?

Could we “automate” data mining for problem solving?
How about producing ideas?
How about creativity? Could it be automated?

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 6

 Problem solving in general
”Everyone knows that creativity has to be fun, lively, and crazy:
so how can we have serious creativity?”
(Edward de Bono, a Maltese psychologist and author who proposed the
concept of lateral thinking = breaking frameworks and logical formulas in
creating new things)

• Requirements (what is needed in the beginning).

• Concepts and definitions (what are the considered matters).
• Problem categories (what to solve).
• Problem solving processes (how to solve).
• Creating ideas (how to stimulate your thinking).
• Innovations and creativity (possibilities and limitations).
• Illustration and documentation (how to present the solution).

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 7

 Requirements: do you have to be a genius?
Source: TIEDE.FI June 2013.

• Intelligence quotient (IQ).

• IQ is varying among known masters.
• For example:
• 190-199: Leibniz (philosophy) and Goethe (literature).
• 189-189: Laplace (natural science), Berkeley (philosophy), Voltaire
(literature).
…
• 150-159: Kepler (natural science), da Vinci (painting), Mozart
(music).
…
• 130-139: Copernicus (natural science), Rousseau (philosophy),
Burns (literature), Rossini (music).

• Thus, IQ is varying very much.

• Another matter:
• How to measure IQ and is the current way the best possible?

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 8

Concepts and definitions
Source: V. Virkkala. Luova ongelmanratkaisu. Vilkko Virkkala, Helsinki, 1994. (in Finnish)

• Problem solving: use data by combing as reasonable pieces of information.

• Raw data vs. intelligent information: “just” data vs. a smart plan to process
data.
• Raw: In the winter there are more road accidents than in the summer.
• Intelligent: A plan to increase the road safety in the winter.
• Qualitative vs. quantitative information: how to measure and to define?
• Quantitative: the temperature outdoors is +30 C.
• Qualitative: it is hot weather.
• Sources of information: selection and critical thinking (reliable source?).
• Critical evaluation of data/information (who talks and what):
• “Expert filter”.
• “Salesman filter”.
• “Number filter”.
• “Research filter”.
• “Prejudice filter”.
• “Showing own competence filter”.
©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 9
Problem categories
Source: V. Virkkala. Luova ongelmanratkaisu, Helsinki, 1994.

Analysis problems:
• For example: Owners of the company are interested in the
competitiveness of the company in the market segments of their products.
• What matters affect the competitiveness and is the matter causal
(genuine causality between the matter and the competitiveness)?

Synthesis problems:
• For example: A product of the company has been found to increasingly
cause returns.
• What should be the quality assessment of this product to get optimal
profits?

Valuation problems:
• For example: The salary system of the company is based on the demand
level of the job and the results.
• What salary should be paid to employees with different kinds of jobs,
and on what grounds?
©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 10
Problem categories: research examples .

Analysis problems:
• Bacteriorhodopsin is a photosensitive protein which can be separated from
a sheet of Halobacterium salinarium. An electric response can be measured
from a thin membrane manufactured from this material.
• What is this response based on?

Synthesis problems:
• The Bacteriorhodopsin membrane transfers photosensitivity to the electric
response.
• What kind a color camera could be built based on the Bacteriorhodopsin
membrane?

Valuation problems:
• A sheet of Halobacterium salinarium is grown biotechnically in a laboratory.
The living material is being broken in the centrifuge in order to separate
photosensitive protein.
• Is this right ethically?

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 11

 Problem solving process
Source: V. Virkkala. Luova ongelmanratkaisu,, Helsinki, 1994. (in Finnish)

1) Planning
• The problem to be solved
⇒Facts
⇒Ideas
⇒Solution

2) Implementation
• The solution in practice.

3) Verification and acceptance

• Make sure that the solution works.
• Convince people to accept the solution.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 12

Creating ideas
Source: V.Virkkala. Luova ongelmanratkaisu, Helsinki, 1994.

• Many techniques: some of them introduced here.

• Considering known solutions:
• Is this possible consider widely enough available solutions?
• Joining remote ideas: analogies and “crazy” combinations:
• Join ideas which are distant to each other.
• Jump out of the box in your thinking.
• For example, watershed segmentation in the segmentation of
digital images where the idea actually comes from geology.
• Brainstorming:
• Steps: pre, warming up, creating ideas, selecting ideas.
• Documentation and analysis (SWOT).
• Strengths, weaknesses, opportunities, and threats.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 13

Creating ideas (continued)
• Modifying ideas via the list of questions:
• Enlarge/shrink, divide/combine, replace, point out, reverse upside
down.
• “Double team”:
• Two persons as a pair first create ideas how to solve the problem
and then select the best ideas.
• Next two pairs are combined as the group of 4 persons, and the
ideas are merged, and the best ideas are selected.
• Hierarchical structure: groups of 2 => groups of 4 => groups of 8 =>
• Finally, the most popular ideas are selected.
• Techniques to create ideas are not limited.
• What does happen when the ideas of two groups are combined.
The risk to lose some ideas?
• Now and in future: creating ideas using AI-based tools.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 14

 Innovations and creativity
Source: V. Virkkala. Luova ongelmanratkaisu, Helsinki, 1994 (in Finnish).

• Innovations:
• It is not so easy: “… results obtained by systematic creating ideas
are very varying …”
• Motivating creativity:
• Environment, rewarding, own goals, own opinions, …
• Obstacles of creativity:
• Environment, attitude, leadership and management, …
• What would be a creative organization?
• Not too much restrictions of thinking?
• Not too much limitations of actions?
• Democracy vs. dictatorship?

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 15

 Illustration and documentation
Source: V. Virkkala. Luova ongelmanratkaisu, Helsinki, 1994 (in Finnish).

• To understand the solution, and especially to receive

acceptance from other people, the solution must be
documented:
• Written sentences.
• Evaluation matrices.
• Mind maps.
• Flow charts.
• ...
• Different roles, habits, and work cultures influence to
illustration and documentation:
• Researchers, engineers, salespersons, directors, etc.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 16

 Summary: problem solving in general

• Problem solving can be defined in general as combing data

for functioning steps to create the solution.

• Finding new solutions require (academic) research and data

mining, processing, combining, and analysis.

• Innovative and critical thinking & qualified documentation are

very important.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 17

 Towards algorithmic problem solving:
About the example problems
Problem 1:
Connect nine points
Let us specify more:

Connect nine points whose coordinates are stored in

the memory, connecting them with continuous lines
which are formed of four straight line segments at
maximum.
(More definitions could be introduced to understand the problem better, but let
us not go too much into details since the main goal here is just to
understand the type of the problem as compared to the other examples)
©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 18
 Problem 2:
Fix the internet connection
which does not work

Fix the connection using the properties of a

computer and the interface to a service
provider.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 19

 Problem 3:
Cope with the climate change

Make the plan to limit the climate change by

taking advantage of all possible views on the
Internet and the scientific facts about the history
and the current situation.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 20

 Algorithmic problem solving: the starting point

If well defined problems which can be formalized

appear continuously again in the same form, it is
worthwhile to find the repeatable solution to avoid
routine work.
Is it possible mechanize (automate) problem solving?
If the solution to a certain problem exists, is it possible
to mechanize solving the corresponding problems?
How to define an algorithm? What is it?

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 21

 Algorithmic problem solving
”An algorithm must be seen to be believed”
(Prof. Donald E. Knuth, an American computer scientist and
mathematician, Caltech, Princeton, Stanford)
• Planning algorithms.
• Properties of algorithms.
• Presenting algorithms.

Step Problem solving in general Algorithmic problem solving

1 Understanding the problem ?

2 Making the plan to solve the ?

problem
3 Implementing the plan ?
4 Evaluating the solution ?

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 22

 Algorithms and making them
Source: J. Boberg, Johdatus tietojenkäsittelytieteeseen, Turun yliopisto, 2010 (in Finnish).

• Algorithmic problem solving is to present the exact step-

by-step instructions to solve a given problem.
• This set of the instructions is called as the algorithm.
• To make the algorithm is needed
• an inclusive and unambiguous task description,
since for a person who uses the algorithm
• following the set of the instructions and
• the instructions themselves
must be totally clear.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 23

 Making vs. executing the algorithms
Source: J. Boberg, Johdatus tietojenkäsittelytieteeseen, Turun yliopisto, 2010 (in Finnish).

• Making algorithms is not the algorithmic action, as creativity is

needed in making
⇒ it is not possible to write an algorithm
to make an algorithm.
• It is quite a systematic task to use the algorithm which has
been made to solve a given problem:
• Program the algorithm.
• Transfer (compile) the program to the computer-based
language (machine language).
• Execute the transferred (complied) program.
⇒ the execution of the algorithm is a mechanical operation.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 24

 The computer: the tool for executing algorithms
Von Neumann architecture
(John von Neumann/János Lajos Neumann, 1945, a Hungarian computer
scientist, mathematician, physicist, engineer, University of Pázmány Péter,
University of Berlin, Princeton University)

Source: Wikipedia

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 25

 Algorithm: as a program for the computer
Algorithm
↓
Programming (with the rules of the used language)
↓
Program (higher level programming language)
↓
Compilation (with the compiler)
↓
Program (machine language)
↓
Linking (other needed programs linked)
↓
Program (executable machine language)
↓
Input → Execution →Output

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 26

 Algorithmic problem solving:
real problem => real solution

Real-world problem =>

From Task definition =>
a real-world problem Formalized problem =>
via a formalized solution Problem solving =>
and programming Algorithmic solution:
back to algorithm + data structure =>
a real-world solution. Programming =>
Program =>
Execution of program =>
Formalized solution =>
Interpretation of solution =>
Real-world solution
©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 27
 Algorithms: required properties
Source: J. Boberg, Johdatus tietojenkäsittelytieteeseen, Turun yliopisto, 2010 (in Finnish).
Source:E. Horowitz et al. Fundamentals of Data Structures in C. Computer Science Press, 1993.

• Generalizability: suitable to all the cases of the task.

• Determinism: the solution must be deterministic and at each
step it is known unambiguously what to do next.
• Output:
• Correctness: the result is always correct.
• Finiteness: the algorithm always terminates.
• Input: the range of input and how it affects the result.
• Effectiveness: how robust the solution is as a function of time
and space.

Are these requirements unambiguous?

For example, non-deterministic problems like playing games.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 28

 Algorithmic problem solving: steps
G. Polya, How to Solve It, 1945.
J.G. Bookshear: Computer Science - An overview, 10th Edition, Addison-Wesley, 2009.

Step Problem solving in general Algorithmic problem solving

1 Understanding the problem Understanding the problem

2 Making the plan to solve the Planning the algorithm

problem
3 Implementing the plan Making the algorithm and
presenting it as a program
4 Evaluating the solution Evaluating accuracy and
generalizability

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 29

 Presenting the algorithm
• Language or flow chart?
• Natural (expressive) language vs. formal (strictly defined)
language.
• Concepts of the language:
• Syntax ≈ grammar.
• ”A monkey ate a banana”: subject verb object => ok.
• Semantics ≈ meaning.
• ”A banana ate a monkey”: syntax ok, semantics not ok.
• Pragmatics ≈ context-dependent meaning.
• For example, sentences of a paragraph affect the
meaning of individual sentences and words.
• Maybe you have heard: “My comments have been quoted
out of the context, and I would like to correct …”.
• Images, mind maps, flow charts, state machines, …

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 30

Example: algorithm based on the (English) language
• After a couple weeks of studying, Mr. Brian Kottarainen faced a
problem: he ran out of clean socks. Happily, Brian found a
washing machine in the basement of his LOAS student house.
• However, how to wash clothes?
• Please help Brian and write an algorithm for him. Based on the
English language it could be as follows:

1. Gather the dirty socks.

2. Go to the washing room.
3. Wash the socks.
4. Dry the socks.
5. Pick up the socks from the washing room.

The algorithm can be extended and refined step by step.

This will be considered in the next lectures.
©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 31
 Summary

Algorithmic problem solving is based on making the set of

step-by-step instructions to solve the problem, or the
algorithm.

The algorithm is exactly defined concept: it must satisfy the

required properties.

There is no algorithm to make the algorithm since making

needs creativity.

Executing the algorithm is (supposed to be) mechanical

operation.

©LUT BM40A0102 Prof. Heikki Kälviäinen & Prof. Lasse Lensu 32