Qualitative methods.

Expert methods.
The most common qualitative research
methods:
focus group discussions,
case study,
in-depth interviews,
observation,
questionnaire.

2
 Expert

An expert is a qualified specialist who

expresses his or her opinion in the
forecasting, planning or decision-making
process.

3
 Expert

An expert is a specialist in a specific field or a specialist in

management,
who has the necessary knowledge,
capable of analyzing new information,
who is able to assess the object of expertise within the
limits of his / her competence,
who is familiar with technologies, is able to assess their
use in a specific situation,
Is responsible for his / her conclusions and
recommendations.
4
 Expert assessment

Individual expert assessment is a

specialist's judgment about the research
object, guided by the task.

Collective expert assessment is a set of

expert assessments expressed by a group
of experts.
5
Expertise (verb, not noun) is a procedure
for obtaining expert opinions

Expert assessments can be obtained both

in the form of an interview and by filling
in questionnaires

6
Expert methods are used :
• if there is no information about the research
object or it is insufficient or inaccurate;
• gathering information is a very expensive and
labor-intensive process;
• the information about the research object is of
qualitative nature;
• there is uncertainty in the functioning of the
object;

7
What new professions will be required in the labor
market in next 5 years, that haven’t been demanded or
didn’t exist until now?
1
2
3
4
5
6
7
…

8
 9
Avots: Pētījums „Nereģistrētās nodarbinātības novērtējums”, Latvijas Universitāte, 2007.
• the forecasted object is new, it has no
analogues;
• rapid changes are possible in the
development of the researched object;
• there is a time deficit and therefore a
quantitative analysis cannot be performed.

10
Combine with other methods:
• qualitative analysis of the research object
is required;
• the factors influencing the development of
the object are not known;

11
 Scheme of data collection, data extrapolation,
compilation of expert assessments and development of
forecast variants

12
Source: Research on Forecasted Changes in Labor and Building Materials Costs in the Construction Sector in Latvia
 Please assess the influence of these factors on
the apartment price per sq.m. in a new project.
Factor Scale
Location of the project
Territory amenities
Construction materials used in the project
Urban communications infrastructure
Parking availability
Number of floors
Mean number of flats per floor
Security system
Mean height of ceilings
Type and size of windows
Availability of an elevator
Interior design
Balcony, terrace
Monthly management fee
Immobility liquidity in the area
View from the project
Apartment layout
Area of the apartment 13
Number of rooms
Experts give specific forecasts based
for a market

14
Experts assess outcomes of different
scenarious

15
• it is necessary to assess the qualitative
factors that significantly affect the
research object;
• forecast variants must be ranked;
• grouping, ranking and prioritization of
various objects and indicators.

16
 17
Avots: Ekspertu anketa, Valentīna Bogdanova, 2007.
 Objectives of application of expert methods

• Goal setting
• Expert forecasts
• Situation development scenarios
• Generation of alternatives
• Rating determination
• Evaluation systems
• Collective decision making
18
Past scenarios with a cost / output
ratio looking at critical periods of
development
Expert estimates of cost growth under various
construction output growth scenarios
 What to do with unregistered employment?
1st alternative 2nd alternative 3rd alternative
Increase penalties Introduce Create a new
Alternatives / for employers penalties for agency fighting
Consequences employees unregistered
employment

General economic
development
National image on the
international stage
National Short term and long term consequences
competitiveness on different economic areas
Convergence of labor
market to the EU level Scale -2 to +2
Re-emigration of
skilled workforce
Immigration of skilled
workforce etc. 21
 22
Avots: Pētījums „Nereģistrētās nodarbinātības novērtējums”, Latvijas Universitāte, 2007.
 Stages of the expertise
1. Establishment of a group of organizers of expertise
2. Preparation of expertise
3. Selection of experts and forming an expert group(s)
4. Expert survey
5. Statistical analysis of survey results
6. Analysis of the results of the expertise

24
 Selection of experts

Expert selection criteria:

• Industry experience
• General experience
• Intuition
• Psychological type
• Stability of the opinion
25
Assessment of the level of competence
of experts:
1. The competence of experts is
assessed by analysts
2. Experts carry out self-assessment
3. Experts evaluate each other
26
 Level of expert competence

v 
j 1
j j

k m

v
j 1
max j

vj – expert evaluation in points on the issue j

vmax – the maximum value of the scale in issue j
j – j-th issue weight coefficient 27
 Number of experts in the group:

Too many experts reduce the average

competence of the group

Too few experts allow individuals to

influence the results

28
 Selection of experts

• The most competent expert candidate is

selected
• Ask for a list of the 10 most competent experts
• For each of them is asked to nominate the 10
most competent experts
• Repeat until no one can name any new person
• The most frequently mentioned ones are
selected from the obtained list of experts

29
30
 Delphi method

• The survey takes place in several rounds,

specifying and detailing the questions in each
round
• After each round, the results are summarized
and presented to all experts
• If an expert gives a significantly different
assessment, he is asked to explain his opinion
• The expert survey is usually anonymous
31
32
33
 Experts fill in gaps

Please indicate what information is provided on each

connection
1
2
3
…
Please indicate missing connections, elements, errors etc.
…

34
Avots: Pētījums „Nereģistrētās nodarbinātības novērtējums”, Latvijas Universitāte, 2007.
35
 "The Delphi method structures the group
communication process in such a way that it
allows a group of individuals, as a whole, to
solve complex problems effectively."
Harold A. Linstone, Murray Turoff

36
 Delphi method

The aim is to achieve unity of opinion.

In the Delphi method, the thesis and the anti-

thesis are combined to achieve a synthesis that
is recognized by both parties and becomes the
new thesis.

37
 Delphi method
A bit of history:
• Created for military research
• US Air Force order for Project Delphi in the
1950s
• An alternative to expensive methods
• 1964, Gordon, Helmer “Forecasts of Science
and Technology Development”

38
 Delphi method

Preparatory stages
1. Preliminary research of the object
2. Research of group opinions
3. Evaluation of differences of opinion
4. Analysis of information from previous stages
and submission for evaluation

39
 Delphi method

• Not comparable to analytical methods, gives a

subjective, collective assessment
• Participants are not interconnected
• More participants than in face-to-face
methods
• Cheaper than face-to-face methods
• Resolves disputes, opposing views
• Diversity of participants must be ensured
40
 Advantages of the Delphi method

• Reduced conformism
• Reduced influence of incompetent experts
• More reliable results
• It is possible to assess the stability of opinions
over time

41
 Disadvantages of the Delphi method

• Sometimes use can be expensive

• High time consumption
• Underestimated individual competence
• Fewer new ideas
• Difficult organizational process

42
 SEER method

• System for Event Evaluation and Review

• DELFI method sub-method

43
 SEER method

• In the first round, experts-theorists compile a

list of possible events, assess when they
could occur and with what probability
• In the second round, experts - practitioners
supplement the list of obtained events,
determine the most important ones, specify
and adjust the complex forecast

44
 Program forecasting method

• Determines the probabilities of occurrence

of events and deadlines for occurrence,
including conditions.

45
 Program forecasting method

• List of events to be evaluated

• Round 1 - list of conditions and probability and
time of occurrence of events
• A network of conditions is being created
• Round 2 - adds conditions, selects condition
experts
• Round 3 - probability and time of occurrence of
conditions
• List of networked works with probabilities and
deadlines
46
 “Brainstorming” method

An unstructured process of generating and

evaluating ideas.

Used as a non-traditional idea generator for

solving complex or new problems.

47
 “Brainstorming” method

The first stage

1. Initial formulation of the problem
2. Establishment of an expert group
3. Invitation of experts in person
4. (Problem reformulation)
5. (Choice of problem wording)
6. Idea generation
48
 “Brainstorming” method

The second stage

1. Make a list of ideas
2. Discard inefficiently used ideas
3. Note the simplest ideas
4. Note the most promising ideas
5. Note additional ideas worth extra research
6. Separates the useful aspects of silly ideas
49
 “Brainstorming” method

The most suitable areas

• All areas where many solutions are possible
• Areas directly related to group members
• Challenges for experts in different fields to
work together

50
 “Brainstorming” method

The method is not suitable

• If the problem is complex
• Special knowledge required
• Conflicts may occur in the group

51
 “Brainstorming” method

Necessary support during brainstorming:

• Organizational support
• Methodological support
• Psychological support
• Informational support

52
 “Brainstorming” method
Duties of the moderator:
• formulate the problem at the beginning and recall it
after a while
• ensure the observance of the basic principles of the
method
• stop putting forward ideas if everyone is talking at the
same time
• give "quiet" participants a chance to speak
• prevent the evaluation of expressions
• record all ideas
• prepare proposals in the event of a break
53
 “Brainstorming” metode

Principles to be followed
• Clearly defined task
• Any criticism is forbidden
• No restrictions on ideas
• Lots of ideas
• Capture every idea
• Look for combinations and synergies
• Presence of creative experts
• Optimal group size (6-12 experts) 54
 Commission method

• A special commission of experts discusses

certain issues with a view to reconciling
views and developing a common solution.

55
 Commission method

1. Problem formulation and commission

formation
2. Independent work of experts
3. Experts reports to the commission
4. Discussion
5. Choosing the best ideas or combination
6. Voting
56
 Commission method

Advantages
• More, comprehensive information
• Lower risk of making mistakes
• Majority support
• The group is more willing to take
responsibility
• Complex problems can be solved
57
 Commission method

Disadvantages
• A long and laborious process
• Compromise can prevail
• May not agree
• Influence of personalities
• Possibility to misinform the group

58
59
 Expert survey

Central aspect of expert method

60
 Expert survey

One of the central aspects of expert

method is the organization of an expert
survey.

61
Prerequisites for the survey
• Information provided
• Unambiguous questions
(Terms, multiple questions)
• Independence of expression
• Low time consumption
• Anonymity
62
Please assess the minimum and maximum price levels and the average price level
for residential projects by regions in EUR per square meter.
Region 2019 2020 2021

Min Av Max Min Av Max Min Av Max

Riga center 1490 3500 6940 1550 3650 7240

Riga suburbs 900 1680 5340 1090 2030 6450

Jurmala 1650 3260 5350 1700 3360 5510

Other regions 480 760 1490 540 850 1670

63
 What shall we do with state owned companies –
Latvian Postal Service, AirBaltic, Latvian Railways,
Latvenergo?

o Privatize
o Privatize partially
o Leave in state ownership

64
Source: Association of Economists 2010, survey “Development and economic challenges of Latvia”, 2009.
65
 Expert survey forms

Breakdown of survey forms:

• Individual vs group
• Face-to-face vs distant
• Open vs closed (results)

66
The form of the survey is chosen depending on:
• the objectives and tasks of the expertise,
• the nature and complexity of the problem,
• time resources,
• financial resources,
• expert qualifications.

67
Breakdown of questions:
• on the essence of the researched
problem,
• ancillary questions.

68
Data about the respondent:
Name, surname
Position
Company name
Contact information
Industry
Region

69
Avots: Ekspertu anketa, Jeļena Buraja (ar papildinājumiem), 2007.
 What is the number of employees in your company?

What is the number of employees in your company working on BIM projects?

Which BIM elements have been introduced in your company?

70
Source: Ieguldījumu novērtējums būvju informācijas modelēšanas (BIM) ieviešanai būvniecības pasūtītāju, projektēšanas un būvniecības uzņēmumos, 2019.
Breakdown of questions by form:
• open vs closed

71
Please evaluate the level of professional mobility in coming 5 years.
Significant increase | Increase | Stable | Decrease | Significant decrease

72
Source: Pētījums „Darbaspēka profesionālā mobilitāte”, Latvijas Universitāte, 2007.
Breakdown of questions by type of answer:
• Rating
• Ranking
• Grouping
• Direct value assessment
• Structure value assessment
• Choice of option
73
 Rating scales
Types:
Numeric, Verbal, Visual, Likert, Graphic

Properties:
Range
Symmetric vs Asymmetric
Neutral vs No-Neutral

74
Please evaluate the residential building project market development
tendencies using the scale provided!
Potential price change – significant increase / stable / significant decrease
+3 +2 +1 0 -1 -2 -3
Time period sig. price stable prices sig. price
increase decrease

2020 Q4

2021 Q1

Market activity – very active / stable / very inactive

+3 +2 +1 0 -1 -2 -3
Time period very active stable not active
market market market
Increased Stable Decreased
2020 Q4 number of number of number of
transactions transactions transactions

2021 Q1

75
Source: Ekspertu anketa, Jeļena Buraja, 2007.
Governmental institutions intendent on implementing these activities to
fight grey economy. What effect will these activities have on the
workforce costs and material costs in the construction industry until
2022?
Scale from “significantly decrease” to “significantly increase”

76
Avots: Pētījums par prognozētām izmaiņām darbaspēka un būvmateriālu izmaksās būvniecības nozarē Latvijā, 2018
Please mark the position on the scale that indicates your expectation on
balance of power between the County Council and Parish Council after
the Territorial Reform.

77
Avots: Ekspertu anketa, Sandra Duļbinska, 2007.
Please evaluate the influence of these factors on a price per sq.m. for an
apartment in a new project.
# Factor Ranking (top) Rating (0-10)
1. Location of the project 2 10
2. Territory 10
3. Building materials used 10
4. Urban infrastructure 1 10
5. Parking availability 3 …
6. Domicile infrastructure
7. Number of floors
8. Mean number of flats per floor
9. Availability of a security system 4
10. Mean height of ceilings in the project
11. Window dimensions
12. Availability of an elevator
13. Interior decoration
14. Balcony, terrace
15. Monthly maintenance bill
16. Loan terms
17. Liquidity of apartments in the project
18. View from the apartment
19. Room planning
20. Area of the apartment
21. Mean number of rooms per apartment
22. Extras (fireplace, air-conditioning etc.)
78
Source: Ekspertu anketa, Jeļena Buraja, 2007.
What could be the optimal government decision regarding the minimal wage
by taking the interests of both the employees and employers into
consideration?
A The State defines a minimal wage proportionally to the average wage in
the country.

B The State differentiates minimal wage in three industries with the

highest rate of grey economy

C Minimal wage is determined per industry based on a general agreement

between the employers and employees or unions

D Other solution, please indicate here:

79
Avots: Ekspertu anketa, Armands Žubulis, 2007.
80
Expenditure by consumption %
purpose
Housing, water, …

Transport

Food and non-alcoholic…

Miscellaneous goods…

Recreation and culture

Restaurants and hotels

Furnishings, household equipment…

Clothing and footwear

Alcoholic beverages, tobacco…

Health

Communications

Education

Total: 100%

81
82
Expenditure by consumption %
purpose
Housing, water, …

Transport

Food and non-alcoholic…

Miscellaneous goods…

Recreation and culture

Restaurants and hotels

Others

Total: 100%

83
 Assignment “Expertise preparation”
1) Examination preparation stage
a) What is the research problem?
b) What is the purpose of the survey?
c) What decisions will be made and by whom?
2) Choosing experts
a) What criteria should be used for choosing experts for this research?
b) What organizations should be involved? How can they be classified?
c) How would the experts be motivated to participate in the research?
3) Creating methodology for your research
a) What method(s) will be used? Why?
b) What historical data will be presented for experts?
c) How will the future be assessed by the experts?
d) How will experts evaluate scenarios and outcomes?
e) On what topics will experts help to make decisions?

84
 Assignment “Expert survey”
1) Create a expert survey form with 5 closed questions and 1 open question
about the research problem and 2 ancillary questions.
a) Have an introduction describing the research object in short and main
objective of the survey.
b) Use at least three different types of answers (ranking, rating, grouping,
direct forecasting, structure forecasting, choice).
c) Have at least one question about assessing the level of impact of at least 10
factors or events.
d) Have at least one question with data and chart provided about the
development of an economic indicator in the past and experts evaluating
possible future growth. Bonus, if scenarios are provided.
2) Submit in PDF format on e-studies. Survey can be re-submitted two
times.

Next stage will include field test by using other students as pseudo-experts.

85