See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/236007499

Ways to study - Criteria for scientific study and design

Chapter · September 2002

CITATIONS READS

21 2,100

2 authors, including:

Theo J M Van der Voordt

Delft University of Technology
521 PUBLICATIONS 3,545 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Master Project Evelien Plijter View project

Management of higher education real estate and facilities View project

All content following this page was uploaded by Theo J M Van der Voordt on 21 May 2014.

The user has requested enhancement of the downloaded file.

1 Page 1 of 14

3 CRITERIA FOR SCIENTIFIC STUDY AND DESIGN

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 2 of 14

3 CRITERIA FOR SCIENTIFIC STUDY AND DESIGN

TAEKE DE JONG, THEO VAN DER VOORDT

3.1 Study and research............................................... 1

3.2 Design related study.............................................. 1
3.3 A definition of science........................................... 4
3.4 Criteria for empirical study (research)................... 4
3.5 Differences between research and design.......... 6
3.6 Design as a science.............................................. 6
3.7 Criteria for design related study ex post............... 7
3.8 Criteria for design related study ex ante............... 8

Could a design be the product of scientific work to be compared with a scientific report? If so,
under which conditions and when? The topic is eagerly discussed both within and outside of faculties
of architecture. On the web-site of the Design Research Society (DRS) there is a lively debate on
what a design study and a study by design really are and when a designer can also be designated a
[1]
scientist. These questions stood central during the 1996 EAAE Congress organised by the Delft
[2]
Faculty of Architecture on the theme ‘Doctorates in Design + Architecture’. In order to answer
these questions we discuss first the terms ‘research’ and ‘study’ and the usual pre-requisites that must
be met for study to be designated ‘scientific’. Next, similarities and differences between designing
and studying are dealt with. Following that, we discuss the usual way in the scientific community of
looking at the criteria for a design to be branded as a product of scientific study. For that purpose a
summary is given of the requirements the Technical University in Delft associates with a scientific
design. Subsequently a short comment is given as well as the rôle it played during the initiative
leading to this handbook of design related study. Finally we give a specimen of criteria for evaluation
of a scientific architectural design (ex post) and of a proposal for a design related study (ex ante).

3.1 Study and research

Study is a collective term for generating knowledge by thoroughly thinking through a problem,
carrying out experiments and collecting, processing and analysing data. When the primary purpose of
a study is to know more, it is termed a fundamental study. When a study is mainly focused on
practical usefulness – e. g. making better buildings, or contributing to more effective and efficient
building processes – one speaks of an ‘applied’ study. If a study is mainly aiming at making a new
product – a prototype of a (industrial) product, a constructive solution or a building method – it is
termed (product)development.
In a sense, each and every one of us deals one time or another with study. Even the student studying
Ernst Neufert’s ‘Architect’s Data’ to ascertain how large a class-room should be, busies himself with
[3]
study. In that case searching, re-searching and retrieving come to the fore. For a study, generating
new knowledge is typical. The contrasts are not always well-defined. Just looking up something may
develop into a lot of retrieving and you might find yourself in a genuine study.

In scholarly circles the term ‘research’ is often employed, rather than the concept ‘study’. Designers
also use both terms; e. g. the ‘Design Research Society’, and its ‘Design Research Newsletter’, the
‘Journal of Design Research’ and the magazine ‘Design Studies’. Another term is ‘inquiry’, as in the
[4]
title of John Zeisel’s book ‘Inquiry by Design’. Related terms are ‘survey’, ‘investigation’ and
‘examination’. This book is trying to conform itself to the British English distinction between study
and research. Research is roughly the empirical form of study. The term ‘research’ originated some
hundred years ago and is used a synonym for the older term ‘study’, especially in the USA. Britons
tend to employ the term ‘study’ for looking for something that does not exist as yet, in a broad sense:
such as for subjects from non-empirical branches of logic and mathematics, but also for studies of
Rembrandt, designers and students.

3.2 Design related study

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 3 of 14

A design does not follow unequivocally and reproductably from a programme like a scientific
prediction repeatable from its basic assumptions, ‘ceteris paribus’. In making a design, the
preliminary investigation and its conclusion, the programme of requirements, direct the solution
only partly. Even within the boundaries of a strict programme, unexpected and
unpredictable alternatives are possible in design. Most design decisions about form, subsequent
structure (set of necessary connections and separations to keep the form) and even subsequent
function (freedom of unexpected use) must be made without empirical evidence. This is most
explicit in building design. The choice of a final alternative is determined by the total context of
the object to be designed. The programme of requirements reflects only a small part of that context.
Location, market and designer (context of invention) belong to the broader present and future
managerial, cultural, economical, technical, ecological, and mass-space-time context and perspective
of the object. ‘Context’ is different on different levels of scale and cannot be foreseen completely in
the programme.

The number of imaginable alternatives for buildings, mostly with a long term multi-functional
programme of (conflicting) demands, is unconceivably large, subject to a combinatoric
explosion (see page 208) of possible forms. Buildings and urban designs have a long period of
use and are earthbound. So they have to function in a changing context that is unpredictable and not
influenced by the programming authority, designer or user. From the viewpoint of durability they
should be able to accommodate varying programmes and the daily changing aims of inhabitants
and users. This quality of building design is called ‘robustness’. ‘Flexibility’ is only part of it. So,
from all artefacts, buildings have the most context sensitive function for use, perception and
market, not to be evaluated without that context and, therefore, hardly comparable to each other
(sometimes even unique).

Even with a comparable programme of requirements, not only the diversity of solutions, but also
the diversity of contexts or perspectives to function in, is very large. Consequently, the diversity of
rational reasons (determined by context) to opt for a final alternative is even larger. So, building
design research often has the character of an n=1 study (case study) with limited general value to
other designs. Design research, based on more examples than one, is often ignored by designers,
because on location many design relevant circumstances appear different from what the examined
examples had in common. The principal often demands a unique design, ‘exploiting’ rare and
distinctive qualities of context. The descriptive interpretation of context by researchers differs
from the imaginative interpretation of designers that stresses possibilities rather than probabilities.

In a University of Technology, designs are made not only intuitively, but based upon study (design
study) and documented, examined and evaluated (design research). Design research concerns
determined objects within determined contexts. ‘Study by design’, in a broad sense, varies either
the object (design study) or the context (typological research) or even both (study by design):

Figure 1 Types of design-related study

The terms from this matrix may be explained as follows:

Design research
Design research describes and analyses existing designs with a known context, often in the form of
comparative study. For that reason it is evaluating study ex post. Not only their function is involved,
but also their form, structure and the way they were made, the design tools employed in each stage
and the way in which they were applied: the making proper.

Typological research

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 4 of 14

Whenever the identical architectural form, structure, technique, function or concept is recognised in
different contexts the notion of a ‘type’ is involved. A type only becomes a consistent model if it has
been elaborated for evaluation by design in a context. A type is a design tool, not yet a model. The
study of such types, their use in the making of designs (a special kind of models) is called typological
research.
Design study
Making a design in a relatively well-known context of potential users, investors, available techniques,
building materials, political, ecological and spatial restrictions, entails many stages of a type of study
termed in this book ‘design study’. If, in the case of grand projects, parts of it are sub-contracted, the
parlance is ‘study for the designing’ or ‘research driven design’.

Study by design
Characteristic for this type of study is generating knowledge and understanding by studying the
effects of actively and systematically varying of both design solutions and their context.

Only if both context and object have been determined (design research ex post), pure empirical
study may be largely depended upon ; although that should also be done with the eye – and
sometimes the hand – of the designer. Empirical (historical) design study is sometimes calling for a
[5]
design re-construction of the design or of the design process.
For the other three guises of study the designing itself must play a crucial rôle, although an empirical
component will remain present in the form of researching inventories, descriptions, programming or
evaluating research. Also, for the broader context of these studies the designing study may be the
object of design.
In the case of a type of study with a determined object or context, the typology and the design
study (daily practice of the profession) a lot of experience has been attained. When both are variable
(study by design), a way out may be found in inter-changing typological research and design-
study. This way, now the object, then the context is varied. However, it can not be excluded that this
study can also stand on its own legs without using both methods of study. The first signs are the
[6] [7]
studies of Vollers and of Frieling . Vollers’ point of departure is the means for design as they
manifested themselves in the usage of Computer Aided Design, from where possible objects and
contexts for application are getting shape. Frieling’s point of departure is a dynamic public weighing
between projects on a small scale (objects) and perspectives on a grand scale (contexts), within the
domain of coming to decisions for the Delta Metropolis. Graduation – when those who graduate are
allowed to determine themselves context and object – has resulted in an archive of experiments, some
more successful than others, exactly in the field of study by design.

Taken together, these modes of study are termed ‘design related study’. Because of the inter-action
between designing and studying, the borderline between both is not always clear cut. Actually a
gliding scale between art and science applies.

Figure 2 Domains according to Van der Voordt

In this diagram studying and designing both feature their own domain; while the two overlap. At that
point one may imagine study activities without design and design without activities of study.

One may also maintain that all empirical study pre-supposes a designed hypothesis (possibly put to
work by way of a model) and a toolbox of research, so that empirical study pre-supposes some

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 5 of 14

kind of designing as well; for the model of the reality (hypothesis) to be checked against that reality
and the toolbox enabling observing, checking and predicting must have been designed earlier
themselves. Without these conditions study cannot be imagined. In this sense the telescope as well
as several branches of mathematics have been designed for modern empirical astronomy; next they
were a condition for it; and finally it pre-supposes them. In that way design is always pre-supposed
in study and research.

Figure 3 Domains according to De Jong

If these pre-suppositions are forgotten one forgets as well that their reliability is always open for
discussion.

Experimental empirical research can also produce unexpected possible futures, however, only
[8] [9]
because it pre-supposes the design of the experiment and its instruments. Bacon , cited by Kant
in his Preface (Praefatio), states that science has not to be concerned as opinion, but as work (‘…non
Opinionem, sed Opus…’). Elsewhere Bacon states that nature has to be forced to answer the question
of the scientist. The scientist has firstly to design the experiment in order to produce improbable
events in some future. Kant states in his preface that scientists before Bacon understood that human
reason only recognises what it produces itself by design (‘…das die Vernunft nur das einsieht, was sie
selbst nach ihrem Entwurf hervorbringt.’).

Thomas Kuhn, who created the ‘paradigm’ concept, associates his description of scientific
[10]
revolutions closely with making tools and instruments. Van der Meer, the Dutch engineer who
designed for the Geneva Cyclotron the type of improvement that caused the discovery of crucial new
facts in nuclear physics, got the Nobel Prize, not for those facts, but for bringing them to light. When
one regards mathematics as a tool box for working models, mathematics and models built with it are
also design instruments in need of design. This happens during empirical study and is caused by it.
All statistical checks came into existence this way. In its usual, more narrow sense, designing does
not relate to models simulating probable futures, but possible futures; even if they are not likely. This
narrower sense of ‘designing’ is emphasised in this book.
3.3 A definition of science
An important question is now: when may a study be termed ‘scientific’? And especially: when is
design-related study scientific? In order to answer this question, one must first define what ‘scientific’
entails. Although several definitions are current, over and over again a number of properties are
returning in almost all definitions. This leads to the following definition:

Science equals any collection of statements that features a reliable relationship to reality, a valid mutual relationship and
a critical potential with regard to other statements in the same domain.

The term ‘reliable’ may not only relate to ‘true’ and ‘probable’ in empirical sciences, but also to
‘feasible’, ‘working’, and, therefore, ‘possible’ in technical sciences. In both cases ‘reliable’ pre-

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 6 of 14

supposes ‘verifiable’, ‘documentable’ and by the same token a public domain and accessibility of
sources and methods (see page 92).

The term ‘valid’ (see page 92) pre-supposes the validity with which lines of reasoning may be
constructed out of propositions and propositions out of statements (logic, see Chapter 23 , page 189)
or the completeness with which the context is taken into account in a proposal or demonstration. This
last criterion is particularly important for technical sciences. Completeness not only concerns a larger,
but also a smaller context of gaps filled with tacit suppositions. Incomplete knowledge is half truth:
incomplete technology is failure. In empirical science the completeness may be partly covered by the
'ceteris paribus’ pre-supposition (“for so far the rest is equal”); in technical sciences this is seldom
feasible. Completeness can never be reached entirely and is by the same token a relative concept, but
a given proposal (like a design) may be more complete than another (like considering more contextual
effects of the proposal).

The term ‘critical’ pre-supposes that it is possible to make a statement that refutes other (e.g.
‘popular’) statements and that it is also possible to refute the statement itself (see paragraph 3.4, open
to criticism, able to criticise).
The term ‘domain’ relates to the collection of subjects from reality evoking a statement. The term
‘statement’ encompasses propositions about a probable reality as well as proposals for a possible
reality. The term ‘reality’, therefore, has a wider meaning than empirical ‘existing reality’ (think of
notions like realistic e.t.q.). The concept ‘potential with regard to’ is wider than the concept
‘connected to’.
[11]
Many other definitions are current. We choose this one in order to give technical sciences a place
next to empirical sciences (here denotated by ‘empiry’). Scientific study is the activity needed to
arrive at scientific statements. Not all study is scientific. It is characteristic for scientific study that its
results are reliable, valid, capable and open to criticism. In the case of empirical research these criteria
may be further specified.
3.4 Criteria for empirical study (research)
Reliability
Empirical reliability entails that repetition of a measurement under unchanged conditions renders
identical results of measurement. A ‘face value’ assessment of the constructive quality of a building is
less reliable than measuring its physical shortcomings. ‘internal reliability’ is the parlance if the
same investigator judges a particular situation more then once and each time comes to the same
conclusion.
‘External reliability’ means that different investigators judge the same situation with the same
results. Only if reliable instruments of measurement are used can a sufficient degree of objectivity
apply.

Reliability necessitates formulation of a criterion of objectivity. Those who study, or are engaged in
carrying out a particular project study, should strive to keep personal opinions from influencing the
study and refrain from making personal value judgements. Someone else should be able to get the
same results when using the same method. The instruments of measurement are thus severely tested.
In a study of notions hard to measure, like architectural quality, or importance of a specific
intervention in urban design, complete objectivity can hardly be realised. A careful description of
concepts and measuring instruments, additional independent measurements (e. g. repetitive
measurements by different investigators) and intermediary verification of findings by third parties
does increase the probability of objectivity, often alluded to as ‘inter-subjectivity’.

A second criterion derived from reliability is verifiability. In order to qualify as a scientific study
the structure of the study, the collecting of data, analysis of the material and interpretation should be
made comprehensible to outsiders: it should be clear how the investigator reached his conclusions.
This enables scientific debate. What is more: it offers other investigators opportunity to repeat the
study; in different times, at different places. Naturally the requirement of verifiability requires a clear
presentation and publication of the study.

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 7 of 14

While interpreting the data of a study and drawing conclusions, it is almost impossible to exclude
personal (pre-)suppositions. Therefore, it is recommended to separate in the report of the study as
much as possible the factual study results from the interpretation and conclusions. This leaves the
possibility open to reach different conclusions based on the same material.

The term ‘value free’ is closely related to objectivity. This entails that the end – scientifically
founded knowledge – justifies the undertaking of study, even if its results would clash with prevailing
norms and values, or if the study would work out negatively for segments of a community. These
days, practically everyone agrees that study can not, and should not, be value-free. Norms and values
are important while choosing the inquiries of the study and the application of the study results. This
does not preclude that within the given context the reasoning should be valid.

Validity
A second primary criterion for scientific study is validity. Amongst others, it means that what is
measured is what is reported to be measured. Does measurement of temperature and humidity entail
comfort? Who studies the effect of the presence of a ‘major domus’ (caretaker) on the intensity of
vandalism in an apartment building by way of asking his opinion on it runs the risk to study rather the
legitimacy of his appointment than the real effect. Measurement of costs of repair before and after his
appointment is a more valid instrument of measurement.

This leads to the derived criterion that the investigator should think about the way how he might find
efficiently and effectively the answer to the study question: he is looking for a methodical way,
allowing research. Detours to reach a conclusion should be avoided. They might result in mistakes in
the reasoning difficult to verify. Yardsticks are efficiency (using not more measurements, means or
pre-suppositions than is necessary) en effectiveness (the method should be the answer to the
question of the study). This requires thorough analysis of the problem, an inventory of sources of
information available, a clear and unequivocally formulated statement of the problem and the purpose
of the study, and critical reflection on the most appropriate study methods.

Over the years an extensive methodology of research has been developed. Presently there is a
large variety in methods and techniques of research. For a survey and reflection on advantages and
[12]
disadvantages as well as considerations as to selection we refer to the professional literature. A
methodological approach, by the way, does not mean that each step to be taken may be thought out in
advance. Often progressing insight manifests itself while new steps are developed during the study.
Additionally, rather accidental finds occur, some of them inspired by creative ‘flashes’ of insight:
serendipity.

Open to criticism, able to criticise

A study may be reliable and valid, and yet not assessable. An important criterion of demarcation is
Karl Popper’s ‘refutability’ (openness to criticism, possibility of negation, falsification) of study
[13]
results. At stake is the possibility of counter-examples undermining the general conclusion.
For a long time the opinion prevailed that it would suffice, if the study results could be checked by
others (verification). However, certain statements have been formulated in such a way that they may
be verified always: speaking for themselves and by themselves. A well-known example is:
“Everything is tripartite”. When an investigator shows in refutation a safety match, the object is
broken into three by way of verification. Statements of that type are adding nothing scientific to
existing knowledge, according to Popper. In contrast an example of refutable knowledge is
Einstein’s theory of relativity, stating that light is bent by mass. Only in the years following
publication of the theory this could have been refuted, if at a solar eclipse it would show that a star
behind the sun would appear precisely on the moment predicted by the calculations of its course. The
star appeared a little earlier, verifying the theory of relativity. Essential in this is that at that moment
the possibility existed that the theory was refuted. The proposition was risky

In adddition to verifiable and open to criticism by third parties, scientific statements should also be
critical themselves. The falsification principle of Sir Karl Popper is not only a passive, but also an

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 8 of 14

active one. This means that science is open to both confirming and refuting existing opinions and
views, for the time being seen as hypotheses. By checking them empirically or in terms of logical
consistency, these hypotheses can be unmasked with more certainty as true or false. The potential to
get away from myths is an important characteristic of science. This brings the criterion of scientific
relevance into view.
Scientific relevance
Scientific study should widen and deepen development of the scientific discipline. The renewal or
deepening may comprise contributing to the development of theory (generation of new knowledge,
refuting or amending existing views), new methods and techniques of study, policy instruments and
product development. Study limited to inventory of data is widely disregarded as scientific study.
Even if the criteria of a methodological approach: reliability (objectivity, verifiability) and validity are
honoured, scientific relevance is low, if it cannot criticise any existing suppositions. By itself this has
nothing to do with social relevance or ethical admissibility: the contribution to improvement of
the quality of life. At a level of low scientific relevance this might be high.
3.5 Differences between research and design
In spite of kinship between research and design differences apply:
• The primary product of research is general knowledge in the form of probability. In a more narrow sense it is also the
description of existing reality or truth belonging to it. This knowledge may, or may not be applied in (design) practice.
The primary product of designing is the representation of a possibility; also if it is not a likely one. A design
demonstrates what is possible and thus may become reality. Knowledge of what is probable is always incorporated in a
design, often implicitly; e. g. that a brick can endure a well defined pressure.

• Research deals mainly with analysis; with a design process the focus is on synthesis. Analysis (etymologically
‘loosening’) severs a phenomenon from circumstance (context, set of conditions) and components (reductions)
that are different elsewhere, in order to retain what may be made comparable fit for study (operational). That enables
(ceteris paribus) statements that may be generalised. Synthesis integrates diverging requirements and interests, but
adds in passing also conditions leading to new consequences for use and experiencing. Continuously the design
process offers new opportunity, not be described ex ante and often not in words.

• Research strives towards development of knowledge that may apply in several contexts. Research deals mainly with
reality and experiences (empiry). Thus, this research is empirically orientated and its thrust is towards probability. What
the reality should be like may also be subject of research. From social goals and norms (points of departure) one
reasons backwards to means for reaching them (normative study). The personal opinion of the researcher, however,
is not allowed to play a rôle in the interpretation of the data of the study (objectivity). In order to restrict an explosion of
possibilities caused by combinatorics, designing is almost by definition coloured by personal preferences (selective
attention for empirical facts) of the designer (subjectivity). Designing may be normatively biased; a characteristic it
shares with the arts.

• Usually different methods and techniques are employed in research and design. For instance: research of the
literature, polls, interviews, measurement of characteristics of a building, and experimenting are common study
methods. Common design methods include usage of metaphors, adapting existing types, or application of design
[14]
principles. Lynch proposes for example design principles in order to create a ‘legible built environment’.

3.6 Design as a science

Given what was suggested until now, it is obvious, that designing as an activity, and a design as a
product of this activity can only pass muster as a science, if the usual criteria for scientific activity
have been obeyed. In an Advice of the Working Group Criteria Designing Disciplines (December
[15]
1999) this position is taken. If a (tentative) design applies for being branded as result of scientific
activity, it should comply with general requirements put to the scientific approach, to wit: inter-
subjectivity, reliability and verifiability in an empirical sense. ‘Inter-subjective’ was defined by that
group as ‘interpreted by different people in the same way’. ‘Reliable’ means here that the design
demonstrates under different circumstances determined ex ante, behaviours that also determined ex
ante. Verifiable points to the description of the design in terms of the grounds on which the decisions
have been taken, including validity and tenability. Additionally, ‘verifiable’ includes, that the design
can be specified according to concrete situations and can be generalised to possible applications in
different situations or contexts. The working group derived the following concrete criteria:

• Novelty vis-à-vis state of the art of technique and originality

• Design methodological approach with a subjectivity that is argued

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 9 of 14

• Construction and materialisation in reality, if applicable

• Evaluation of actual performance of the design, compared to the performance intended
• Integration of design, development and study
• Integration of designing on different levels of scale (vertical integration)
• Integration of partial designs and aspects (horizontal integration)
• A vision on future development of the domain – in terms of design, discipline and science – the programme deals
with.

To judge a design on these criteria, its presentation should include a description giving attention to
these aspects.

During the EAAE Congress, mentioned earlier, ‘Doctorates in design and architecture’, comparable
criteria emerged. Many scientists and designers agree that a design as a produce of scientific work
should be based on a transparent process that may be assessed; a logically valid argumentation and
accessible source of documentation. Originality, validity, economical use of means, clarity as to the
underlying values and openness vis-à-vis verification and refutation are widely accepted criteria.
Nevertheless these conventional criteria allow some remarks. They have been strongly suggested to
non-designing, truth directed disciplines with a preference for general knowledge. This may be
a consequence of the fact that a lot of design related study has been done by social scientists,
organisation experts, historians and technicians, not by the designers themselves. There must be
something left over concerning design itself.

Restrictions to reliability
For multi-functional facilities or facilities used during a very long period, leaving open more
possibilities of usage than foreseen, causes the requirement of reliability to be discussed. Someone
immediately sees these possibilities, someone else after some time. A great number of possibilities of
use and freedom to choose between them restricts reliability. The value of a multi-functional
design sometimes increases with the number of possibilities of use in different contexts
(robustness). By the same token, a conflict between this robustness and the reliability of the
assessment may exist when evaluating an architectural object. Mono-functional facilities like a public
water closet on the other hand may be evaluated reliably up to a point.

Restrictions to validity
Usage of an architectural design is, then, even more context sensitive than, for instance, usage of a
petrol engine and consequently difficult to generalise. What works in one spatial, ecological,
technical, economical, cultural and political context needs not to work the same way elsewhere.
While architectural designers are hired particularly for solving, in a unique way, problems connected
to place and context in a dynamic and many-faceted society, the classic empirical scientific striving
towards statements that can be generalised may be frustrated.

There is an important distinction between the modalities “to be” and “can”. Everyone senses the
incorrectness of the statement “That is not so, therefore it cannot be done this way”. Between
[16]
empirical and technical sciences there is an important difference in modality. What is
probable inter-subjectively is per definition possible, but what is possible is not always probable.
Improbable possibilities are seldom inter-subjective, as long as they have not been demonstrated
by realisation. Before demonstration just a belief applies (with the possibility of realisation).

Designing concentrates on discovering these improbable possibilities. This puts the criterion of inter-
subjectivity into jeopardy. Even after realisation proving the possibility of spatial construction inter-
subjectively, the use of the facilities built in its parts is in principle unpredictable, as long as one
believes in the freedom to choose on behalf of users. The value of an architectural design is
determined by the degree in which the design offers its user new possibilities to choose from. A home
does not cause homing, it just makes homing possible. By the same token, design thinking is less
[17]
focused on causality than on conditionality.

Restrictions to evaluative potential

A final remark regards verifiability in the case of categories which are not to be compared. Each and

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 10 of 14

[18]
every design features elements like usefulness, beauty and sturdiness, that cannot be compared.
Nevertheless, it is precisely the way in which these incomparable categories have been unified
consistently within a specific context that determines the value of the design. Before the building
can demonstrate its value on the market ex post, the validity of considerations between these
principally incomparable categories and defending them can not be objectified. Even if a building
proves its value this way, this does not ensure that the experiment will lead to the same result
somewhere else. In addition, it often happens that context specific reference material is lacking
against which a design before execution (ex ante) can be checked, when thorough evaluations of
comparable cases after realisation (ex post) are absent.
3.7 Criteria for design related study ex post
A scientific design should not be required to meet the criterion that its result is probable, as is the
case of a study carried out in an empirical context. This puts a number of scientific criteria
mentioned into jeopardy. There is even no need to require that a design is desirable, while
improbable innovations often may not be imagined before they are proposed in a design. This is a
crucial function of scientific design. As long as one does not know what is possible, one cannot
know what one wants.

However, one must require that realisation of the design in one context or another is possible. The
question is whether it must be socially possible at the same time. What is socially not feasible at
present may become so when the possibilities have been brought into light. Even the question
whether a design is economically feasible at present is no scientific yardstick, although a
perspective may be required within which realisation may become possible at a certain time.
Associated with this one should not require that the design has also been developed in a goal-
directed way based on a statement of problems and aim ex ante (programme of
requirements or brief). Rather, paradoxically, this pre-supposes an imagination of the result ex ante
(hypothesis). It may be an experimental study orientated to a means with uncertain functions as
[19]
a result.

It is in order to ask which criteria remain. There are less of them than in empirical study, but from a
viewpoint of the requirement of completeness there are also more. A suitable and extensive survey
[20]
has been given by Eindhoven Technical University. We restrict ourselves here to a minimum
based on experience with evaluating matriculation designs and designs in other educational projects.

The following general criteria for technical university design on the level of a dissertation could apply
to all technical sciences:

A. The scientific design should be understandable to others in the culture given so that it can be
judged by them (to be expressed in a rich way) and, therewith, open to control, criticism
[21]
and refutation. The scientific design has been drawn up, documented and discussed by the
designer with a clarity sufficing for a potential refutation. A possible refutation by third parties
does not need to be a blemish on the proof of academic competence. To this criterion belongs
the possibility of retrieving the sources on which the design and its argumentation is based.
The requirements of the design drawing as a document to be judged scientifically are further
detailed in Chapter .

B. The academic design should bring possibilities to light that are essentially new (‘invention’ or
‘find’). This novelty value should show by comparing it to an added, accompanying, inventory
of similar existing designs in order to provide the person evaluating with the wherewithal for his
task. The technical-scientific design should bring improbable possibilities, those not to be
deducted by mere prognosis. With this, the novelty value exceeds new knowledge (discovery)
of phenomena at empirical study, probable by themselves.

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 11 of 14

C. It should be made acceptable that these possibilities are presently technically viable, at a future
time economically and in any perspective as well as socially. The design should include a
vision on the range of technical execution and social implementation in that perspective.

D. The design should include an effect analysis (for an evaluation ex ante, see Chapter ) of this
book). This analysis should minimally include a physical (spatial, ecological, technical) and a
social (economical, cultural, political) effect in different perspectives. These effects may be
intended in the first stage (potential of the site, intention, social need for the programme) and
unintended afterwards. The effect analysis comprises particularly the unintended effects; for
the intended ones, relevant during the comment on the design and the argumentation, would lead
to circular reasoning. Unintended effects may be judged negatively afterwards in certain
perspectives. They cannot be a basis for discrediting design and study competence. If
demonstrated by the designer himself, on the contrary, it should be regarded as a proof of a
scientific propensity. Additional illustration on effect analysis is given in Section .

E. The intended social effect should be admissible in terms of ethics. Of the unintended effects the
ethical admissibility should be checked.

3.8 Criteria for design related study ex ante

One should also distinguish – next to evaluations before and after execution – between evaluations of
a design related study ex post and a proposal for such study to that effect before a design has been
made. Judging a study proposal in advance is more difficult since there is less material at hand: it is
just a promise of a study. In spite of that many study committees daily face the task of judging the
potential and relevance of study proposals. Authorities distributing budgets always wrestle with the
difficulty that a study proposal can not yet be judged on its result: the criteria of paragraph 3.7 can not
be applied.

This book was written in the framework of the pilot project ‘The Architectural Intervention’; a
[22]
number of workshops where teachers and students study and publish together. The project
proposals have been judged ex ante as a study proposal by the Methodology Committee of the pilot
project for admission to the pilot project according to the following criteria.
A Affinity with designing
Affinity with designing can be shown from at least two images (photo’s, drawings), which are
somehow comparable, or which in previous studies stood for reference or design model for an
important field of interest from the participating researchers/graduate(s) and for the studio as a whole.
The images may be a portrayal of different locations (at any scale), but also from the same location in
two development phases. A correct way to do this is an entry in the Interactief Beeld
[23]
Archief (IAAI, see the Internet site).
B University latitude
University latitude can arise from a specification of the context and the perspective of the research,
from participating disciplines and contacts with (inter) faculty research and graduating in the studio.
An external referee can take on the rôle of an imaginary assignment initiator if (s)he is prepared
to remain involved with the research/ graduation up until the final publication.

C Concept formation and transferability.

Concept formation with regard to design-orientated thinking follows the course of describing to
accurate formulation in key-words. Concepts are defined, or, if they cannot yet be defined, receive a
conditional ‘position’ (see page 41). Implicit, not commonly accepted assumptions such as the
supposedly self-evident conditional and causal connections are made explicit. Scale
falsification and overlapping concepts are avoided. What is stated as desirable, possible or
probable is differentiated as such. That which can be expressed in an image, is not solely verbally
expressed. In this case an image is made accessible by means of an unambiguous legend or
drawing code.

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 12 of 14

D Retrievability and Accumulating Capacity

A correct way to reach retrievability and accumulating capacity is a personal internet site for
each individual researcher with respect to the research/graduation and a site for the researcher studios
and projects as a whole.

Referring to other authors

Provisional literature lists should be equipped with a number of key-words per title, from which it
can be deduced that the proposal makes an input, uses, assesses or attempts to reject.

Making one’s own publications retrievable

An accurate, distinct and sigificant title for the project and for the sub-researches within the project
should be found. A determination of identity, not a solitary example but the placing within or beside
an existing research, as well as a series of key-words, which reproduce the fascination,
knowledge and the presentation of the researchers’ questions are required. Reference words or
distinguishing features relative to the design in the drawings and sub-projects, allow them to be
retrieved in an image archive by differently orientated researchers. A few of these key-words can be
elaborated upon in an explanation, which reproduces the theme, the study question and the study
approach.

E Methodical accountability and depth

Such accountability, of the way in which (using which method) one will arrive at a result, should
make possible that someone else using the same method can (not: will) arrive at a similar outcome.
There should be evidence to which extent the study is aim-orientated or means-orientated,
empirically orientated or orientated towards the development of means of design. In the first
case the starting points must contain a problem proposition and an objective proposition. An
hypothesis and a research method must be specified.
A collective starting point for means orientated study can be perceived as follows: a location or a
category of locations should become more meaningful using the design - for whatever purpose -
than the current interpretation. The hypothesis of means-orientated research is always ‘There are
means of design for different purposes to come’, which must be proven by the design.
The depth of means-orientated research can be proven with at least two criteria of each drawing,
whereby they can be compared. Their differences or transformation can be evaluated and an
explanation of the manner in which (method used) they can be compared.
This explanation can concentrate on the framework, the research field to which the comparison is
reduced (for example a building physics, history and functionality comparison). This can also be used
in order to specify which internal or external study programmes will be linked to.

F Ability to be criticised and to criticise

Ability to be criticised (ability to be refuted) offers others the opportunity to selectively make
progress by building upon technical scientifical know-how and knowledge (accumulation) obtained
through study. Statements are only of a scientific interest when they are bold and do not solely use
risk-free citations, self-evident aspects or even clichés, on the contrary, statements must
question these. This daring must not only be apparent in the project design but in particular during
execution.
Ability to be criticised can be shown from a readiness and initiative to expose the results in their
consecutive phases, to publish them for instance on the internet in a refutable manner, and in this
way open them to criticism in all phases of the research even though these phases are unripe.
Drawings and arguments must not conceal their weaknesses.
G Convergence and limitations
• A proposal for the nature of the end product is required with a summation of the sub-projects. Questions to be
answered in the proposal are:
• Who is ultimately responsible, who takes part, how often do they meet, what do they organise, how do they
divide the common tasks, how is a synergy created, which facilities are desired?
• How is it to be represented (on the website, in book form, in a conference)?
• How do the sub-projects converge?

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

1 Page 13 of 14

This methodology book tries to help.

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

 1 Page 14 of 14

[1] Participating in the DRS newsgroup is possible by mailing a message ‘subscribe drs’ to [email protected]
[2] The European Association for Architectural Education (EAAE) is an international organisation of European educational institutes in the field of architecture.
Its aim is exchange of ideas and results related to architectural education and study. For the benefit of the EAAE conference on Doctorates in Design +
Architecture a conference book (‘Book of Abstracts’) has been published. The proceedings comprise two volumes, edited by Theo van der Voordt and
Herman van Wegen: Voordt, D.J.M. van der and H.B.R. van Wegen (1996) State of the Art, Proceedings of the Doctorates in Design and Architecture
conference, vol. 1; Voordt, D.J.M. van der and H.B.R. van Wegen (1996) Results and Reflections, Proceedings of the Doctorates in Design and Architecture
conference, vol. 2.
[3] Kuypers, G. (1984) ABC van een onderzoeksopzet; 2e dr.
[4]
Zeisel, J. (1985) Inquiry by design: tools for environment-behavior research.
[5] Jong, T.M. de and J. Achterberg (1996) 25 plannen voor de Randstad. This study compared twenty-five designs by re-design for one million inhabitants.
[6]
Vollers, K. (2001) Twist & Build, creating non-orthogonal architecture. See also Chapter 55 in the present book.
[7]
See Chapter 56 in the present book.
[8]
Bacon, Francis (1620) Instauratio magna; The Great Instauration. Bacon, Francis (1645) Novum organum scientiarum.
[9]
Kant, I. (1787) Critik der reinen Vernunft.
[10] Kuhn, T.S. (1962) The structure of scientific revolutions. Dutch translation: Kuhn, T.S. (1972) De structuur van wetenschappelijke revoluties.
[11] See: Kroes, P.A. (1996) Ideaalbeelden van wetenschap, een inleiding tot de wetenschapsfilosofie.
[12] Readily accessible methodological books include: Baarda, D.B. and M.P.M. de Goede (2001) Basisboek methoden en technieken. Baarda, D.B.,
M.P.M. de Goede et al. (1996) Basisboek open interviewen. Baarda, D.B., M.P.M. de Goede et al. (2001) Basisboek kwalitatief onderzoek. A little less
recent, but with more examples on the field of construction is the book by Korteweg, P.J., J. van Weesep et al. (1983) Ruimtelijk onderzoek: leidraad voor
opzet, uitvoering en verwerking. Especially for constructors is Zeisel, J. (1985) Inquiry by design: tools for environment-behavior research. For a systematic
approach to formal (plan)analysis see, for instance: Clark, R.H. and M. Pause (1985) Precedents in architecture.
[13] Popper, K.R. (1963) Conjectures and refutations: the growth of scientific knowledge. Partly translated in Dutch: Popper, K.R. (1978) De groei van
kennis.
[14]
Lynch, K. (1960) The image of the city.
[15] Werkgroep Beoordelingscriteria Ontwerpdisciplines (december, 1999) Advies van de Werkgroep Beoordelingscriteria Technische Universiteit Delft.
[16] The concept m o d a l i t y has a well-described function in philosophy (A r i s t o t l e, K a n t), l o g i c (m o d a l l o g i c) and l i n g u i s t i c s (verbs of modality)
to express the difference between probability, likelihood, possibility or desirability. See also Chapter 23 on logic .
[17] Jong, T.M. de (1992) Kleine methodologie voor ontwerpend onderzoek. This book further develops this c o n d i t i o n a l i t y in a technical sense. It is
shown that c o n d i t i o n a l i t y implies also a s e q u e n c e that was ascribed previously exclusively to c a u s a l i t y. A provisional translation is available at:
http://www.bk.tudelft.nl/users/dejongt/internet/008000000.htm.
[18] Vitruvius and M. Morgan (1960) Vitruvius: The ten books on Architecture.
[19]
In the case of a m e a n s - o r i e n t a t e d s t u d y the design solutions are generated first, and next is studied which aims could benefit from them (e.g. ‘
designing as an art to seduce’).
[20] BCO, Bestuurscommissie Ontwerpers- en korte Onderzoeksopleidingen (1994) Op weg naar promotie op proefontwerp.
[21] ‘Culture’ is defined here as the set of tacit pre-suppositions while communicating; for example the meaning of the units of the legends in the
drawing.
[22] See publications of the Architectonische Interventie.
[23] http://iaai.bk.tudelft.nl/

mhtml:file://D:\word\publicaties Theo\Ways to Study\WaysToStudy_H3_CriteriaScien... 19-8-2009

View publication stats