Department of Management Science and Information Systems

Semester 2 2002

INFOSYS 751 RESEARCH METHODS II

INSTRUCTOR: DR JIM SHEFFIELD

COURSE DESCRIPTION

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ Department of Management Science and Information Systems INFOSYS 751 SC 2001 RESEARCH METHODS II COURSE INFORMATION 1. CONTACT INFORMATION Dr Jim Sheffield, Room 1.32 Old Choral Hall, Phone x.7157 Email: [email protected] Office Hours: Thursdays and Fridays 12:00 to 1:00pm and by appointment or drop by. Thursdays 10-1 A-L, Fridays 9-12 M-Z. Students will form teams of size 6. These teams may cross streams providing teams attend class together and the streams are of equal size. Room 336 Executive Programmes Building 810 Short St

Instructor

Time Place

2.

DESCRIPTION

It is the mark of an instructed mind to rest satisfied with the degree of precision which the nature of the subject admits, and not to seek exactness when only an approximation of truth is possible.. Aristotle. Theres no point in being exact about something if you dont even know what you are talking about. John von Neumann. Things should be made as simple as possible, but no simpler. Einstein. This paper is the second of a two-part sequence on research methodology, and is designed to be complementary to Research Methods I. Both papers prepare students to pursue research and development in information systems and/or management science and operations management. Both papers are compulsory for all PhD, Masters and Honours students in the Department of MSIS. Whereas Infosys 750 Research Methods 1 focused on the application of quantitative techniques for data analysis, Infosys 751 Research Methods 2 focuses on the qualitative methods that support learning and measure the quality, value and coherence of the outcome. Research Methods 2 introduces qualitative methods for the exploration, description and evaluation of concepts, information and knowledge. The classroom learning process involves participation in a social world, observation of a material world, and experience of a personal world for all of which the exactness afforded by numbers is illusory. 2

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ 3. AIM This course is designed to assist MCom students to complete their theses and BCom (Honours) students to enrich their dissertations. Theory guidelines are kept relatively simple and parsimonious so that time is available during the course for practice. Students apply their book learning by participating in collaborative action research. Students are expected to be willing and able to enunciate what it feels like to eat their own cooking. Students may also experience, and may need to resolve, some minor issues relating to individual motivation and teamwork. 4. LEARNING OBJECTIVES
THE MAXIMUM OF 15 POINTS IS ALLOCATED AS FOLLOWS:
1 6

1. THE IDEA Why will pursuing this idea add value?

6. THE PAYOFF Why did pursuing this idea add value?

To reduce equivocality the..

2. THE OBJECTIVE What is our (my? your?) objective?

ONE POINT FOR EACH BOX AND DIRECTED LINE

2

..key reference is Habermas

5. THE RESULTS What is the evidence that we have met our (my? your?) objective?

5
PLUS AN EXTRA POINT EACH FOR BOXES 4 AND 5

To reduce uncertainty the..

3. THE ACTION PLAN How will we meet each sub-objective? Start. Planning.

..key reference is McGrath

4. THE PLAN IN ACTION How strong is the evidence that we have met each sub-objective? Implementation. Finish.

Links between Intention & Outcome.

THE BASIC 15 POINT METRIC FOR THESIS QUALITY

Upon completing this course students, individually and as part of a group, will be able: 1. To employ a consistent framework (viz, the Systems V model) to analyze the chain of evidence in any completed research in the Faculty of Business and Economics. 2. To identify the strategic choices that reduce equivocality about three worlds (viz, Social, Material and Personal see Habermas 1984; 1987) an/or world views (viz Interpretivist, Positivist and Critical - see Cavana, Delahaye and Sekaran 2001). 3. To identify the strategic choices that reduce uncertainty about research methods (McGrath 1982), especially methods such as field studies, proof of concept, and action research that are commonly used in the Department of MSIS. 4. To apply the Systems V model to evaluate the quality and coherence of business research to meet one of three objectives (exploration, description or evaluation) in each of three

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ broad application areas of research in the Department of MSIS (concept engineering, information engineering and knowledge engineering). 5. To undertake Interpretive Research in Concept Engineering. The nature of the engagement necessary to understand why human subjects in a particular context may choose to use (or not use) artifacts or tools such as those provided by Information Technology. The self-organizing, and emergent behavior by which we humans enact a social consensus on soft systems. The complex web of language, meaning, social practices, rules and resources that link beliefs and actions, intentions and outcomes in a particular context. The image of the window frame. 6. To undertake Positivist Research in Information Engineering. The process engineering or software engineering research techniques and technologies that deliver proof of concept or proof by existence. The key roles of (formal) language, specification and (assumed) social context in the iterative development and demonstration of how an emergent software object or art/i/f/act behaves. The image of the mirror. 7. To undertake Critical Research in Knowledge Engineering. The action research processes to build personal commitment to a social consensus for informed, emancipatory change. Inquiry into Why, What and How individuals and groups seek empowerment, and relief from systems crises, via Information Technology. Does technology resolve or create systems crises? Are there winners and losers? Who decides? Who cares? How do we tell? Why do we care? The image of the lantern. 8. To develop integrated sets of quality indicators for research in the Department of MSIS. Teams will make presentations to the class of the results of benchmarking (i.e., comparing and contrasting) multiple, conceptually related theses against the quality metric they have developed. Informed class-wide debate on best research practices is expected from all students. This dialogue should lead to an understanding of why some theses are of high quality and should be used as a reference model for other researchers. 5. ASSESSMENT In-Course Assessment 50%

Weeks 1-4: Interpretivist Research: Exploring Concept Engineering 15% Most but not all of the evaluation is based on the oral presentation and report in week 4 Weeks 5-8: Positivist Research: Describing Information Engineering 15% Most but not all of the evaluation is based on the oral presentation and report in week 8 Weeks 9-12: Critical Research: Evaluating Knowledge Engineering 15% Most but not all of the evaluation is based on the oral presentation and report in week 12 Occasional quizzes, exercises and/or team mini-tutorials (5@1%) 5% Weeks for exercises and/or team mini-tutorials only to be announced in advance The grade points associated with the in-course assessments will reward active exploration of the opportunities presented and full participation in class meetings. The in-course activities target the same skills as those assessed in the Final Exam. The in-course assessments are expected to be highly correlated with, although higher than, the mark earned in the final exam. 4

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ Final Exam 50% A copy of a Final Exam for a recent year is included at the end of this Course Description. 6. LEARNING AND TEACHING STRATEGIES Compared with the quantitative methods covered in Research Methods1, the qualitative methods studied in Research Methods 2 are easy to state in procedural terms but deceptively difficult to understand and to perform in practice. To the uninitiated, the concepts appear both vague and overly abstract. It appears difficult to make them concrete, to operationalize and internalize them. A balance is sought between the theory of qualitative research methods and actual hands-on practice in simple yet relevant research activities. Because qualitative methods require personal experience to develop know how, Research Methods 2 employs an experiential and skill-building approach. Students work, individually and in teams, on a variety of exercises. These hands-on exercises provide opportunities for participation, personal experience, surprise, dialogue, observation and self-knowledge. A key element is the personal frame-breaking experience that exposes hidden beliefs and structured contradictions. 7. LEARNING RESOURCES

Required Texts 1. Cavana, Delahaye and Sekaran. Applied Business Research: Qualitative and Quantitative Methods. Third Edition. Wiley. 2001. This text should be brought to every class, including the first, as well as the Final Exam. 2. Yin, Robert K. Case Study Research: Design and methods. Second Edition. Sage. 1994. (Expensive! Look for second hand copies.) Theses Copy 2 of Management Science and Information Systems Department theses are to be checked out of the library and brought to class on a regular basis. See the list of theses at the end of this document. Readings on Research Methodology (i.e., conceptual frameworks for discussing, denoting and expressing research participation, observations and experiences): Required and/or Highly Recommended Readings (Extracts are supplied in class) 1. 2. 3. Checkland, Peter and Sue Holwell. Information, Systems and Information Systems. Wiley. 1998. Cohen, Louis and Lawrence Manion. Research Methods in Education. Fourth Edition. Routledge. 1994. Collins, Jeff and Howard Selina. Introducing Heidegger. Icon Books Ltd. 1999. 5

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ 4. Crotty, Michael. The Foundations of Social Research: Meaning and Perspective in the Research Process. Allen & Unwin. 1998. 5. Garvin, David A. Learning in Action. A guide to putting the learning organization to work. Harvard Business School Press. 2000. 6. Gellatly, Angus and Oscar Zarate. Introducing Mind & Brain. Icon Books Ltd. 1999. 7. McGrath, Joseph E. Martin, Joanne. Kulka, Richard A. Judgement Calls in Research. Sage Publications. 1982. This is the second book in a series entitled Studying Organizations: Innovations in Methodology. 8. McLachlan, Ron. Management initiatives and just-in-time manufacturing. Journal of Operations management, Vol. 15, pages 271-292. 1997. 9. Nunamaker Jr, Jay F., Minder Chen, Titus D.M. Purdin. System Development in Information Systems Research. Journal of Management Information Systems, Winter 1990-91, Vol. 7, No. 3, pp. 89-106. 10. Pinsonneault, Alain and Kraemer, Kenneth L. Survey Research Methodology in Management Information Systems: An Assessment. Journal of Management Information Systems, Vol. 10, No. 2, pp. 75-105. Fall 1993. 11. Rosenhead, Johnathan and John Mingers, Editors. Rational Analysis for a Problematic World Revisited: Problem Structuring Methods for Complexity, Uncertainty and Conflict. Second Edition. Wiley 2001. 12. Shank, Gary D. Qualitative Research: A Personal Skills Approach. Merrill Prentice Hall. 2002. 13. Taylor, James R. and Elizabeth J. Van Every. The Emergent Organization: Communication as Its Site and Surface. Lawrence Erlbaum Associates. 2000. 14. Trask, R.L. and Bill Mayblin. Introducing Linguistics. Icon Books Ltd. 2000. 15. Winograd, Terry and Fernando Flores. Understanding Computers and Cognition. A new foundation for design. Addison-Wesley. 1987. 16. Weick, Karl E. The Social Psychology of Organizing. Second Edition. Random House. 1979. 8. CONTENT OUTLINE AND SCHEDULE The course is divided into three complementary sections: Weeks 1-4: Interpretive Research: Exploring Concept Engineering Weeks 5-8: Positivist Research: Describing Information Engineering Weeks 9-12: Critical Research: Evaluating Knowledge Engineering See the initial pages of each section of the course book for assessment and schedule details.

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________

AN EXAMPLE OF A RECENT FINAL EXAMINATION

Note That The Course Has Changed By 15%-25% In The Interim . There Is Now Somewhat More Emphasis On Qualitative Research (The first module is now entitled Interpretive Research: Exploring Concept Engineering) About The Same Emphasis On Software Engineering (The second module is now entitled Positivist Research: Describing Info. Engineering) And Somewhat Less Emphasis On The Cavana, Delahaye and Sekaran Text (The third module is now entitled Critical Research: Evaluating Knowledge Engineering as it focuses on Conflict, Values and the Imperative for Change) ___________________________________________________________________________

THE UNIVERSITY OF AUCKLAND

EXAMINATION FOR MCom BCom(Hons) ETC 2000

MANAGEMENT SCIENCE AND INFORMATION SYSTEMS Research Methods II (Time allowed: THREE hours) NOTE: 1. This examination counts for 50% of your final assessment and is marked out of 100. 2. You must answer ALL QUESTIONS. The points associated with each question range from 6% to 30%. 3. All materials supplied in the course plus all materials that you prepared during the course may be consulted.

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ -21. Case Study: Problems at Roadway Hospitality John Campbell, the CEO of Roadway Hospitality was wondering how to differentiate among the three different types of facilities offered under the Roadway Hospitality flagship so as to attract the right type of clients to each of the facilities. The Roadway Deluxe was meant for business travelers, the Roadway Express was meant for those looking for the least expensive accommodation, and the Roadway Royal was meant to provide high quality services for big spenders. Campbell felt that revenues could be quadrupled if only clients understood the distinction among the three types of facilities offered. Keen on developing a viable strategy to eliminate the brand confusion and make clear the distinction, John Campbell conducted a customer survey of those who had used each type of facility and found the following. The customers were blissfully unaware of the differences among the three types of facilities. Many complained about how old the buildings were and how poorly the facilities were maintained. The quality of services was also rated as poor. Furthermore, when rumour seems to have spread that one of the ideas of Campbell was a name change to differentiate the facilities, franchise owners became angry and the mixed messages they gave to the customers had not helped clients to understand the differences. Campbell thought that he first needed to understand how the different classifications would be important to the several classes of clients, and then he could develop a marketing strategy that would enhance revenues. Simultaneously, he recognized that unless the franchise owners fully cooperated with him in all his plans, mere face lifting and improvement of customer service would not bring in the added revenues he hoped for. (a) (b) (c) Identify the main problem Develop a theoretical framework Develop at least four hypotheses (4 points) (4 points) (4 points)

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ -32. Why is the unit of analysis an integral part of the research design? points) 3. (a) What three conditions should be met to establish that variable X causes variable Y? (6

(5 (3

points) (b) Define external validity and internal validity points) (c) What research method or methods would you choose: (i) To achieve high external validity? (ii) To achieve high internal validity? (iii) To achieve high external validity and high internal validity? 4. (a) (2 points) (2 points) (2 points)

The following variables were observed at the Ellerslie Race Course. Identify the type of scale that can be used to measure each of these variables: (i) The number on the back of each jockey (1.5 points) (ii) The placing (e.g., first, second, third, etc) of each horse in the race (1.5 points) (iii) How far each horse was off the pace (i.e., how many minutes and seconds it finished behind the winner) (1.5 points) (iv) How long each horse took to complete the race (1.5 points) (4 points)

(b) 5.

28 C is twice as hot as 14 C. Discuss.

A valid instrument is always reliable, but a reliable instrument may not always be valid. Discuss. (8 points)

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ -46. This question is based on Mclachlin, Ron. Management Initiatives And Just-In-Time Manufacturing. Journal Of Operations Management, Vol. 15, Pages 271-292. 1997. The McLachlin paper describes Yins pattern matching technique and uses it to logically test (rather than statistically test) twenty propositions. Briefly outline the logic that lies at the heart of this test with the aid of one or more worked examples. (10 points) 7. Discuss the behaviour that accompanied various experiential exercises (such as Global Warming, String, Bookworm, Dictionary, and Population ) in the light of the guidelines and explanations in Winograd and Flores. (10 points) Provide a detailed and critical assessment of the research reported in papers (a) and (b) below. For the purpose of this assessment these two papers should be treated as a single piece of research. The assessment should be based on the principles enunciated in paper (c). a) Taurima, Wayne, and Michael Cash. Researching by Networking: Developing a Draft Protocol on Researching Maori Business Enterprises . Proceedings of the NZSMS Educators Conference, pages 111-118. 1998. b) Taurima, Wayne, and Michael Cash. Aroha as Strategy?: Suggestions towards a post-initial theory of Maori business development. PECNZSM, Vol 1, Number 1, 167-173. 1999. c) Klein, Heinz K., and Michael D Myers. A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems. MIS Quarterly, Vol. 23, No 1, pages 67-94. March 1999. (30 points)

8.

______________________________________________________________

10

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________

THE UNIVERSITY OF AUCKLAND

___________________________________
SECOND SEMESTER, 2001

Campus: City ___________________________________

INFORMATION SYSTEMS Research Methods II (Time allowed: THREE hours)

NOTE: 1. This examination will be evaluated in accordance with the Guidelines for Evaluating all Assignments (see Course book 1, Item #9) 2. All materials supplied in the course plus all materials that you prepared during the course may be consulted. 3. This examination counts for 50% of your final assessment and is marked out of 100 points. 4. Each question is worth 25 points. 5. YOU MUST ANSWER ANY FOUR OF THE SIX QUESTIONS.

11

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ 2. 1. This question requires you to critique certain aspects of Theoretical Bases for Action Research through the theoretical lens provided by Cohen and Manion; Cavana, Delahaye and Sekaran pages 8-11; Winograd and Flores. In the Introduction, and in the answer to Research Question 1, Theoretical Bases for Action Research suggests that the objectives of a workshop process for the strategic evaluation of a comprehensive urban plan should match the Habermas criteria for Rational Communication. (a) Explain why you agree or disagree with this suggestion. (b) Explain why this is an important issue. 2. (12.5 points) (12.5 points)

This question requires you to critique certain aspects of Theoretical Bases for Action Research through the theoretical lens provided by Cohen and Manion; Cavana, Delahaye and Sekaran pages 8-11; Winograd and Flores. In the answer to Research Question 2, Theoretical Bases for Action Research suggests that the critical, interpretive and positivist modes of enquiry are required to evaluate a comprehensive urban plan. (a) What is the value of the insights provided by each mode? (12.5 points)

(b) What is the validity of the evaluation outcome produced by each mode of enquiry? (12.5 points) 3. This question requires you to critique certain aspects of Theoretical Bases for Action Research through the theoretical lens provided by Checkland and Holwell, and the Pop exercise. In answering Research Question 3, Theoretical Bases for Action Research suggests that a workshop process for the strategic evaluation of a comprehensive urban plan may be supported by facilitated, electronically-supported dialogue. (a) Explain how facilitated, electronically-supported dialogue impacted workshop processes. (12.5 points) (b) Explain how facilitated, electronically-supported dialogue impacted workshop outcomes. (12.5 points)

12

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ 3. 4. Examine the behaviour associated with the experiential exercises Global Warming, Pop, String and Bookworm through the theoretical lens provided by: (a) An Introduction to Cognitive Engineering (b) Dawson 5. (12.5 points) (12.5 points)

This question is composed of three parts each of which contains sub-parts. Your answers may be very brief. No explanation is required. (a) Name the research strategy or strategies McGrath associates with the following. (i) A study in a setting that is existentially real for the participants (1 point)

(ii) A study emphasizing precision of measurement of behaviour in a generic setting (1 point) (iii) A study where universal human behaviour systems are studied in an unobtrusive manner (1 point) (iv) Studies that do not involve human behaviour (2 points)

(b) Case Study 1: The Packing Department A visitor to a factory observes that the workers in the packing department have to interact with one another to get their jobs done. The more they interact, the more they seem to tend to stay after hours and go to the local pub together for a drink. However, the women packers, even though they interact with the others as much as the men, do not stay late and visit the pub after work hours. Produce a diagram that: (i) Names the variables (ii) Identifies the type of each variable (iii) Illustrates the relationships among the variables point) (3 points) (3 points) (1

(c) Case Study 2: The Machining Department A foreman believes that the low efficiency of the machine tool operators is directly linked to the high level of fumes emitted in the workshop. He would 13

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________ 4. like to demonstrate this to his supervisor through a research study so that action may be taken to fix the problem. (i) (ii) Would this be a causal or a correlational study? point) Name the variables in the study. (1 (2 points)

(iii) There are three flows of causality that are consistent with the above statement of the foremans belief. Draw a simple diagram to illustrate each. (6 points) (iv) Is this an exploratory, descriptive or evaluative (explanatory or hypothesis-testing) study? (1 point) (v) Is this a field study, lab experiment or field experiment? point) (1 (2 points)

(vi) What is the unit of analysis? 6.

This question is composed of three parts some of which contains sub-parts. For all but the first part and sub-part, brief explanations are required. (a) Operational definitions (i) What is the C, D and E of an operational definition? (ii) What is the purpose of an operational definition? (iii) What should not be included in an operational definition? (b) Scales (i) 28 C is twice as hot as 14 C. Discuss. (3 points) (1.5 points) (2 points) (1.5 points)

(ii) On page 16 of Theoretical Bases for Action Research the performance of two options against five criteria is measured on five-point scales. Options and criteria are complex qualitative concepts that lack well-defined dimensions. The scale markers are ++, +, 0 (i.e., zero), -, and --. Does the presence of the zero mean that this is a ratio scale? If not, what kind of scale should it be? Why? (9 points) (c) Goodness of measures. A valid instrument is always reliable, but a reliable instrument may not always be valid. Discuss. (8 points)
_______________________________

14

INFOSYS 751 SC 2002 DESCRIPTION

RESEARCH METHODS II

COURSE

___________________________________________________________________________

15