IVITCMDtalk
Download as PPT, PDF1 like332 views
The document discusses using cognitive science theories and approaches to study information visualization and interaction. It proposes taking a human-centered, theoretically grounded approach to decompose tasks based on cognitive architectures and identify challenges to cognitive processing. This would involve using "toy world" studies and computer models to test systems and gain quantitative predictions that can inform interaction design. It provides examples of previous work on multiple object tracking and air traffic control research.
IVITCMDtalk
- 1. Cognitive Approaches for Studying Information Visualization and Interaction Fisher and Dill
- 2. Cognitive “Interaction Science” of Visual Analytics High need for Cogsci theory base for Visual Analytics Cogsci theories evocative for Visual Analytics Enactive Cog: Perception-Cognition-Action loops Distributed Cog: Problem-solving in joint activities Level of description is too high-- how to use? Interaction science should ground theory in use Cognitive Architecture task decomposition Toy-world studies, CogArch tests e.g.: Fishtank air traffic control & FINSTS Combine observation & computer models How to decompose multiple screen collaborative VA?
- 3. Background Molecular Biology, Medical Biophysics, Hiram/CWRU Cognitive Neuroscience, Experimental Psychology, UCSC Human-Computer Interaction Inst. for Robotics & Intelligent Systems, UWO Cognitive Science, Rutgers SFU School Interactive Arts and Technology UBC Media And Graphics Interdisciplinary Centre
- 4. Design Approach Human-centred Theoretically motivated Grounded in practice of design Emphasize design process Evolve human sciences to interaction science Framework for performance analysis Taxonomy of interaction Integrate with (software) engineering models
- 5. Pasteur’s Quadrant (Stokes) MAGIC
- 6. Research approach Decompose task based on cognitive architecture Identify interaction challenges to cognitive processing modules Devise “toy world” test of that module’s robustness to challenge Use-inspired basic research Research interaction, not the mind Model data for quantitative prediction
- 7. Air traffic control research Free Flight ATC “fishtank” projection Change camera position for better view How will global motion affect tracking?
- 8. Multiple object tracking (Pylyshyn)
- 10. Test performance at different speeds...
- 11. Fit human tracking function (Lui)
- 12. ... Then add display motion
- 13. Tracking vs object speed
- 14. Tracking vs 2 object speeds
- 15. Tracking in warped space
- 16. Tracking in warped space
- 17. Recap Pasteur: Basic research on fermentation will inform applied work on disease (and beer) We think: Basic research on tracking will give us design guidelines for fishtank ATC displays Should generalize to other systems Modeling leads to quantitative prediction and customization for “personal equations” This is not psychology
- 18. Design space explosion Buxton/Kasik
- 19. As interactive technology becomes immersive, multimodal, and ubiquitous The emphasis in interaction design shifts to facilitating perceptual cognition and embodied interaction The emphasis in interaction design shifts to facilitating perceptual cognition and embodied interaction The emphasis in interaction design shifts to facilitating perceptual cognition and embodied interaction
- 20. Interaction science & design Paul Ehrlich Knew chemistry, but not biochem Used science to reduce design space to 900 Had animal model for testing Number 606 = Salvarsan Advance science and treatment Today Medical science is distributed cognition Clinicians, physiologists biochem... Many will never be first author
- 21. Interaction design future? Design for key sensory & motor systems Assess specific aspects of interaction Walkthrough or experiment Implement prototype Interaction Science
- 22. Implementations Perception & attention in dynamic workspaces Air traffic control (HRL, NSERC), Multimodal driving interfaces (Nissan) Cognitive & collaborative aspects of tasks CZWeb/CZTalk Laser pointers, cell phones + situated displays (NTT, Panasonic) CSCL video, attention management/privacy(CANARIE, EU Kaleidoscope) Space constancy in VR Wayfinding in CAD worlds (Boeing) Display customization for perceptual abilities (GM)
- 23. Acknowledgements Profs Jim Enns (UBC) Sid Fels (UBC) Ron Rensink (UBC) Grads Colin Swindels Ritchie Argue Erin Austin Geniva Lui Alexander Stevenson • Postdocs/Staff – Jason Harrison (UBC) – Barry Po • Grants & partners – Nissan – Boeing – GM – HRL – NSERC – IRIS – MAGIC Endowment Fund
- 24. Spatial Cognition in complex environments Multiple events at different locations Multimodal events (sight, sound, touch) Action in space Tools FINSTs and indexical cognition Space Constancy Functional space constancy: Ability to interact directly Apparent space constancy: Ability to make judgments Sensory integration (FLMP, calibration by pairing)
- 25. Communicative Pragmatics & Groupware Psycholinguistic pragmatics theories and methods extended to interactive environments Support for familiar F2F metachannels Embodied communication (gesture, facial expression, body language Prosody Interaction methods to support underlying processes Acknowledgement and repair mechanisms Advancement Layering
- 26. Personalization Personal Equation: Individual differences in perceptual, indexical attentive, and cognitive processes Test users Model data Describe individual differences in model parameters Customize display for PEI Attentive systems adapt PEI Within a session: fatigue, attentiveness Between sessions: aging changes
- 28. Other work Combine qualitative and mathematical techniques Highly skilled interaction Study control and communication
- 29. Grounded theory Open coding Themes arise from the data, phenomena are observed Axial coding Relations are drawn between themes and observations in the data Triangulation Mixed methods to test theories
- 30. Add HCI aspects/analyses Replace flutes with analog synth Allows us to capture Control: Breath and fingering Communication: Gestural motion Can add HCI variables such as lag, bias, noise etc. to understand how skilled performers compensate May feedback to music controller design
- 31. Mapping to Visual analytics Build toolset: “Thick Description” + Math models Understand “gold standard” for interaction Technology to support expertise - “virtuosity” How information from multiple senses is integrated “ Tight loop” Perception/action patterns, sequences, rhythms Development of coordination between users
Editor's Notes
- #22: One casualty of the move to more complex interactive environments that “download” processing to perceptual systems may be the basic spiral model of software development. The larger design space of multimodal and immersive environments makes the design stage more difficult-- if designers themselves lack valid intuitions about the impact of their designs on users’ perceptual systems how can they design them effectively? Similarly, what constitutes a test of the perceptual interaction of a given design?
- #30: Contrast grounded theory with traditional music theory Objectivist vs constructivist Grounded theory follows a set of assumptions about the nature of knowledge. It assumes that in order for a theory to be meaningful, it must be drawn from real world data.
- #31: Low level observation based on higher level goals. What we see here is the beginnings of the kind of mature cross disciplinary collaboration that characterizes bio-medical research and is elsewhere absent.
- #32: Affording virtuosity