COMP 4010 Lecture7 3D User Interfaces for Virtual Reality
13 likes4,187 views
Lecture 7 of the COMP 4010 course in Virtural Reality. This lecture was about 3D User Interfaces for Virtual Reality. The lecture was taught by Mark Billinghurst on September 13th 2016 at the University of South Australia.
More Related Content
What's hot (20)
Viewers also liked (7)
Similar to COMP 4010 Lecture7 3D User Interfaces for Virtual Reality (20)
More from Mark Billinghurst (19)
Recently uploaded (20)
COMP 4010 Lecture7 3D User Interfaces for Virtual Reality
- 1. LECTURE 7: 3D USER INTERFACES FOR VIRTUAL REALITY COMP 4026 – Advanced HCI Semester 5 – 2016 Bruce Thomas, Mark Billinghurst University of South Australia September 13th 2016
- 3. Background: Human-computer interaction • HCI studies communication • Users and computers communicate via the interface • Traditional UI design issues: • Input device • Interaction style • Feedback to the user • Gulf of execution / gulf of evaluation • All these are equally relevant for 3D UIs
- 4. 4 Goals of Interface Design • Performance • efficiency, accuracy, productivity • Usability • ease of use, ease of learning, user comfort • Usefulness • users focus on tasks, meet user goals
- 5. Some Definitions • 3D Interaction: • Human-computer interaction in which the user’s tasks are carried out in a 3D spatial context • 3D input devices, 2D input devices mapping into 3D • 3D user interface (3D UI): • A UI that involves 3D interaction • 3D interaction technique: • A method (hardware and software) allowing a user to accomplish a task in a 3D UI 5
- 6. What makes 3D interaction difficult? • Spatial input • Lack of constraints • Lack of standards • Lack of tools • Lack of precision • Fatigue • Layout more complex • Perception
- 7. Interface Design Path 1/ Prototype Demonstration 2/ Adoption of Interaction Techniques from other interface metaphors 3/ Development of new interface metaphors appropriate to the medium 4/ Development of formal theoretical models for predicting and modeling user actions Desktop WIMP Virtual Reality
- 8. From Menus to Natural Interaction 2D UI in 3D Natural 3D Interaction
- 10. Universal 3D Interaction Tasks • Object Interaction • Selection: Picking object(s) from a set • Manipulation: Modifying object properties • Navigation • Travel: motor component of viewpoint motion • Wayfinding: cognitive component; decision-making • System control • Issuing a command to change system state or mode
- 11. Selection and Manipulation • Selection: • specifying one or more objects from a set • Manipulation: • modifying object properties • position, orientation, scale, shape, color, texture, behavior, etc.
- 12. Goals of selection • Indicate action on object • Query object • Make object active • Travel to object location • Set up manipulation
- 13. Selection performance • Variables affecting user performance • Object distance from user • Object (visual) size • Density of objects in area • Occluders
- 14. Classification of Selection Techniques • asdf
- 15. Common Selection Techniques • Simple virtual hand • Ray-casting • Occlusion • Go-go (arm-extension)
- 16. Simple virtual hand technique • Process • One-to-one mapping between physical and virtual hands • Object can be selected by “touching” with virtual hand • “Natural” mapping • Limitation: • Only select objects in hand reach
- 17. Ray-casting technique • “Laser pointer” attached to virtual hand • First object intersected by ray may be selected • User only needs to control 2 DOFs • Proven to perform well for remote selection • Variants: • Cone casting • Snap-to-object rays
- 19. Occlusion technique • Image-plane technique - truly 2D • Occlude/cover desired object with selector object (e.g. finger) • Nearest object along ray from eye through finger may be selected
- 20. Image Plane Interaction • Pierce, J., Forsberg, A., Conway, M., Hong, S., Zeleznik, R., & Mine, M. (1997). Image Plane Interaction Techniques in 3D Immersive Environments. Proceedings of the ACM Symposium on Interactive 3D Graphics, 39-44.
- 22. Go-Go Technique • Arm-extension technique • Non-linear mapping between physical and virtual hand position • Local and distant regions (linear < D, non-linear > D) Poupyrev, I., Billinghurst, M., Weghorst, S., & Ichikawa, T. (1996). The Go-Go Interaction Technique: Non-linear Mapping for Direct Manipulation in VR. Proceedings of the ACM Symposium on User Interface Software and Technology, 79-80.
- 23. Precise 3D selection techniques • Increase selection area • Cone-casting (Liang, 1993) • Snapping (de Haan, 2005) • 3D Bubble Cursor (Vanacken, 2007) • Increase control/display ratio • PRISM (Frees, 2007) • ARM (Kopper, 2010) Not ideal for cluttered environments (high density, occlusion) May require careful interaction
- 25. Goals of manipulation • Object placement • Design • Layout • Grouping • Tool usage • Travel
- 26. Classification of Manipulation Techniques • asdfa
- 27. Technique Classification by Components Manipulation Object Attachment Object Position Object Orientation Feedback attach to hand attach to gaze hand moves to object object moves to hand user/object scaling no control 1-to-N hand to object motion maintain body-hand relation other hand mappings indirect control no control 1-to-N hand to object rotation other hand mappings indirect control graphical force/tactile audio
- 28. Common Manipulation Techniques • Simple virtual hand • HOMER • Scaled-world grab • World-in-miniature
- 29. HOMER technique Hand-Centered Object Manipulation Extending Ray-Casting • Selection: ray-casting • Manipulate: directly with virtual hand • Include linear mapping to allow wider range of placement in depth Time Bowman, D., & Hodges, L. (1997). An Evaluation of Techniques for Grabbing and Manipulating Remote Objects in Immersive Virtual Environments. Proceedings of the ACM Symposium on Interactive 3D Graphics, 35-38.
- 31. Scaled-world Grab Technique • Often used w/ occlusion • At selection, scale user up (or world down) so that virtual hand is actually touching selected object • User doesn‘t notice a change in the image until he moves Mine, M., Brooks, F., & Sequin, C. (1997). Moving Objects in Space: Exploiting Proprioception in Virtual Environment Interaction. Proceedings of ACM SIGGRAPH, 19-26
- 32. World-in-miniature (WIM) technique • “Dollhouse” world held in user’s hand • Miniature objects can be manipulated directly • Moving miniature objects affects full-scale objects • Can also be used for navigation Stoakley, R., Conway, M., & Pausch, R. (1995). Virtual Reality on a WIM: Interactive Worlds in Miniature. Proceedings of CHI: Human Factors in Computing Systems, 265-272, and Pausch, R., Burnette, T., Brockway, D., & Weiblen, M. (1995). Navigation and Locomotion in Virtual Worlds via Flight into Hand-Held Miniatures. Proceedings of ACM SIGGRAPH, 399-400.
- 33. Voodoo Doll Interaction • Manipulate miniature objects • Two handed technique • One hand sets stationary frame of reference • Second hand manipulates object Pierce, J. S., Stearns, B. C., & Pausch, R. (1999). Voodoo dolls: seamless interaction at multiple scales in virtual environments. In Proceedings of the 1999 symposium on Interactive 3D graphics (pp. 141-145). ACM.
- 34. Navigation • How we move from place to place within an environment • The combination of travel with wayfinding • Wayfinding: cognitive component of navigation • Travel: motor component of navigation • Travel without wayfinding: "exploring", "wandering”
- 35. Travel • The motor component of navigation • Movement between 2 locations, setting the position (and orientation) of the user’s viewpoint • The most basic and common VE interaction technique, used in almost any large-scale VE
- 36. Types of Travel • Exploration • No explicit goal for the movement • Search • Moving to specific target location • Naïve – target position not known • Primed – position of target known • Maneuvering • Short, precise movements changing viewpoint
- 37. Movement Process • Focusing on user control
- 38. Technique classification • Physical locomotion metaphors (treadmills, cycles, …) • Steering metaphor • Route planning metaphor • Target specification metaphor • Manual manipulation metaphor • Scaling metaphor
- 40. Taxonomy of Travel Techniques • Focusing on sub-task of travel
- 41. Gaze Directed Steering • Move in direction that you are looking • Very intuitive, natural navigation • Can be used on simple HMDs (Google Cardboard • But: Can’t look in different direction while moving
- 43. Pointing Technique • A “steering” technique • Use hand tracker instead of head tracker • Slightly more complex, cognitively, than gaze-directed steering • Allows travel and gaze in different directions – good for relative motion
- 45. Grabbing the Air Technique • Use hand gestures to move yourself through the world • Metaphor of pulling a rope • Often a two-handed technique • May be implemented using Pinch Gloves Mapes, D., & Moshell, J. (1995). A Two-Handed Interface for Object Manipulation in Virtual Environments. Presence: Teleoperators and Virtual Environments, 4(4), 403-416.
- 48. Redirected Walking • Address problem of limited walking space • Warp VR graphics view of space • Create illusion of walking straight, while walking in circles Razzaque, S., Kohn, Z., & Whitton, M. C. (2001, September). Redirected walking. In Proceedings of EUROGRAPHICS (Vol. 9, pp. 105-106).
- 50. Redirected Walking with Walls https://www.youtube.com/watch?v=u8pw81VbMUU
- 51. Guided Navigation Technique • Water skiing metaphor for VR movement
- 52. Example
- 53. Wayfinding • The means of • determining (and maintaining) awareness of where one is located (in space and time), • and ascertaining a path through the environment to the desired destination • Problem: 6DOF makes wayfinding hard • human beings have different abilities to orient themselves in an environment, extra freedom can disorient people easily • Purposes of wayfinding tasks in virtual environments • Transferring spatial knowledge to the real world • Navigation through complex environments in support of other tasks
- 54. Wayfinding – Making Cognitive Maps • Goal of Wayfinding is to build Mental Model (Cognitive Map) • Types of spatial knowledge in a mental model • landmark knowledge • procedural knowledge (sequence of actions required to follow a path) • map-like (topological) knowledge • Creating a mental model • systematic study of a map • exploration of the real space • exploration of a copy of the real space • Problem: Sometimes perceptual judgments are incorrect within a virtual environment • e.g. users wearing a HMD often underestimate dimensions of space, possibly caused by limited field of view
- 55. Wayfinding as a Decision Making Process
- 56. Designing VE to Support Wayfinding • Provide Landmarks • Any obvious, distinct and non-mobile object can serve as a landmark • A good landmark can be seen from several locations (e.g. tall) • Audio beacons can also serve as landmarks • Use Maps • Copy real world maps • Ego-centric vs. Exocentric map cues • World in Miniature • Map based navigation
- 57. Wayfinding Aids • Path following • Easy method of wayfinding • Multiple paths through a single space may be denoted by colors • For example, hospitals that use colored lines to indicate how to get to certain locations. • Bread crumbs (leaving a trail) • leaving a trail of markers - like Hänsel and Gretel • allows participant to know when they've been somewhere before • having too many markers can make the space be overly cluttered • Compass • may also be other form of direction indicator (e.g. artificial horizon) • may specify directions in 2D space or 3D space
- 58. Examples
- 59. System Control • Issuing a command to change system state or mode • Examples • Launching application • Changing system settings • Opening a file • Etc. • Key points • Make commands visible to user • Support easy selection
- 60. System Control Menu Types • Variety of menu types can be used
- 61. Example: GearVR Interface • 2D Interface in 3D Environment • Head pointing and click to select
- 62. TULIP Menu • Menu items attached to virtual finger tips • Ideal for pinch glove interaction • Use one finger to select menu option from another Bowman, D. A., & Wingrave, C. A. (2001, March). Design and evaluation of menu systems for immersive virtual environments. In Virtual Reality, 2001. Proceedings. IEEE (pp. 149-156). IEEE.
- 63. Resources • Excellent book • 3D User Interfaces: Theory and Practice • Doug Bowman, Ernst Kruijff, Joseph, LaViola, Ivan Poupyrev • Great Website • http://www.uxofvr.com/ • International 3DUI group • Mailing list, annotated bibliography • www.3dui.org • 3DI research at Virginia Tech. • research.cs.vt.edu/3di/
- 64. Acknowledgments • Doug Bowman, Virginia Tech • Joe LaViola, University of Central Florida • Ernst Kruijff, Graz Univ. of Technology • Ivan Poupyrev, Google