SAE AR/VR - The challenges of creating a VR application with Unity
4 likes2,331 views
This document discusses the challenges of creating VR applications in Unity. It outlines some of the key challenges including maintaining presence, managing devices and displays, interactions, deployment to different hardware, and rendering speed. It then introduces MiddleVR, a plugin for Unity that aims to simplify VR development by providing abstractions and functionality for many of these challenging areas such as device management, display management across multiple screens, and cluster rendering. MiddleVR supports a variety of VR hardware and aims to improve the development process for VR applications in Unity.
![[1C7] Developing with Oculus](https://cdn.slidesharecdn.com/ss_thumbnails/1c7developingwithoculus-140928204557-phpapp01-thumbnail.jpg?width=560&fit=bounds)
Ad
More Related Content
What's hot (20)
Similar to SAE AR/VR - The challenges of creating a VR application with Unity (20)
Ad
Recently uploaded (20)
Ad
SAE AR/VR - The challenges of creating a VR application with Unity
- 1. The Challenges of! Creating VR Applications! in Unity Sébastien Kuntz! [email protected]! CEO
- 2. Sébastien Kuntz ! • Founder & President! • 12 years in VR! ! • - French railways! ! •
- 3. • Simplify VR! • Creation! • Deployment! • So YOU can spend more time creating applications
- 4. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Rendering speed! • MiddleVR for Unity! • Conclusion
- 5. Plan Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
- 6. • Platform for creating interactive 3d applications! • Widely used! • 2 million registered developers! • Certified by the US Army and Air Force! • Clients : Bigpoint, Cartoon Network, CocaCola, Disney, Electronic Arts, LEGO, Microsoft, NASA, Nickelodeon, Ubisoft, US Army, Warner Bros.
- 7. • High-quality rendering! • Deferred renderer! • Occlusion culling! • Realtime shadows! • Ambient occlusion! • Light mapping
- 9. Unity 3D Features • Programming! • C#, Javascript! 3D formats ! • Profiler! • Physics! • NVidia PhysX! • Cloth! • Soft/rigid bodies! • Ragdolls! • Cars FBX, Collada, 3DS, OBJ…! Audio engine! Networking! Terrain
- 10. Editor • Simple to use! • Create your own tools inside Unity
- 11. Asset store • Buy / sell assets! • 3D models! • Textures! • Shaders! • Scripts! • Editor plugins
- 12. Desktop
- 13. Plan • Unity! Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
- 14. Plan • Unity! • Challenges! Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
- 15. Virtual reality Presence in a virtual world (c) eMagin (c) Barco
- 16. Experience of presence is in your brain !
- 18. To create and maintain presence, you need to ... • Build hardware • Build software • Build user experience
- 19. Plan • Unity! • Challenges! • Presence! Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
- 21. Device management Tracker 1! Driver Tracker 1 Application Tracker 2! ! Driver Tracker 2
- 23. Plan • Unity! • Challenges! • Presence! • Device management! Display management! • Cluster management! • Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
- 24. Correct perspective (c) Johnny Lee
- 26. Stereoscopy
- 28. Graphics cards limitations • Number of outputs! • Processing power
- 29. multiple graphics cards • Pro! • More outputs! • More processing power! • Con! • 3D engines are not optimized for multi-pipe
- 30. Cluster • Pro! • More outputs! • More processing power! • Con! • 3D engines don’t support clustering
- 31. Cluster • No seams across displays require 3 layers of sync! • 3D objects synchro (Scenelock)! • New images display (Swaplock)! • Left/Right eyes (Genlock)
- 32. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! Interactions! • Deployment! • Speed! • MiddleVR for Unity! • Conclusion
- 33. Interactions • Many different 3D user interactions! • Navigation! • Selection! • Manipulation! • Draw! • System control! • etc.
- 34. Interactions are dependent on hardware
- 35. Menus
- 37. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! Deployment! • Speed! • MiddleVR for Unity! • Conclusion
- 38. Deployment • Modify application to! • Change drivers! • Change displays! • Change interactions
- 39. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! Rendering speed! • MiddleVR for Unity! • Conclusion
- 40. Rendering speed
- 41. Plan • Unity! Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Rendering Speed! • MiddleVR for Unity! • Conclusion
- 42. Application
- 43. Plan • Unity! • Challenges! • Presence! • Device management! • Display management! • Cluster management! • Interactions! • Deployment! • Speed! MiddleVR for Unity! • Conclusion
- 44. To create and maintain presence, you, the author, need to ... • Build hardware • Build software • Build user experience
- 45. “VR is mainly a software problem”! Jaron Lanier
- 46. What is MiddleVR ? • VR plugin ! • Functionalities ○Manage input devices •3D trackers, keyboard, mouse, joystick ! ○Manage display •Viewports, cameras, stereoscopy, cluster
- 47. Desktop
- 53. Supported Devices • Via VRPN : • A.R.T • Vicon • Optitrack • Wiimote ! • Polhemus Fastrak Tracker, 3Space Tracker, Patriot Trackers, • • • • • • • • Ascension Flock of Bird Intersense IS600, IS900 3dTech HiBall-3000 Wide Area Tracker, Advanced Realtime Tracking Gmbh DTrack Client, World Viz Precision Position Tracker PPT 1.2, Natural Point Optitrack Rigid Body Toolkit, Phasespace optical tracking OWL, ... • Native drivers • Oculus Rift • Leap Motion • Kinect (Microsoft SDK) • PNI SpacePoint Fusion • GameTrak • Razer Hydra • Motion Analysis (beta)
- 54. First integration of Haption’s haptic devices http://www.youtube.com/watch?v=lnoE_VclnOI
- 55. Stereoscopy • Active stereoscopy (OpenGL Quad-Buffer)! • Which Unity can’t natively handle! • Passive stereoscopy
- 56. Cluster & Multi-pipe • Scenelock! • Swaplock! • Genlock! • Multi-pipe (soon)
- 57. Supported displays • • • • • • • • • Virtually any HMD VR-Wall, Powerwall Workbench, Holobench HoloStage CAVEs 3D TVs zSpace Igloo Vision …
- 58. Deployment • Abstractions! • Devices! • 3D Nodes (User)! • Interactions (soon)
- 59. Device management Driver Tracker 1 Application Driver Tracker 2
- 60. Device abstraction Driver Tracker 1 Virtual Device Driver Tracker 2 Application
- 62. User abstraction Virtual Device 1 Virtual Device 2 Inverse Kinematics Hand (3D Node) Application
- 63. Interactions • MiddleVR includes high-level interactions! • Navigation! • Selection! • Manipulation! • … More to come! • Future: Interactions abstractions! • Dynamically change interactions based on hardware
- 65. Menus • Ability to create menus and GUIs in HTML5 (soon)
- 66. • Unity! Speed • Many options for optimizations! • Occlusion culling! • Static batching! • Lightmap baking! • etc.! • 32-bit editor only! • But player is 32-bit & 64-bit !! • Subdivide parts in editor! • Compositing! • Cloud
- 69. VR in the Cloud • Temporary increase in power! • Combine with compositing
- 70. Perceptive illusions • Playing with limitations of human perception! • Sensitivity of senses! • Selectivity of attention! • Useful to lower cost/complexity of VR systems
- 71. Perceptive illusions Redirected walking (c) Franck Steinicke - Wurzburg University
- 72. Perceptive illusions Change blindness Change blindness : (c) Evan Suma - USC/ICT