Volume 4, Issue 3, March – 2019 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

User Interface and User Experience for Virtual Reality

Using Biopotential: A Virtual Reality System with
Cross Platform and Plug-and-Play Capability
Pavithren. T, Praveenkumar. M, Ramesh B, Dr. Sujitha. J AP(Sl.G)
Department of Electronics and Communication Engineering
Sri Ramakrishna Engineering College
Coimbatore, India

Abstract:- The developing virtual reality and augmented sent back to the device. The haptic feedback usually being
reality platforms broaden the need for a more intuitive vibration motors and actuators, can give a sense of physical
form of Human-Computer Interaction. Although many parameter to the virtual environment for the user. Although
devices implement a virtual reality into their design, it the feedback can be accurate and precise, it becomes
focuses mostly on visuals and audible environment, the dependent on input electrical power as the feedback intensity
touch sensory haptic is a very significant factor that is raised, also adding to the mechanical system that can be
determines the physical interaction between the virtual bulky. There is very much a need for mechanical hardware so
environment and the user which for the most part is as to translate the virtual parameters as a force and touch
overlooked or very crudely implemented. Most of the feedback. The limitations of a haptic glove will be its
haptic devices currently available are made for working physical structure and the energy required to run such
with specific programs or require an adaptation to devices. To maximize efficiency with ease of use, IMU
perform with various tools and applications. This limits sensors are used most commonly used to track motions for
the multiplatform usage and also hinders plug-and-play more accurate implementation of hand motion tracking and
functionality. By simplifying the data stream from such capture. Unfortunately, IMU sensors are much expensive
virtual devices, this haptic arm focuses on an interface for than passive sensors and are very sensitive to even slight
virtual reality devices that can be applied as a Human motions. A filter algorithm must be used to reduce noise and
Interface Device for pluggable-USB communications, provide a one to one motion scaling (displacement in the
making way for a much cheaper and multiplatform virtual environment must match displacement in reality). All
capable virtual reality haptic devices (i.e. VR gloves, in all, making the computation of such devices to be defined
Haptic gloves, Motion capture suits etc.) and one-to-one by the manufacturer and calibrated to match the required
animatronic (i.e. Exoskeleton, robotics, control panel etc.) functionality, thus the use of ad-hoc programs are needed to
without use of expensive digital sensors or graphics fully handle the device, resulting in limitations such as cross-
intensive platforms. The advent of virtual reality and 3D platform usage and preferable specific system requirements
graphics simulation paved the way for a more intuitive to handle the data stream and process that data. A device of
Human-Computer Interface that bridges the gap between such complexity can make it hard to calibrate and actuate for
humans and machines. Furthermore providing a realistic various applications and usage. In this design, a basic
experience of human interaction in the form of vision, protocol for a common parameter of sensory data and haptic
sound and touch. feedback control is built in such a way to imitate HID
interfaces used to pair peripherals such as a mouse, keyboard
Keywords:- Exoskeleton-Arm; Haptic-Arm; Virual Reality- and joystick. With a simple user interface and cross-platform
Arm functionality of HID based interaction, virtual reality devices
can be easily manipulated and used as a pluggable USB
I. INTRODUCTION device like a mouse, keyboard etc. To make computations
even easier and intuitive, the hand motion must be tracked
The currently used haptic glove or data glove in virtual into individual axis for the specific joints in the arm. Instead
reality applications have a specific set of data recorded or of using IMU or flex sensors, the Exoskeleton-arm uses daisy
sensed from the human hand to be processed by the ad-hoc chain links to calculate the position of individual joints of the
program designed to analyze and track the hand motion in human arm and feedback parameters, allowing ease of post-
space. Using this information, the program can then decide processing and manipulation later in the computer GUI.
the graphics processing and also the necessary feedback to be

IJISRT19MA719 www.ijisrt.com 404

Volume 4, Issue 3, March – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
II. HUMAN INTERACTIONS IV. SENSORS

A. Purpose There are many ways to track motion, in which case the
The interaction of a human with any computer or generally used IMU sensor is not effective and cost efficient
technology is made more productive and less abstract when for multiple point tracking. Thus in this case with the aid of
the full sensory of vision, audio and touch are felt. This can an Exoskeleton-arm, we use the following components for
be observed in the use of touch screen compared to the use of tracking motion.
a mouse or keyboard. The visual aesthetic of touching an
element on the screen is much more productive and effective A. Thin-Film Variable Potentiometer
in usage, rather than to use keyboard strokes or mouse The sensing element of each joint is made as a plastic
movement. By combining all the three essential elements of membrane that works much like a slider potentiometer but
interaction, one can experience transfer the experience of the with only two thin sheets of the carbon layer and an
real world physics into a virtual environment that is entirely electrically conductive sheet sliding between the two layers.
abstract in a computer but made physical using this The setup is attached over the joints such that bending of the
technology. fingers or arm causes the layers to displace within. This
enables us to register the movement of a joint as a change in
B. Scope value ohm's across the thin-film potentiometer. This reduces
The Human-Computer Interface is a much necessary the need for any mechanical aspect for the sensor and can be
feature for any technology or device that actively needs the layered with the structure of the exoskeleton-arm.
human to interact within a multidimensional field. This
haptic arm intends to make the wearable virtual reality suit to B. Rotary Encoders
be easily adaptable to any sort of environment by providing a Another useful sensing method is by using rotary
much flexible user interface with data from each joint in the encoders or absolute encoders based on the nature of the joint
human arm and also a controllable feedback system to movement. For joint movements that cannot be sensed using
provide a parameter from the environment to actuate physical thin-film potentiometer, a rotary encoder enables the use of
forces. The scope of such a device gives a new dimension for gears and complete cycles of rotations, which can be useful
the use in virtual reality, augmented reality, graphics in terms of magnifying a joint motion by use of gears and or
simulation, gaming, user interfacing, 3D modeling, medical pulleys.
instruments, rehabilitation exoskeleton etc.
C. Force Sensors
III. SENSING MOTION The force sensor is a parameter control element that
determines the range of exerted force on the human arm and
Sensing motion is the most important factor that acts as a feedback control loop for the microcontroller to
determines the number of sensors and how that data is know if the desired force is reached or not.
calculated. By implementing passive sensors, the cost of the
whole device can be greatly reduced. D. IMU Sensors
In the case where the position of the user is needed, a
The various sensors that can be used for motion single or few IMU sensors can be used as a point of reference
tracking feature a multi-axis such as x, y, z-axis degrees of for the rest of the sensors on the exoskeleton-arm. The IMU
freedom. These sensors are digital and need to be processed sensor can be used for positioning the user in relation to the
by a microcontroller to retrieve the data. Using multiple rest of the sensor data that are linked in a daisy chain
digital sensors is expensive and also needs a large data network.
transmission rate between the device and the processor. This
can also lose energy efficiency for a battery-powered device. E. Electromyography Sensor
Thus a digital sensor is not an option for this haptic arm, In purpose for analyzing amputated limbs, the EMG
instead, passive sensors are used to make the device collect sensors are implemented to sense the electrical signals sent to
more data and be energy efficient and also be easy to service the muscles from the brain. This helps in rehabilitation and
as modules. To make the exoskeleton-arm more robust and analyzing how the brain interacts with the amputated limb.
sleek without much mechanical bulk, the sensors are
manufactured as thin film membranes. V. SENSING METHODS

The virtual reality implies the interaction of the human

body with the virtual world; this mainly focuses on the hand
and how it interacts with the physical geometry of the virtual
environment as shown in the Figure 1.

IJISRT19MA719 www.ijisrt.com 405

Volume 4, Issue 3, March – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

Fig 1:- Control Flow of Exoskeleton-arm

A. Hand Motion Sensing application that's using this virtual reality protocol. With the
Hand motion tracking can be sectioned into two types, addition to the movement of joints, we also sense the force
individual finger movement and position of the whole hand. and capacitive touch of the fingertips so that the haptic
To compute the position of the hand in space, an external feedback can use this data to trigger the actuator.
imaging method, ultrasonic scanning or Internal sensors in
the device are the most commonly used types of motion VI. CONTROL SYSTEM
sensing for low-level virtual reality data gloves. Some high-
end motion tracking uses image processing or IR camera The control system is the electronic processing of the
motion tracking etc. In the above methods, most of the various inputs and outputs that are determined by the virtual
sensing methods are digital sensors, preferably for ease of environment and the parameters set by the user.
application and 3d motion sensing. Though this can seem
effective, it restricts the level motion tracking of individual A. Internal Microcontroller Unit
joints in the arm. With one sensor computing position in Although the device used in such a virtual reality
degrees of freedom of multiple axes and also presumably the platform is known to use high-end processors for data
movement of the individual fingers, it reduces the need for acquisition, this Exoskeleton-arm is designed to use any
more sensors in its place. This also greatly reduces the microcontroller that can handle multiple GPIO. It is
flexibility of individual control of the joints. Instead of recommended that the MCU has a fairly high processing
tracking the motion of the sensor in space or using image speed since the number of data lines to be processed is large
processing, we use individual passive sensors placed at every and is directly proportional to the resolution of the data
joint of the arm, giving us full control of the data from the collected from the exoskeleton-arm.
glove.
B. Multiplexor
B. Sensors Network Connection Since there are only a few limited analogue GPIO ports
The multiple sensors used in these exoskeleton-arm in any standard MCU, we use a multiplexer to give more
results in an abstract data set with no actual interrelationship control over the number of analogue input channels that are
between the sensors. By implementing a daisy chain necessary. The multiplexer is a very important part of the
algorithm to link each sensor to relate to both the previous processing because it helps simplify the multiple data stream
and next sensor, this helps us to manipulate the virtual into a single array of data set, which is easy for data
environment with relation to the individual axis from the acquisition and transmission compared to single channel data
glove. We used approximately 32 various inputs from the transmission.
glove. This is continuously sent to the HID interface for
controlling and mapping the address of the axis to the

IJISRT19MA719 www.ijisrt.com 406

Volume 4, Issue 3, March – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
C. Excitational Feedback surface that comes in immediate contact with the human arm
Related to haptic feedback, this data consists of the will be enabled with a force sensor, which is a parameter
basic trigger signal to which the haptic drivers must react. control for regulating the force acting on the body. This
This signal is an external interrupt given to the MCU from a completes the control flow loop of an exoskeleton-arm that
computer or other feedback system. This signal is designed to receives inputs from the hand movement and transmits it to
tell which output feedback driver must be activated and what the host, which returns feedback that actuates the haptic
is the parameter that has to be reached. This can be positional transducer.
feedback or force feedback, depending on the parameters set
by the computer or external feedback system such as VIII. PERFORMANCE
robotics.
The performance of the device is determined by initial
D. Data Acquisition calibration of the sensors and the host system. This alignment
From the data of the various joints in the hand, we now is crucial for a one-to-one replication of hand movement. The
have to map these data to a target host, which can be a haptic feedback is actuated completely by the host computer
computer application, simulation, game, robotics or or robotic which sets the parameter for actuation. This has to
instruments. This is fairly simple as the device is intended to be aligned to meet the desired effect of touch. The intention
be HID compatible USB, such as a mouse, keyboard or of the haptic arm is to make the device work with passive
joystick. This removes the need for a graphics processor to sensors and have no need for a graphics processing of the
work separately to process the exoskeleton-arm data. This data. This greatly reduces the processing power needed to
also means that the device is compatible with all major operate such devices.
operating system regardless of its processor specification.
The data can be used to control the actuation of external IX. DISCUSSION
devices such as motors in robots.
The choice of passive sensors is debatable to how
VII. HAPTIC SYSTEM accurate and sensitive it can be to a motion, but it is to be
kept in mind that the design intends to reduce production cost
The haptic system is the tactile feedback and force and also make it compatible with almost all-processing
feedback that enables the experience of geometry and surface system. Implementing dedicated thin-film sensors for the
contact that is made in the virtual world. It combines joints increases accuracy and sensitivity. The daisy chain
mechanical system and electrical transducers that imitate method of the sensor network is used to eliminate the need
tactile and force feedback in real life. for digital sensors; it is required for the whole arm to be
sensed in order for this to work. The haptic transducers are
A. Haptic Feedback relative to the power supplied to the drivers, more power is
Haptic feedback can be simplified into two types, translated to more powerful force feedback. The HID-
tactile sensation and forced feedback sensation. Tactile compliant connection is not necessarily a graphics device, but
sensation also includes touch sensation of texture and thermal it can provide an application with all the necessary data to
conductivity, and also the immediate restriction of movement manipulate any 3D intensive program. Though this is an
caused by physical interaction. Forced feedback is the indirect interaction, it eliminates the need for the 3-
implied force exerted by an external object such as dimensional tracking of the arm and finger. The device is
experienced in carrying the weight of an object. To intended for ease of service and repair, as the parts used are
understand these two types of haptic feedback, the tactile very cheap and readily available or easy to manufacture.
sensation is analogous to touching a textured object such as a
metal ball. Forced feedback is the force applied on a body X. CONCLUSION
such as experienced when dribbling a basketball, the force
exerted when the ball hits the hand. With these two types of The importance of a Human-Computer Interface is
haptic, the implementation of actuators to apply necessary growing more necessary than just a feature with the advent of
force on the user's hand is possible with the use of the HID virtual reality and augmented reality environment. The
interface. The mechanical aspect of the arm is designed to devices that enable hand controlled interfaces make the
work passively and triggered by solenoids. peripheral more ad-hoc to specific functions rather than a
universally adaptive device that can be used with any
B. Haptic Drivers For Tactile and Force Feedback platform of operation. This opens a new dimension into 3D
The haptic driver sends out PWM signal from the MCU interfacing that can finally enable the user to not just see and
that enables accurate control of the solenoids that are used as hear but also to feel a virtual object.
the transducer for the haptic feedback. The transducer has a
tightly wound coil that actuates an armature with opposing
magnets on each end. This enables a linear position actuation,
which is used for force feedback and touch actuation. The

IJISRT19MA719 www.ijisrt.com 407

Volume 4, Issue 3, March – 2019 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
REFERENCES Workshop on Understanding the Human Hand for
Advancing Robotic Manipulation, 2009.
[1]. Argelaguet, F., Kunert, A., Kulik, A., and Froehlich, B. [14]. Gupta, A. and O’Malley, M. K. "Design of a haptic arm
"Improving co-located collaboration with show-through exoskeleton for training and rehabilitation."
techniques". In Proceedings of the IEEE Symposium on IEEE/ASME Transactions on Mechatronics, Volume
3D User Interfaces, pp. 55–62, 2010. 11, Issue no.3, pp. 280–289, 2006.
[2]. Argelaguet, F., Sato, T., Duval, T., Kitamura, Y., and [15]. Hirota, K. and Hirose, M. "Dexterous object
Lécuyer, A. "Collaborative pseudo-haptics: Two-user manipulation based on collision response". In
stiffness discrimination based on visual feedback". In Proceedings of Virtual Reality, pp. 232, 2003.
Proceedings of EuroHaptics, pp. 49–54, 2014. [16]. Koutek, M. and Post, F. H. "Spring-based manipulation
[3]. Balakrishnan, R., Ware, C., and Smith, T. " Virtual tools for virtual environments". In Proceedings of
hand tool with force feedback". In Proceedings of Eurographics Workshop on Virtual Environments, pp.
Conference on Human Factors in Computing Systems, 61–70, 2001
pp. 83–84,1994.
[4]. Ban, Y., Kajinami, T., Narumi, T., Tanikawa, T., and
Hirose, M. "Modifying an identified curved surface
shape using pseudo-haptic effect". In Proceedings of
International Symposium on Haptic Interfaces for
Virtual Environment and Teleoperator Systems, pp.
211–216, 2012.
[5]. Barbagli, F., Devengenzo, R., and Salisbury, K.
"Enabling multi-finger, multi-hand virtualized
grasping". In Proceedings of IEEE International
Conference on Robotics and Automation, pp. 806–815,
2003.
[6]. Barbagli, F. and Jr., J. K. S. "The effect of
sensor/actuator asymmetries in haptic interfaces". In
Proceedings of International Symposium on Haptic
Interfaces for Virtual Environment and Teleoperator
Systems, pp.140, 2003.
[7]. Borst, C. W. and Indugula, A. P. "A spring model for
whole-hand virtual grasping". Presence, Volume 15,
Issue no. 1, pp. 47–61, 2006.
[8]. Bullion, C. and Gurocak, H. "Haptic glove with MR
brakes for distributed finger force feedback". Presence,
Volume 18, Issue no. 6, pp. 421–433, 2009.
[9]. Burdea, G., Gomez, D., Langrana, N., Roskos, E., and
Richard, P. "Virtual reality graphics simulation with
force feedback". International Journal in Computer
Simulation, Volume 5, Issue no. 3, pp. 287–303,1995.
[10]. Casiez, G., Plénacoste, P., Chaillou, C., and Semail, B.
"The DigiHaptic, a new three degrees of freedom multi-
finger haptic device". In Proceedings of Virtual Reality,
pp. 35–39, 2003.
[11]. Casiez, G., Roussel, N., and Vogel, D. "1€ filter: a
simple speed-based low-pass filter for noisy input in
interactive systems". In Proceedings of Conference on
Human Factors in Computing Systems, pp. 2527–2530,
2012.
[12]. Cirio, G., Marchal, M., Hillaire, S., and Lécuyer, A.
"Six-degrees-of-freedom haptic interaction with fluids".
IEEE Transactions on Visualizations and Computer
Graphics, Volume 17, Issue 11, pp.1714–1727, 2011.
[13]. Feix, T., Pawlik, R., Schmiedmayer, H., Romero, J.,
and Kragic, D. "A comprehensive grasp taxonomy". In
Proceedings of Robotics, Science and Systems –

IJISRT19MA719 www.ijisrt.com 408