Scribd
Upload
32 views

Video Based 3D Human Activities Analysis: By: Jacob Randall Autumn Smith Muhammad Farooq

The document describes research on analyzing 3D human activities through video. It discusses developing algorithms to track a person's movements in video, convert that information to a 3D model, and match the model's movements to the original video. The goals are to reconstruct motions and gestures as accurately as possible using techniques like motion capture suits, feature vector extraction from skeleton images, and correlating input videos to a 3D model library.

Uploaded by

farooqespn
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
32 views

Video Based 3D Human Activities Analysis: By: Jacob Randall Autumn Smith Muhammad Farooq

The document describes research on analyzing 3D human activities through video. It discusses developing algorithms to track a person's movements in video, convert that information to a 3D model, and match the model's movements to the original video. The goals are to reconstruct motions and gestures as accurately as possible using techniques like motion capture suits, feature vector extraction from skeleton images, and correlating input videos to a 3D model library.

Uploaded by

farooqespn
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 25

Video Based 3D Human Activities Analysis

By: Jacob Randall Autumn Smith Muhammad Farooq

Outline

Background and motivation Goals and flow map Methods and Algorithms Results and Discussions Service Learning Project Questions and comments

Background and Motivation

Object Implantation, Removal, and 3D Modeling in films Human Machine Interface (HMI) & Virtual Reality
o

Methods

Green screen
Body suits
o

Cost Efficient Sometimes unrealistic.

Background & Motivation

Green screens

Pressure sensor

Motion capture suit

Virtual Reality

Motion reconstruction

Gesture Reconstruction

Goals and Objectives



Track a person on film Convert information to 3D walker Trajectory, follow movements Match 3D model as closely as possible

Method & Algorithm

Motion reconstruction (feature vector)

Gesture reconstruction (correlation based)

Feature Vector Algorithm

Results and Discussion

Background Subtraction
o

This difference between two images gives the foreground objects. If Ii is the current frame and Ib is the background image the foreground image Ifi is Ifi = Ii Ib

Converted to Binary Image Otsus Method for finding optimum threshold for binary

conversion

Motion Segmentation

Noise Removal
o

Median Filtering

Smoothing operation
o

Erosion Completely remove objects smaller that the structuring element

Structuring Element = 15

Skeleton Image Extraction


o

Needed to extract features Uses MatLabs built-in function

Stick Model
o

Skeleton Image

Angle information gathered from silhouette of the image is used to form the stick model

Stick Model

Stick model over video

Feature Extraction
o

Using skeleton image find the mean location of all pixels with value of 1 above the first line.

Draw a vertical line passing through the head dividing image into 4 parts. In each part find the point which is at the maximum distance from the reference. Draw lines from these points to the points of intersection of the two horizontal lines

Find the angles of these lines giving us 4 angles (legs and arms). These angles create the feature
vectors.

End points of silhouette image

Feature vector
Get the feature vector
[ang1 ang2 ang3 ang3]

Get the feature vector


[ang1 ang2 ang3 ang3]

Input these to the 3D model


ang1 ang3

ang2

Input these to the 3D model

ang4

In which direction to move?

Some Challenges
Direction of motion
Left or right
arm/leg?

Check for the change in the part of the image

Left or right leg/arm?


Look for patterns in motion

Model is rotating its legs and arms at the same location?


Translate the model in each frame Change the axis for each frame

2D Walker to 3D Model Videos


Original Video Skeleton Model Stick model superimposed on the original video Motion reconstruction using 3D model Alternative view of 3D model

Gesture Recognition Algorithm

Correlation

3D Model Library

Gesture Recognition

Background subtraction Image is then converted to a binary image Otsus Algorithm was used to find optimum threshold Normalization
o o

Manually done using correlation coefficients computation. Normalized images from the input human video are compared to each image in library consisting of 3D model image silhouettes Multiply each matrix together to generate a correlation coefficient

Best Image is chosen

3D Motion Reconstruction
o

Each 3D image silhouette corresponded to the 3D image it was derived from.

Once all the 3D image silhouette images were determined, these images
were used to construct the 3D video based on the corresponding 3D images.

Gesture Reconstruction Videos


Binary Input Binary 3D 3D Model

Service Learning Project

Camera Installation
o

Decided camera location


Camera Software

Issues
o

Limited on cameras (6 total) Limited outlets

Layout of Kiddie World

Camera Software

Camera was turned on and the discovered by the program.

Main Screen on Camera Software

List of Cameras in Area

IP address was then set to the local address All cameras were set to the same IP Connected using HTTP

General settings for camera

Set IP address

Conclusion

We can reconstruct human movements using a 3D model using the algorithm

Algorithm can be improved in future work


o

The performance will increase if some Machine learning algorithm is applied to differentiate between the left and right arms and legs

Clustering techniques (unsupervised algorithm) can be used to find cluster of data and to calculate angles between these clusters.

Staff Contribution

All group members contributed to this project.


o

Muhammads main contribution was towards the implementation of feature vector based motion reconstruction. Jacob was mainly responsible for the implementation of the correlation

based gesture reconstruction.


o

Autumn was responsible for videos and 3D model based reconstruction implementation for both methods.

Reference

http://www.nbb.cornell.edu/neurobio/land/projects/hierarchy/ http://qh.eng.ua.edu/classes/fall2010/ece582/index_files/projects/group5 /index.html

qh.eng.ua.edu/research/iSMART/codes/simulation/3dReconstruction/wal
ker_test.rar

You might also like