Motion Analysis in Image Processing using ML
- 1. Motion Analysis and Object Tracking • The goal of this chapter is to cover motion analysis and the tracking of objects. • The following topics are covered in detail in this chapter: • Object tracking techniques, including using frame differencing to learn some information about an object in motion • Background and foreground subtraction • Using optical flow techniques for object feature tracking • Building interactive object tracking using the mean shift and cam shift techniques
- 2. Introduction to Object Tracking • Object tracking is the process of estimating the exact position of an object while the object is in motion or across consecutive image frames within a video. • Figure 1 shows the basic steps for tracking an object.
- 3. Applications • There are many applications of object tracking, such as security surveillance, augmented reality, traffic monitoring, self-driving cars, action recognition, and so on. • The ability to track an object in a video depends on multiple factors, such as knowledge about what the object in context is, what parameters of the object are being tracked, and what type of video is showing the object.
- 4. Challenges of Object Tracking • the following challenges need to be dealt with: Object occlusion: • When the target image is hidden behind something, it is difficult to both detect and update when future images come in. Speed: • When the motion of an object is fast, the output video usually is blurred or jittery. Hence, any sudden changes in the motion of cameras lead to problems in tracking applications.
- 5. Challenges of Object Tracking • Shape: • Tracking objects that are non rigid (i.e. the • shape is not constant) will result in failure on object • detection and thus tracking. • False positives: • When there are multiple similar objects, • it is hard to match which object is targeted in subsequent images. The tracker may lose the current object in terms of detection and start tracking a similar object.
- 6. Object Detection Techniques for Tracking • The first step in the process of object tracking is to identify objects of interest in the video sequence and to cluster pixels of these objects. • Since moving objects are typically the primary source of information, most methods focus on the detection of such objects. • This is also referred to as tracking by detection
- 7. Object detection techniques applied for object tracking.
- 8. Frame Differentiation • Frame differencing is a technique to measure the difference between two video frames by observing the position of one or more objects in context. • The pixel definitions are observed for any changes as this is an indication of changes in the image. A pixel change indicates a change in the image.
- 9. Frame Differentiation • Frame differentiation is all about determining the presence of moving objects by calculating the pixel difference between two consecutive images in a video. • Frame differentiation techniques have high accuracy and relatively lower or moderate computational time; this method works well for static backgrounds.
- 10. Background Subtraction • Background subtraction is an important preprocessing technique in vision-based applications because it helps separate the background from the foreground in video streams.
- 11. Background Subtraction • An interesting use case for this technique is a ticket counter where the background is static but the foreground has visitors coming to the counter to buy tickets. • The requirement could be counting the number of visitors coming to the counter in the day. In this case, you first need to extract each person alone.
- 12. Background Subtraction • If there is an image or a frame of video that just has the static background and no moving visitors, it is a straightforward task because all you need to do is subtract the new image from the background to extract the foreground alone.
- 13. Optical Flow • Optical flow denotes the motion of the objects in an image from one frame to another that is caused by either the motion of the image or the camera. • It is represented as a 2D vector field that has each element representing the movement of the points from one frame to another. Figure 2 represents the movement of a ball from one position to another across five consecutive frames.
- 15. Optical Flow • The optical flow method assumes that there is no change in the pixel intensities of an object between consecutive frames, and neighboring pixels also have similar motion. • Optical flow methods are relatively high in computational time and moderate on accuracy.
- 16. Object Classification Object classification techniques
- 17. Shaped-Based Classification • There are many descriptions of shape information about motion regions such as the representation of points, blobs, or boxes to classify a given object. • Classification is done on every frame for the object, and the results are stored in a histogram. • Shape-based classification has a lower computational time and a relatively lower accuracy because template matching techniques can be applied
- 18. Motion-Based Classification • Moving objects have a periodic property called residual flow that can be used for classification. • Residual flow is used to analyze the rigidity and periodicity of the moving objects. • Rigid objects present a little residual flow, whereas a nonrigid moving object like a human being has a higher average residual flow and displays a periodic component.
- 19. Motion-Based Classification • Motion-based classification has a high computational time and relatively lower accuracy. • Though it doesn’t require templates, it fails to identify a static human/nonrigid object.
- 20. Color-Based Classification • Color usually is not the most appropriate feature of an object to use for classification, but, among all the object features, color is fairly constant and can also be easily acquired. • Furthermore, it is one of the features that can be exploited when needed. • Color histograms are used to detect and track vehicles in real time.
- 21. Color-Based Classification • A Gaussian distribution model is used to understand the color distribution in a sequence of images, which is useful to segment the background and the object. • Color-based classification has a higher computational time and relatively higher accuracy because template matching techniques can be applied.
- 22. Texture-Based Classification • This techniques uses gradient orientation in the selected portions of the image. • This method can result in more accuracy because it uses overlapping contrast normalization in a dense grid of uniformly spaced calls. • Texture-based classification has a higher computational time and relatively higher accuracy than other methods.
- 23. Object Tracking Methods • An object is tracked to extract objects, recognize and track objects, and make decisions about activities. • Object tracking, at a high level, can be classified as point tracking, kernel-based tracking, and silhouette-based tracking
- 24. Some Object Tracking Techniques
- 25. Point Tracking Method • Point tracking is done using the feature points of the moving object. There are three methods for point tracking: 1. Kalman filtering 2. particle filtering 3. and multiple hypothesis.
- 26. Kalman Filtering • Kalman filter estimates the state of some quantities by combining two kinds of information • It uses a restrictive probability density propagation algorithm. • It supports estimation of past, present, and future states using its efficient recursive estimation techniques. • The goal of KF is to reduce the noise from measurements • There are two kinds of equations: 1. time update equations. 2. Measurement update equations.
- 27. Cont : • Time update equations provide a future state using the details of the current state and error covariance estimations, and measurement update equations help in the feedback process in the recursive flow.
- 28. Particle filtering • Considers a variable at one time and • generates all the models for that variable. • This method supports the dynamicity of variable states and also allows for a new operation of resampling. • Particle filtering overcomes the restrictions that Kalman filters pose because they use contours, color features, or texture mapping.
- 29. Multiple hypothesis tracking • It observes more than one frame for better tracking results. MHT is an iterative mechanism as well. • Every iteration starts with an existing track and a hypothesis that has a set of disconnected tracks. • For each hypothesis, the future position of the object in the next frame is predicted. • Each of these predictions is compared using distance • measures. • MHT can track multiple objects and also handle occlusions.
- 30. Kernel-Based Tracking Methods • Kernel-based tracking methods measure a moving object’s emerging region between frames. • The object’s movement can be a parametric • motion such as a translation, conformal, affine, and so on.