University of Maryland College Park

Department of Electrical and Computer

Engineering

ENEE 461
Control System Laboratory
Spring 2024

Final Project
Lab Title: Robo Keeper

By
Saleh Kemal, Austin Lin, Jennifer Riley

Date: 5/16/2024
Introduction
The goal of this project was to build an automatic goalkeeper, a robokeeper. This robokeeper
would be able to detect a ball and predict the trajectory of its path so that the goalkeeper could
move to block the ball from scoring a goal. This project was inspired by Fraunhofer's
RoboKeeper which can be used for soccer as well as hockey, and has been able to block goals
from professional players due to its high speed response time. Because of its goal blocking
efficiency, the robokeeper is a great tool that can be used to train athletes and improve their
strategies, for better overall game performance. Due to time and material constraints and for ease
setup and testing we built a tabletop sized version of the robokeeper.

Set Up and Parts list

Parts:
● DC motor
● Motor driver
● Arduino Uno
● Arduino and jumper cables
● Camera @ about 100 FPS
● Ping pong ball
● Cardboard field and goal ‘net’
● 3D printed goalie
● 3D printed motor attachment (to hold the motor in place)
● Rectifier (for the motor driver)

We attached a 3D printed goalkeeper to a DC motor and motor driver (which required a

rectifier). We programmed and controlled our robokeeper with an arduino uno board. We built a
field and goal ‘net’ out of cardboard to test our desktop demo. We then attached a camera to the
top of the ‘net’ to track the ball. Our setup is depicted below.
 Fig. 1: Field testing setup

HSV and Servo Angle

We used HSV (Hue, Saturation, Value) color space to detect the ball. The detection parameters
included threshold values for the HSV color space to isolate the ball's color from the
background. Using the center of the frame as a reference, we calculated the angle by applying
basic trigonometry. By connecting a line from the center to the point being measured, this line
became the hypotenuse of a right triangle, allowing us to find the angle.

Once the angle was found, we used the motor's gear ratio to calculate the target position. The
motor had an encoder with 800 pulses per 360 degrees. We multiplied the angle by 800/360 to
convert it to encoder pulses. The motor had a 50:1 turn ratio, and this ratio was used to determine
the precise position the motor needed to achieve.
Figure 2. Angle calculated with respect to Center

Figure 3.. HSV Mask

Software: Threading

We employed threading to continuously capture frames and calculate the target position. This
method allowed us to avoid issues with capturing and sampling simultaneously. With threading,
target values were continuously updated as a global variable. When sampling was needed, it was
grabbed by the Arduino thread. Capturing, calculating, and sending data in one loop caused the
system to vibrate. This is discussed later in the report.

Controller
In our project, we implemented a Proportional-Integral-Derivative (PID) controller to manage
the position of a DC motor, aiming to align it with a target position represented by the ball. This
controller is similar to the one used in previous experiments, specifically LAB 3 and 4. The
primary objective was to precisely control the DC motor's position to match the target location.

Implementation
To achieve the desired control, we used a PID controller with a sampling interval of 0.01 seconds
for both the servo position and the target position. The camera thread continuously calculated the
target position, although updates occurred every 0.01 seconds. Similarly, the encoder position
was updated at the same interval. As will be discussed later, having different sampling
frequencies caused instability, vibrations, and fluctuations in the encoder readings.

A zero-order hold was employed for the signal between sampling periods. If the camera or
position data were not read during the sampling time, the previous position and target position
were used.

Methodology
We applied discrete-time techniques such as the midpoint approximation for the integral and
discrete-time derivative for the PID controller. The PID value was then fed into the control
system. Our PID Controller calculation is depicted below.

Fig 4: Discrete Time PID controller

Proportional (Kp): Determines the reaction to the current error.
Integral (Ki): Accounts for past errors.
Derivative (Kd): Predicts future errors based on the rate of change.

Then through trial and error, we identified the Kp, Ki, and Kd values that provided the desired
system response.

Initial Testing with Unit Step Input

Initially, we tested the system with a unit step input, simulating a scenario where the motor needs
to rotate to either 0 or 180 degrees, the One Dimensional Problem. This binary problem can be
analyzed as a unit step input. The system's response to this input is shown in the figure below.
The motor tracked the target position accurately, demonstrating effective control by the PID
controller. In the upcoming figures, Target refers to the ball position, while Position refers to the
goal keeper position. Note the Y axis is in encoder pulses. 800 encoder pulses=360 degrees
rotation.

Fig. 5: System response with Unit Step Input.

Camera Input Testing
Next, we tested the system with the camera as the input, moving the ball to 0 degrees and 180
degrees. This test was similar to the unit step input test, but with additional complexity. We had
to carefully manage the sampling rates with the camera and ensure that the system could handle
scenarios where no input was received during a sampling period. In such cases, the system
needed to properly hold the previous position and target goal to maintain stability and accuracy.

During these tests, the controller successfully tracked the ball position, as depicted in the figure
below. The system maintained accurate tracking despite the challenges posed by intermittent
input and varying sampling.

Fig. 6: System response with Camera input.

Small overshoots were observed but were practically negligible, well within the acceptable 30%
range. The DC motor has a 50:1 gear ratio, and such minor overshoots do not affect our desired
response.
Dynamic Testing with Ball Release
Finally, we tested the system by releasing a ball from a ramp towards the goal. The goalie
successfully blocked the ball for most runs, as shown in the figure below. Additional videos and
images can be found in the Google Drive linked in the references section.

Fig 7. Target position from ramp and position of the goalie.

This test demonstrated the system's ability to dynamically adjust and react to moving targets
effectively. However, if the ball was moving too fast, the camera occasionally failed to capture
frames in time, resulting in a slow system reaction. To address this, we would need a faster
camera, as we are currently shooting just above 100 FPS. This limitation also contributed to the
difficulty in solving the 2D problem with our current setup.

Challenges Encountered

Encoder Bouncing and Sampling

One significant challenge we faced was encoder bouncing, which occurred due to non-uniform
sampling. Sampling refers to the process of measuring the state of a system at regular intervals.
In our system, sampling involved reading the position of the encoder and capturing the target
position via the camera at specified intervals. Non-uniform sampling intervals caused instability
and vibrations in the system. This inconsistency led to fluctuating encoder readings, resulting in
poor control performance. Ensuring uniform sampling intervals is crucial to maintain system
stability and accurate position tracking.

Sampling, Frame Grab, and Delay

Initially, our approach involved capturing frames, calculating the target position, and sending this
data to the Arduino within the same while loop. This method worked well for a step function
input from Python. However, when using the camera, the system exhibited erratic behavior. It
would vibrate and proceed to unknown values. This behavior is depicted below. We discovered
that a 0.03-second delay in frame capture caused these issues. While this delay should be
tolerable for a slowly moving ball, the system behaves unpredictably, even when recognizing a
stationary ball.
 Fig 8. System Response When Frame Capture is in the Same While Loop
We hypothesized that the problem might stem from the OpenCV frame capture function or
because the entire system paused during frame capture. We resolved this issue by implementing
threading, as discussed earlier, and sampling every 0.01 seconds. This approach allowed the
camera and the position readings to be processed in parallel, reducing the delay and improving
system stability.

Another challenge was the camera's frame rate, which was around 100 FPS. This rate is
insufficient for accurately tracking a fast-moving ball, especially when using HSV filters for
object tracking, due to motion blur. Therefore, solving the 2D problem with our current setup
proved too difficult.
Arduino Performance
Another critical challenge was the performance of the Arduino. The computer could write data at
faster intervals, but the Arduino was slow at capturing position signals and communicating with
the computer. This bottleneck hindered the overall system performance, as the Arduino could not
keep up with the fast data exchange required for real-time control. Future improvements could
involve using a faster microcontroller or optimizing the communication protocol between the
computer and the Arduino to enhance data transfer rates and reduce latency.

Conclusion
Because of the frame rate of the camera we used and the speed of the motor, the response time of
the robokeeper was rather slow. However when the ball approached the goal at a speed the
camera was able to track the robokeeper was able to block the ball most of the time. To improve
the performance of the robokeeper we could use a camera with a higher frame rate per second,
such as 120 frames/second, so the camera could track the ball when it moves at high speeds. If
the ball was able to go faster we would also need a motor with a faster response time, or the ball
would be in the ‘net’ by the time the motor moves the goalkeeper into position.
 References
Google Drive for Testing videos: Testing Videos

PID Controller:
DC motor PID speed control
https://www.arrow.com/en/research-and-events/articles/pid-controller-basics-and-tutorial-pid-im
plementation-in-arduino

Object Tracking:
https://pyimagesearch.com/2015/09/21/opencv-track-object-movement/

https://openaccess.thecvf.com/content/ACCV2022/papers/Zhang_Tracking_Small_and_Fast_Mo
ving_Objects_A_Benchmark_ACCV_2022_paper.pdf