RECOGNIZING INDIVIDUALS AND THEIR EMOTIONS USING PLANTS AS BIO-

SENSORS THROUGH ELECTRO-STATIC DISCHARGE

Buenyamin Oezkaya
Peter Gloor
MIT Center for Collective Intelligence
[email protected]

ABSTRACT use case applying electric field sensing to plants has not been
described before.
By measuring the electrostatic discharge of human bodies
together with Mimosa Pudica and other plants in response to 2. MEASURING THE ELECTROSTATIC
the human movement we have been able to recognize (a) DISCHARGE OF HUMANS WITH PLANTS
individuals based on their distinctive pattern of body
movements with 66% accuracy as well as (b) positive or For our experiments, we are using the commercially available
negative mood based on their gait characteristics with 85% Plant SpikerBox2, an Arduino-based data acquisition system
accuracy. We use the Plant SpikerBox, a device that measures with filters for plant signal measurement with a low bias-
the electrical action potential while also measuring the current amplifier already installed. For checking the
electrostatic discharge between the electrode on the leaves of environmental conditions, we used a RaspberryPi-based
a plant and the capacitively coupled human body. monitoring system. Using different sensors for soil-moisture,
CO2-concentration, temperature, and humidity, we were able
Index Terms— plants, sensors, IoT, electrostatic
to check the environmental conditions change during a
discharge, human-plant interaction
measuring session. The changes in all measured fields were
negligible.
1. INTRODUCTION Human gait has, in the past, been measured through
electrostatic discharge (Chen et al. 2012). When a human
In this paper, we describe how plants can be integrated as walks, steps, or jumps, static electricity is produced as a result
biological parts into an electric field detection unit to sense of friction between the body and clothing. The separation
human behavior and emotions. Embedding plants into between the human foot and the ground during walking also
sensors is a natural and powerful extension of the Internet of charges the human body (Li et al. 2018).
Things. Plants naturally pervade our environment and are
significantly less expensive in terms of production, operation,
and maintenance costs than any artificial device (Manzella et
al. 2013). Furthermore, plants do not fall under privacy and
GDPR1 concerns. Moreover, conventional wireless systems
usually store data or transmit data through Bluetooth. Unlike
wearables, direct contact between the electrode and the
human body is not necessary. Our approach is affordable,
wear-free, and fit for long-time monitoring during daily
activities. Furthermore, no devices need to be placed on the
subject’s body, which may have been perceived as
uncomfortable or obtrusive. In earlier, related work,
researchers have used plants as sensors for environmental
monitoring, e.g., pollution, and fires, in agriculture, e.g., for
monitoring irrigation, plant health, and the use of chemicals,
and area monitoring, e.g., for avalanches, and flooding
(Chatterjee et al. 2015). To the best of our knowledge, this
(a)

1 2
General Data Protection Regulation https://backyardbrains.com/products/plantspikerbox

1
 789 ;<
𝑖(𝑡) = = 𝐼(𝑡) (2)
7: =>(:)( ?@(:)(
7 !
𝐼(𝑡) ∝ 𝐶EFGH: 7: I" J (3)
#

The influence on the amplitude of measured electric field

strength caused by environmental factors such as temperature,
humidity, and clothing should be taken into consideration
(Chubb 2008) as they will cause noticeable variations on
amplitudes of measured signals. However, these
environmental conditions will not change during a measuring
session. The influence of the session itself is discussed in
section 4.

2.1 Experimental Setup

A high-impedance voltage amplifier that is attached to the

plant and capacitively coupled to the human body can sense
this change in static electricity. Figure 2 visualizes the key
components of our experimental setup. In our approach, we
use a single-ended measurement of the plant stem using the
(b)
soil as the local ground. The electrode can be used for electric
Figure 1. Electrostatic discharge is shown while experimenter walks
field sensing due to the high resistance of the plant. In
by Mimosa Pudica which is connected to plant spiker box (PSB) (a)
Schematic Diagram of Experiment (b) addition to electric field sensing, we can measure the action
potentials of the plant generated by physical stress, like
This change in static electricity arises when the foot makes or touching the leaves.
breaks contact with the floor. A negative charge occurs when
the foot contacts the ground, while a positive charge is
generated when the foot detaches from the ground (Chen et
al. 2012). Therefore, it is possible to distinguish a foot
making and breaking contact. These effects are known as
contact electricity charges. During the walking motion,
human capacitance (CB) can be expressed by Equation (1)
(Chen et al. 2012). The capacitance between the foot and the
Ground (CF) depends on the height of the sole. Both CF and
Figure 2. Schematic of our Data-Acquisition system
the capacitance relative to nearby objects inside the room (Cr,
i) is relative to the position of the individual.
The electrical signals must be converted to features suitable
! ! ! ! for machine learning. Spectrograms are commonly used for
"#
= " + " + ∑-
./! " = 𝑓(𝑡) (1) ECG/EEGs (Colomer et al. 2016) and are utilized(1) for audio
%& %( *,,
classification in deep learning applications (Hershey et al.
2017). In the past, researchers have extracted statistical
The induced current i(t) due to the human walking motion is
features like MFCCs (Mel-frequency cepstral coefficients)
illustrated in equation (2) and (3). Qe is the charge induced on
for audio analysis (Tahon & Devillers 2016). The gathered
the plant electrode. The capacitance between the human body
signals in our approach look similar in envelope, shape, and
and the plant electrode is Cplant. S represents the equivalent
frequency to signals in audio analysis. MFCCs are a small set
area between the human body and the plant. X (t) and y(t)
of 10-20 features that describe the overall shape of a spectral
illustrate the horizontal distance and vertical distance, thus
envelope. In this paper, we used the parameters shown in
considering the radial distance to the plant. The permittivity
Table 1 for feature extraction using MFCC. The sampling rate
of the air is described with 𝜀. I(t) takes to account the induced
is given through the Plant SpikerBox. One reason to calculate
current generated by the feet motion without considering the
MFCCs is to reduce the dimensionality of the amplitude
variation of capacitance between the human body and the
spectrum, as well as to capture its envelope.
plant electrode, e.g., stepping in place.

2
 Features : MFCC Mimosa Pudica
Sampling Rate 10000
Number of MFCCs 20 Cropping and Z-Transformation
Window Size 2500
Hop Length 1250 Mel-Frequency-Spectogram
Exponent of
2
Magnitude

Table 1. Parameters to extract MFCC from a sample

The first step is to measure the energy in the filter banks.

Those triangular filters are spaced over the Mel scale
(Marechal et al. 2019). The denser resolution in low
frequencies compared to higher frequencies fits our approach MFCC Feature Vector
since most of the information in the electrical signal we are
measuring is at lower frequencies. The importance of the
envelope in our application of gait-based electrostatic Random-Forest Classifier
discharge was shown in (Chen et al. 2012). The Window Size Figure 3. The machine learning process for people/emotion
and Hop Length for calculating the MFCCs were chosen to recognition
fit best for our application of electric field sensing and
analyzing human gait of 1-2Hz (Li et al. 2019). Every feature
has an additional time dimension to take walking time into In general, decision trees tend to be robust since they can
account, resulting in additional calculations every 100ms. work with different types of data and outperform other
methods such as linear or logistic regression. They have no
2.2 Applying Machine Learning for Individual Prediction problem in the case of heterogeneous features or mixing
completely different ranges of values (Strobl et al. 2007). To
Figure 3 describes the process we are applying for analysis, avoid overfitting and to make full use of the experimental
converting the electrical signals captured with the Plant data, a 10-fold Cross-validation algorithm was used for the
SpikerBox into 20 MFCC signals, which are used to train a train/test data split. A stratified algorithm was used to ensure
machine learning model. For plants, we have used Mimosa that each fold has the same proportion of observations with a
Pudica, common basil, thyme, and orchids. We have also given categorical value. An additional shuffling of the data
been able to obtain the same electrical signal when before the stratified 10-fold split was introduced to avoid
connecting the electrodes of the plant spiker box to a fresh overfitting even further. For the hyperparameters of our
twig of hazel, beech, or apple tree with the grounding wire random forest, we used a randomized search optimization
connected to the water in the vase where the twig was using over 7920 possible combinations and 100 iterations
standing. (Bergstra & Bengio 2012).

Each gathered electrical signal had been trimmed so that 3. PREDICTING THE MOOD OF DIFFERENT
every sample had the same length. Subsequently, each gait- PEOPLE BASED ON THEIR WALKING PATTERN
sequence was standardized using the Z-Transformation. In
doing that, we eliminated the effect of different amplitude Dysphoric mood can be observed in gait patterns of
dimensions on subsequent analyses. By using MFCCs as our individuals. Characteristics associated with being sad are
main feature, the dataset was reasonably large enough to use smaller amplitudes in vertical movements of the upper body
a decision tree based random forest classifier. Good accuracy and reduced walking speed (Michalak et al. 2009). For our
has been achieved using decision trees to predict dangerous experiment, four individuals have walked “sad” and “happy”
chemicals from electrical signals in plants (Chatterjee et al. accordingly, collecting over 139 samples. Figure 5 shows two
2017). For predicting human gait characteristics using samples of z-transformed signals. Signal A shows the
electric field sensing, decision trees have also been shown induced electric current of someone walking sadly towards a
viable (Li et al. 2019). plant over time. The slower footsteps are shown as the spikes
appear in much lower frequencies. The increasing amplitude
results from the decreasing distance of the capacitively

3
coupled human and the plant. Signal B shows the signal of a
human walking happily away from a plant. The spikes are not
only higher in amplitude but also occur at a higher frequency.
Signal B is shorter than signal A since the walking speed of a
happy person is higher. Using the random forest classifier, an
accuracy of 85.5 % was achieved (Cohens κ=0.711,
AUROC=0.856). Table 2 shows an overview of the used
hyperparameters for the “sad” and “happy” classification.

Classifier : Random-Forest
N 139
Number of Classes 2
10-Fold CV and 80/20
Cross-validation
Train/Test Split (b)
Figure 4. Sample signals of one person walking sadly (a) and
Estimators 100 happily towards a plant (b)
Minimum Samples
5
to split Node 4. PREDICTING INDIVIDUAL PEOPLE THROUGH
Minimum Samples THEIR GAIT
4
per Leaf
Maximum depth Similarly, electrostatic discharge of 6 people was recorded,
100 identifying the correct person based on her/his gait. Two
of Tree
electrical signals gathered from two different persons
Samples drawn
False walking by a plant are shown in Figure 5. The amplitude is
with replacement increasing while walking towards and decreasing when
walking away from the plant in both signals. While the
Table 2. Hyperparameters of our random forest classifier frequency of steps seems to be identical, the way the foot rises
and moves is different. The shapes around the area of the
peaks change accordingly. The envelope of our gathered
signals is comparable to the simulations and measurements of
other studies in electric field sensing (Chen et al. 2012).
Using a random forest classifier with 100 trees, and applying
it on 212 samples classified into six classes, one per person,
66% accuracy was achieved (Cohens κ=0.51,
AUROC=0.876).

(a)

(a)

4
 Besides the 20 MFCC features, the plant type and the location
were included in the prediction, to investigate their influence.
Random forests can be used for feature selection. When the
algorithm is fit, features that are not useful will not be used to
split the data (Breiman 2001). For calculating the feature
importance, we used the Mean Decrease in Impurity (MDI).
Proportionally to the number of samples the tree splits, the
sum over the splits across all trees is calculated. As
mentioned above, besides Mimosa Pudica, different plants
were used. The measurements were taken in Boston at two
different locations, and as well as in Switzerland. As figure
7 illustrates, location and plant type were not essential
predictors; the MFCC features from the electrostatic
discharge signal were far more predictive. The feature
importance indicates that the type of floor, the type of plant,
(b)
Figure 5. Sample signals of two different people walking by
and the location do not play a significant role in identifying a
Mimosa Pudica person. The lower order coefficients of the MFCCs and thus
the lower frequencies of the electrical signal contain most of
In figure 6, we compared the random forest classifier to a the information about the individual human gait
baseline 0R-classifier, which always predicts the class with characteristics.
the most samples. When only two classes are to be predicted,
the accuracy of the random forest classifier reaches 88%
Accuracy comparable to the classification of “happy” and
“sad” walking. With more than four people, the accuracy of
the random forest classifier stays constant at 66%, whereas
the accuracy of the 0R-classifier falls because of the
additional diversification. This trend shows that our approach
could be used in crowded areas or in public events.

Figure 7. Feature importance for predicting different people

5. MEASURING LEG SHAKING WITH PLANTS

Leg shaking is often viewed as a negative behavior and is a

common human stereotype that many people exhibit while
sitting down. A study (Niehaus et al. 2000) found that over
half of the study’s population (N=750) experienced some
form of leg shaking. Leg shaking was reported to disrupt
normal function and carry a negative social stigma while also
being linked to medical conditions such as Attention Deficit
Hyperactivity Disorder (ADHD). However, subjects without
Figure 6. Accuracy of random forest classifier in comparison with
baseline classifier over the number of persons these conditions may also experience leg shaking due to
anxiousness or agitation. Additionally, significant relations

5
between the Big Five personality traits and leg shaking have of 88%. Similar to gait-characteristics, the behavior of sitting
been found. (Oshio 2018) individuals’ leg-shaking can be observed using our system.
Further analysis revealed that low-frequency MFCCs are the
To see how well our method works with leg shaking, we let most predictive feature for walking patterns. The type of plant
individuals sit on a chair and observed their leg shaking used, and the location where the experiment was conducted
besides the plant. In figure 8, the measured electrical signal is were not significant for the prediction.
shown over time. As soon as the individual starts shaking
their leg, a distinct pattern with a frequency of 5-6 Hz is Applications for this plant-based sensing system could be
visible. used in large scale office environments or public events.
(Xia et al. 2018) studied leg shaking prediction using Information about the happiness and other emotions of
accelerometers. They found that the power peak of the signal customers and employees could be used to improve office
is around 6 Hz. Using the ratio of frequencies and feeding it climate and performance. Smartwatch-based collaboration
into a Bayes’ classifier, they were able to get an accuracy of measurement tools such as the Happimeter (Roessler & Gloor
greater than 90%. As shown in figure 8, we were able to pick 2020) could be upgraded to enhance mood prediction using
up the same spectrum of signals using our method. plants as emotion sensors.

6. REFERENCES

[1] Breiman, Leo. (2001). Random Forests. Machine learning. 45.

5-32.
[2] Bergstra, J. & Bengio, Y.. (2012). Random Search for Hyper-
Parameter Optimization. The Journal of Machine Learning
Research. 13. 281-305.
[3] Chatterjee, S.K., Das, S., Maharatna, K., Masi, E., Santopolo,
L., Mancuso, S. and Vitaletti, A. (2015). Exploring Strategies
for Classification of External Stimuli Using Statistical Features
of the Plant Electrical Response. Journal of the Royal Society,
Interface / the Royal Society. 12. 10.1098/rsif.2014.1225.
[4] Chatterjee, S.K., Das, S., Maharatna, K., Masi, E., Santopolo,
L., Colzi, I., Mancuso, S. and Vitaletti, A. (2017). Comparison
Figure 8. Sample signals of one individual shaking their legs of decision tree based classification strategies to detect external
besides the plant chemical stimuli from raw and filtered plant electrical response.
Sensors and Actuators B: Chemical, 249, pp.278-295.
Our system would have applicability for instance for patients [5] Chen, X., Zheng, Z., Cui, Z. Z., & Zheng, W. (2012). A novel
without medical conditions like Restless Leg Syndrome, so remote sensing technique for recognizing human gait based on
they can voluntarily halt their leg shaking once aware of the the measurement of induced electrostatic current. Journal of
motion. The individual can be notified in real time once our Electrostatics, 70(1), 105-110.
setup detects leg shaking. [6] Chubb, J.N.. (2008). Tribocharging studies on inhabited
cleanroom garments. Journal of Electrostatics - J
ELECTROSTAT. 66. 531-537. 10.1016/j.elstat.2008.06.001.
6. CONCLUSION AND FUTURE WORK [7] Colomer, A., Fuentes, F., Naranjo, V., Guixeres, J., Ausín, J.,
Alcañiz Raya, M. (2016). A Comparison of Physiological
In this paper, we investigated how plants and humans can be Signal Analysis Techniques and Classifiers for Automatic
connected through electric field sensing. Simple data Emotional Evaluation of Audiovisual Contents. Frontiers in
acquisition systems and electrodes on plants enable us to pick Computational Neuroscience. 10.
up the electrostatic discharge of humans that are passing by. [8] Niehaus,D.J., Emsley, R., Brink, P., Stein, D. (2000).
These signals contain information on gait characteristics that Stereotypies: Prevalence and Association with Compulsive and
can be extracted using features like MFCC. With a 10-fold Impulsive Symptoms in College Students. Psychopathology.
33. 31-5. 10.1159/000029116.
cross-validated random forest classifier, it was possible to [9] Hershey, S., Chaudhuri, S., Ellis, D.P., Gemmeke, J.F., Jansen,
predict one out of six humans that are walking by the plant A., Moore, R.C., Plakal, M., Platt, D., Saurous, R.A., Seybold,
with 66% accuracy. Dysphoric mood observable in gait- B. and Slaney, M., 2017, March. CNN architectures for large-
patterns of these humans could be predicted with an accuracy scale audio classification. In 2017 ieee international conference
on acoustics, speech and signal processing (icassp) (pp. 131-
135). IEEE.

6
[10] lkov, A. (2006). Plant electrophysiology: Theory and methods. Associated With Dysphoric Mood, Psychosomatic Medicine
10.1007/978-3-540-37843-3. 71:580–587 (2009).
[11] Li, M., Tian, S., Linlin, S., Chen, X.: Gait Analysis for Post- [16] Oshio, A. (2018). Who Shake Their Legs and Bite Their Nails?
Stroke Hemiparetic Patient by Multi-Features Fusion Method. Self-Reported Repetitive Behaviors and Big Five Personality
Sensors 2019, 19, 1737; doi:10.3390/s19071737 (2019) Traits. Psychological Studies. 63. 10.1007/s12646-018-0462-x.
[12] Li, M., Tian, S., Linlin, S., Chen, X.: Estimation of Temporal [17] Strobl, C., Boulesteix, A., Zeileis, A., Hothorn, T. (2007). Bias
Gait Parameters Using a Human Body Electrostatic Sensing- in Random Forest Variable Importance Measures: Illustrations,
Based Method. Sensors 2018, 18, 1737; Sources and a Solution.” BMC Bioinformatics, 8(1), 25. BMC
doi:10.3390/s18061737 (2018) bioinformatics. 8. 25. 10.1186/1471-2105-8-25.
[13] Manzella, V., Gaz, C., Vitaletti, A., Masi, E., Santopolo, L., [18] Tahon, M., Devillers, L.: Towards a small set of robust acoustic
Mancuso, S., Salazar, D., Heras, J. (2013). Plants as sensing features for emotion recognition: Challenges. IEEE/ACM
devices: the PLEASED experience. SenSys 2013 - Proceedings Transactions on Audio, Speech, and Language Processing
of the 11th ACM Conference on Embedded Networked Sensor 24(1), 16–28 (2016). DOI
Systems. 10.1109/TASLP.2015.2487051Authors, “Frobnication
[14] Marechal, C., Mikolajewski, D., Tyburek, K., Prokopowicz, P., tutorial,” 2012, Supplied as additional material tr.pdf.
Bougueroua, L., Ancourt, C., Wegrzyn-Wolska, K.: Survey on [19] Xia, S., Wei, P., Vega, J., Jiang, X. (2018). SPINDLES+: An
AIBased Multimodal Methods for Emotion Detection, pp. 307– adaptive and personalized system for leg shake detection.
324. Springer International Publishing, Cham (2019) Smart Health. 9-10. 10.1016/j.smhl.2018.07.019.
[15] Michalak, J., Troje,N.F., Vollmar, P., Heidenreich, T., Schulte,
D.: Embodiment of Sadness and Depression—Gait Patterns