www.nature.

com/npjflexelectron

ARTICLE OPEN

Deep learning-enabled triboelectric smart socks for IoT-based

gait analysis and VR applications
Zixuan Zhang1,2,3,4,8, Tianyiyi He1,2,3,5,8, Minglu Zhu1,2,3,5, Zhongda Sun 1,2,3
, Qiongfeng Shi 1,2,3,5
, Jianxiong Zhu1,2,3,5,
Bowei Dong1,2,3,5, Mehmet Rasit Yuce6 and Chengkuo Lee 1,2,3,4,5,7 ✉

The era of artificial intelligence and internet of things is rapidly developed by recent advances in wearable electronics. Gait reveals
sensory information in daily life containing personal information, regarding identification and healthcare. Current wearable
electronics of gait analysis are mainly limited by high fabrication cost, operation energy consumption, or inferior analysis methods,
which barely involve machine learning or implement nonoptimal models that require massive datasets for training. Herein, we
developed low-cost triboelectric intelligent socks for harvesting waste energy from low-frequency body motions to transmit
wireless sensory data. The sock equipped with self-powered functionality also can be used as wearable sensors to deliver
information, regarding the identity, health status, and activity of the users. To further address the issue of ineffective analysis
methods, an optimized deep learning model with an end-to-end structure on the socks signals for the gait analysis is proposed,
which produces a 93.54% identification accuracy of 13 participants and detects five different human activities with 96.67%
accuracy. Toward practical application, we map the physical signals collected through the socks in the virtual space to establish a
1234567890():,;

digital human system for sports monitoring, healthcare, identification, and future smart home applications.
npj Flexible Electronics (2020)4:29 ; https://doi.org/10.1038/s41528-020-00092-7

INTRODUCTION long-term connectivity of the whole system. To solve this

Wearable electronics experienced enormous development and foreseeing energy crisis issue, energy harvesting technologies
advancement in the past decades benefiting from its intrinsically and advanced storage devices have emerged to make the waste
superior flexibility and portability1–3. To further improve the energy in our surrounding environment valuable. Motivated by
quality of life, the invention of various wearable sensors based on that, substantial efforts have been made for the development of
electrocardiograph4, electromyogram5, body temperature6, heart wearable systems equipped with the abovementioned advanced
rate7, strain sensors8, etc., offers opportunities in both fitness technologies and potential self-sustainability18–21.
service and medical diagnostics by the long-term monitoring of In the past few decades, several of flexible devices have been
physiological signals. In particular, the detection of the human reported with different device structures and materials22–24.
motions is significantly valuable in creating insights into the user’s Textile as a fundamental part of normal garments has been
health status, activity quantifying, and the establishment of an extensively investigated as a great flexible and stretchable
effective channel between humans and machines9,10. When it electronics platform25–28. There are a few research developments
comes to the continuous and convenient monitoring of diversified of smart textiles based on various mechanisms, aimed to be not
human motion states, cameras and inertial measurement unit only more flexible for an improved comfortability but also
(IMU) sensors are the widely adopted devices for smart home multifunctional, i.e., sensation, perception, or integration29–34.
applications11,12. However, vision recognition by cameras may Triboelectric nanogenerator (TENG) gradually becomes an optimal
cause privacy issues, and the utilization of IMUs are not option for scavenging waste energy, and sensing of both physical
intrinsically flexible and comfortable enough as the preferable and chemical parameters based on the textile platform due to the
wearable solution. In addition, wearing a few bulky IMUs on particular advantages, including various choices of materials, easy
human body could cause inevitable interferences to the motions. fabrication, low-power consumption, and low cost35–39. In this
With the aid of recent advances in the fifth generation wireless regard, textile-based TENGs (T-TENG) exhibit the superb ability of
networks and internet of things (IoT), immense widely allocated structural retention and fatigue resistance, during wearing and
wearable devices are expected to be wirelessly interconnected at washing40–43. Most of the T-TENGs are working under the vertical
rapid data exchange rates to provide concurrent communication contact-separation mode, lateral sliding mode, freestanding
of information about the human body13–17. Among them, wireless mode, and single-electrode mode44. He et al. developed a
sensor networks (WSNs) have become a key technology to analyze narrow-gap T-TENG, which can harvest mechanical energy from
information related to identification, healthcare, human–machine various body parts to power Bluetooth sensors45. Though the
interface (HMI), and human activity monitoring. However, a major narrow-gap textile is in a form most similar to the normal clothes,
bottleneck of the WSNs is the overall power consumption for hence it can be seamlessly integrated with garments, such kind of

1
Department of Electrical & Computer Engineering, National University of Singapore, 4 Engineering Drive 3, Singapore 117576, Singapore. 2Centre for Intelligent Sensors and
MEMS (CISM), National University of Singapore, 4 Engineering Drive 3, Singapore 117576, Singapore. 3Hybrid Integrated Flexible Electronic Systems (HIFES), 5 Engineering Drive 1,
Singapore 117608, Singapore. 4Smart Systems Institute, National University of Singapore, 3 Research Link, Singapore 117602, Singapore. 5National University of Singapore Suzhou
Research Institute (NUSRI), Suzhou Industrial Park, 215123 Suzhou, China. 6Department of Electrical and Computer Systems Engineering, Monash University, Melbourne, VIC 3800,
Australia. 7NUS Graduate School for Integrative Science and Engineering (NGS), National University of Singapore, Singapore 119077, Singapore. 8These authors contributed
equally: Zixuan Zhang, Tianyiyi He. ✉email: [email protected]

Published in partnership with Nanjing Tech University

 Z. Zhang et al.
2
structure is not suitable for pressure sensing, especially on foot for augmented interactions, which shows good potential on smart
due to its limited sensing range. The devices designed for foot home. Hence, this deep learning-enabled triboelectric smart sock
become extremely important because the activities of foot are one provides more comprehensive information to us, which would be
of the major sources for collecting kinetic energy from human advantageous for sports monitoring, healthcare, future smart
body46. Besides, gait contains useful information for the applica- home applications, etc. More importantly, the privacy issue is not
tion of healthcare and personal identification as well47–49. A great comprised when we use the socks for such purposes. In the future,
amount of effort has been devoted to developing insoles with the entire system could be flexible integrating the flexible print
energy harvesting and gait sensing functions, because the circuit boards (PCBs) with our sock, which would further improve
demand for the flexibility and wear comfortability of the insoles the wearing comfortability of the system73–76. In short, with the
is very low50–52. Thus, the socks can be an optimal option for gait integrated information collected from this sock, and additional
analysis in terms of suitability. In addition, compared with the wearable sensors and neural electrodes, this platform paves the
insoles, the socks can be more widely used in indoor scenarios, way to the realization of the digital human technology in near
such as wearing sock at home where shoes are not applicable. Zhu future (Fig. 1).
et al. reported a self-powered sock, which provides the simple
walking pattern recognition and motion-tracking information for
smart home applications, but the reported sock can only RESULTS
distinguish gait patterns with large differences and the recogni- Design, sensing mechanism, and characterization
tion accuracy is highly affectable by the environmental Though a narrow-gap T-TENG is thin and more resemble to the
variations46. normal clothes, the sensitivity and sensing range of it as a pressure
Artificial intelligence (AI) techniques will highly amplify the sensor is quite limited. To build up a T-TENG pressure sensor for
intelligence of wearable electronics, and provide more reliable yet gait detection which should have a large sensing range up to
simpler solutions to more problems and resonating tasks. The 200 kPa, a surface-structured T-TENG sensor is proposed and
conventional method of analyzing sensory information was fabricated. Here, we adopted the mm-scale frustum structure to
limited in the ability of handling natural data, which relies on be patterned on the silicone rubber surface with the aid of the 3D-
the manual extraction of shallow features from the raw data53–56. printed mold, which is low cost and highly scalable for mass
1234567890():,;

For decades, as a subfield of machine learning, deep learning has

production in future. The T-TENG sensor which contains four
shown its great potential in image processing, speech recognition,
functional layers, including a nitrile thin film, a silicone rubber film
human activity recognition, and so on, which provides an efficient
with patterned frustum structures (the detailed dimensions of the
way to learn higher-level features of the raw input from various
frustum with a square base as shown in Fig. 2a on the contact
sensing signals57–60. Hence, deep learning methods provide a
surface), and two conductive textiles attached to the back of the
promising and feasible solution to achieve high accuracy with a
two aforementioned contact electrification layers for charge
low computational cost for wearable sensors. Sundaram et al.
collection. Moreover, two nonconductive textile layers are used
developed a low-cost, scalable tactile glove using deep residual
networks for recognition of different grasping objects61. The to seal the device on the outer surface. The pressure stimulus will
combination of AI technology with the IoT technology will lead to induce charges due to the contact-separation mode based on the
the Artificial Intelligence of Things (AIoT) for supporting more working mechanism of the T-TENG sensor (Fig. 2b), and further
reliable communication between wearable devices and the cloud, flow in the external circuit to transform the mechanical energy
which will improve the capability of prediction in the more into electricity. To clearly demonstrate the necessity of the
complicated and dynamic system62–65. With the advent of the patterned surface structure, a small piece of 3 cm × 3 cm of the
AIoT, the digital twin enables real-time monitoring of systems and textile sensor with no surface structure is fabricated and
processes, seasonable analysis of data to prevent fault diagnosis, characterized. It can be observed that its output voltage saturates
and future upgrades and development66,67. The concept and quickly at 72 kPa with a sensitivity of 0.4 V kPa−1 in the first linear
application of the digital twin can be further broadened beyond range (Fig. 2c). It is advocated that the pressure on foot is in a
our imaginations by endowing the same concept with the large range up to 200–300 kPa, hence a large sensing range at
wearable electronics, i.e., to create digital replicas of the humans least up to 200 kPa is desired to capture the pressure information
in the virtual space. The digital human now becomes a viable of foot motions. Correspondingly, the open-circuit voltage of the
future technology with great opportunities for various applica- T-TENG sensor with the designed surface structures under
tions, ranging from personalized medicine and healthcare ascending applied pressures is measured and shown in Fig. 2c.
treatment, social media, and individual AI brain68–70. Benefiting from the mm-scale frustum structure to be patterned
In this work, we developed the triboelectric smart socks on the silicone rubber surface, a threefold higher sensitivity over
equipped with sensing capabilities that can provide a more the flat-structured textile sensor (first linear range from 10 to
comprehensive long-term monitoring of the user’s physical status. 70 kPa) can be observed, and the sensing range is successfully
In addition, this intelligent sock would be a feasible option for extended to >200 kPa, making it much suitable for gait sensing.
scavenging low-frequency energy from natural human body The open-circuit voltage waveforms under the high pressure of
motions to power a Bluetooth module, and transmit the body 244 kPa of the textile sensors with/without frustum structures are
temperature value detected by the embedded temperature provided in Fig. 2d, e. Notably, the absolute voltage value of the
sensor under IoT framework. The textile-based triboelectric frustum-patterned sensor is also increased by three times for
pressure sensors are designed and developed to be embedded the enlarged contact areas and the heightened spacing. The
on the commercial sock for gait monitoring. The self-generated characterizations of the transferred charge (Q) and the short-
output voltage spares the triboelectric sensors from external circuit current (ISC) of the T-TENG sensor are tested and sketched
power sources for operation compared to other sensing mechan- in Fig. 2f, h. The transferred charge of the T-TENG was measured
isms, such as capacitive sensors and resistive sensors71,72. Besides, under the load of 13, 38, and 244 kPa with the pressing and
by leveraging the deep learning technology for data analysis, the releasing speed of 15 mm s−1 (Fig. 2f). It can be observed that the
intelligent socks are successfully demonstrated to realize the gait value of the transferred charge increases with the applied pressure
identification of group users with the accuracy of 93.54%. The as well due to the increased contact areas. Similarly, the short-
application of the virtual reality (VR) fitness game by using the circuit current of the T-TENG was measured, and the same trend is
proposed sock offers the possibility of a complementary control observed when the load pressure increases (Fig. 2g). Unlike open-
interface besides the common vision and voice control terminal circuit voltage and transferred charge, the amplitude of the

npj Flexible Electronics (2020) 29 Published in partnership with Nanjing Tech University
 Z. Zhang et al.
3

Fig. 1 The schematics and future prospects of the deep learning-enabled socks. The intelligent socks would assist wearable electronics to
move toward digital human for diversified applications: energy harvesting for IoT application, human activity monitoring for VR fitness game,
and gait identification for smart home application.

short-circuit current also depends on the pressing and releasing including the normal walking, loss of stride, and the freezing of
speed. As characterized in Fig. 2h, fewer peaks are observed when gait (FOG) are detected in sequence, resembling the eliciting of
the pressing/releasing speed decreases from 15 to 5 mm s−1, and the PD. The patient normally experiences the loss of stride ahead
the amplitude of the peak current is also lowered due to the of the FOG, which is characterized by the shortened intervals
reduced charge transfer speed. Without the specific illustration, between spikes. During the FOG, the patient couldn’t move the
the frustum-patterned sensor is used for all the following testing feet even with the intention to do so, which can be reflected by
and demonstrations. The durability and washability tests can be the small and irregular oscillating signals from the sensors. In
seen in Supplementary Figs. 3 and 4. addition, the smart sock was used to detect fall-down event from
A T-TENG sensor with a size similar to the foot of the user is the recorded gait signals as shown in Fig. 3d. An abnormal
fabricated and attached to the bottom of a cotton sock, to make negative peak with an extremely high amplitude against previous
an intelligent sock for gait analysis. By wearing the single-sensor normal data can be detected as the falling happens, and there is
sock on the right foot, the user’s gait can be detected and no positive peak generated thereafter, indicating the feet of the
analyzed through the triboelectric output voltages. In general, a user are still off the ground. Furthermore, there is a real-time
gait cycle includes four events as indicated in Fig. 3a, i.e., “heel remote healthcare monitoring system is illustrated, as a demon-
contact”, “toe contact”, “heel leave”, and “toe leave”. Based on the stration of the intelligent sock shown in Supplementary Fig. 7. The
working mechanism of the triboelectric textile sensor, there will be gait signals will be collected by a microcontroller unit (MCU) and
positive peaks generated as the pressure is applied on it, and transmitted to a PC via wireless communication. If the gaits from
reversed charge flow can be detected when the pressure is users appear the irregular signal compared with recorded normal
released. Accordingly, when the user walks forward, two positive walking signals, the emergency medical alert system will display
peaks representing the heal contact and toe contact are observed the corresponding warning, such as loss of stride or falling and
during the half cycle of gait, and two negative peaks representing inform the caregivers (Supplementary Video 1). Hence, the
heal leave and toe leave are detected thereafter (Fig. 3b). Thus, demonstrated real-time system for health condition monitoring
some preliminary features can be extracted from the output and medical emergency alert presents a great prospect of the
voltages, i.e., frequency of gait cycles, positive/negative peak intelligent sock in multiple healthcare applications to assist both
amplitudes, and the time interval of every event in a gait cycle the elderly and patients, which can be further used for smart
(more detail can be found in Supplementary Fig. 6). Through the home, hospitals, and healthcare centers. Meanwhile, this sock
detection of these characteristics, the types of human activities could be very useful in studying walking patterns in people
can be preliminarily monitored. Parkinson’s disease (PD), as a affected by illnesses, such as diabetes, musculoskeletal abnorm-
progressive and neurodegenerative movement disorder, usually ality, and rehabilitation from different injuries.
presents with symptoms of gait disturbance and even falls. To Besides of gait sensing, as intelligent sock works under the self-
show the possible application of healthcare monitoring of our powered triboelectric mechanism, it can also be used as an energy
proposed sock, mimetic motions of the PD that are frequently harvester to scavenge waste energy from foot motions. The
measured among the patients were performed, and the corre- output power of the single sock on the right foot is measured on
sponding detected signals are shown in Fig. 3c. Three conditions different external loads from 0.1 to 100 MΩ. As shown in Fig. 3e,

Published in partnership with Nanjing Tech University npj Flexible Electronics (2020) 29
 Z. Zhang et al.
4

Fig. 2 The basic characterization of T-TENG sensors. a The detailed dimensions of the frustum with a square base. b The working
mechanism of the T-TENG sensor which works in the contact-separation mode. c The open-circuit voltage versus pressure curve of T-TENG
sensors with no structure and frustum structure. d, e The schematics of T-TENG sensor and its open-circuit voltage waveforms at 244 kPa
without frustum structures and with frustum structures. f The transferred charge of the T-TENG under the load of 13, 38, and 244 kPa with the
pressing and releasing speed of 15 mm s−1. g The short-circuit current of the T-TENG under the load of 13, 78, and 244 kPa with the pressing
and releasing speed of 15 mm s−1. h The short-circuit current of the T-TENG with the pressing and releasing speed of 5, 10, and 15 mm s−1
under the load of 244 kPa.

the output power of 0.32 mW is generated from 1 Hz walking on a continuously, which was dropped at rest. As a result, the
44.4 MΩ load. In the case of running with the frequency of 2 Hz, monitored body temperature will provide an insight into the
owing to the larger force and frequency that the foot applied on physical state of a person, and the elevated body temperature will
the sock, a maximum power of 3.18 mW was measured on a be an indication of fever or vigorous exercise. Meanwhile, a TENG
21.3 MΩ (Fig. 3f). Hence, the proposed sock is capable of energy device connected in parallel with a high-voltage diode and a
harvesting from the gait cycles under various motions. The switch can effectively create instantaneous discharging output,
rectified output voltages from the sock charged different with an enhanced amplitude. Hence, an approach to create an
capacitors of 1, 4.7, 10, and 27 μF to be 5 V (Fig. 3g), instantaneous discharging for the socks with the integration of a
demonstrating the good charging capability of it as a reliable high-voltage diode is feasible to speed up the charging time
power source. Within ~300 times of steps in normal walking, the (more details can be found in Supplementary Note 1 and
output from the sock can charge a 27 μF capacitor up to 8 V to Supplementary Figs. 8–10). Through the incorporation of the
power up a Bluetooth module, after which the voltage on the proposed sock with external circuit design and other low-power
capacitor drops to 2 V as shown in Fig. 3h. This discharged power consumption sensors of different functionalities, a more compre-
from sock can activate the Bluetooth module to send the humidity hensive IoT platform could be feasible in near future.
and temperature data to the smartphone, which can be used for
monitoring the temperature on human body under various
exercise intensities. As known, the body temperature is suscep- Development of deep learning-enabled socks
tible to exercise intensities, hence it should be taken into account Considering the privacy concern of the face recognition,
when assessing the body temperature of a subject. The body fingerprint recognition, and acoustics recognition, gait as a
temperature under armpit of a subject was recorded under personal characteristic has many advantages for identification,
different activities states (normal walking, light exercise, moderate i.e., individual gait recognition has fewer environmental limitations
exercise, vigorous exercise, and rest after exercise) at five different for data collection, it is relatively easy and quick to collect data in
moments, as shown in Fig. 3i. It can be observed that the armpit daily life49,77. Differ from the widely used IMU sensors that require
temperature increased gradually when the subject was exercising high computing power and high computing power for precise

npj Flexible Electronics (2020) 29 Published in partnership with Nanjing Tech University
 Z. Zhang et al.
5

Fig. 3 The characterization of T-TENG socks and preliminary gait analysis. a Schematics of four phases of a typical contact cycle and
b corresponding signals of normal walking (right foot). c, d The real-time healthcare monitoring of mimetic walking pattern of Parkinson’s
disease patient and gait signals of a fall-down event. e, f The maximum output powers of a single sock on right foot under 1 Hz walking and
2 Hz running were tested by changing the external load resistances from 0.1 to 100 MΩ. g Charging curve of different capacitors (i.e., 1, 4.7, 10,
and 27 μF) were charged to 5 V. h Charging and discharging curve with the socks on foot, where each voltage drop represents a discharging
to the Bluetooth module. i Monitoring the temperature on armpit under various exercise intensities by Bluetooth module.

body movement monitoring50, our deep learning-enabled sock triboelectric output by its convolutional layers. In addition, the
with the minimal number of triboelectric sensors offers an one-dimensional (1D) CNN-based method for recognizing human
alternative solution for the detection and analysis of the individual activity has been proven to be very effective for the overall
gait. Previously, analysis strategies for triboelectric outputs are to dataset, in which the position of features in the segment is not
manually extract the shallow features from a single waveform, highly correlated; thus, it is suitable for the analysis of time
such as frequency, amplitude, interval of peaks, and holding time, sequences of sensor data82–85. In our work, the raw data from sock
which cannot achieve recognition of complicated features with which contains diversified information about a dynamic gait cycle,
subtle differences among them and is very susceptible to e.g., gait speed, contacting force between feet and shoes,
environmental variations, resulting in reduced recognition accu- triggering sequence of sensors and the individual usage manners
racy37,49,78. The technology fusion of the emerging AI with for a specific user, will be fed into the training 1D CNN model
wearable electronics promoted enormous advances to form a directly without the complicated data processing steps. Figure 4a,
whole intelligent system in technologies related to the process of b shows the gait pattern identification of five participants wearing
data acquisition, processing/analysis, and transmission79,80. the single-sensor sock with the 1D CNN method. A dataset was
Recently, deep learning technologies due to its high level of built from the five participants with proposed sock (45 kg female,
performance across many types of data are becoming a very 40 kg female, 60 kg male, 70 kg male, and 55 kg male), as they
favored subset of machine learning, which offers a promising and walked normally on a 5 m corridor within ~600 s. Next, the middle
feasible solution for triboelectric output analysis to achieve a 500 s data (16,000 datapoints) representing the stable gait of
higher accuracy of complex recognition. For the recent study on participants will be segmented by a sliding window without
the integration of deep learning techniques and triboelectric overlap. For each participant, the whole data were divided into the
devices, Zhao at al. has successfully used multiple deep belief training set (60%), testing set (20%), and validation set (20%).
network to realize the keystroke dynamic identification by a Because individual walking speed is an important consideration
triboelectric keyboard81. Apart from that method, convolutional for gait analysis, the length of the sliding window needs to be
neural network (CNN) becomes another promising technology of carefully determined to ensure that each sample can cover
deep learning for automatically extracting features from enough information. In view of the trade-off between sample

Published in partnership with Nanjing Tech University npj Flexible Electronics (2020) 29
 Z. Zhang et al.
6

Fig. 4 Gait identification of deep learning-enabled socks. a The process of data collection and the 4-s sliding window for the segment of
dataset. b Schematics of the process and parameters for constructing the 1D CNN structure. c Confusion map of the prediction with the gait
patterns of five participants. d Schematic of the triple-sensor socks and its three-channel output characteristics within 600 s. e, f Confusion
maps of the prediction with the triple-sensor socks of 5 and 13 participants.

number and window length, the accuracy rate reaches the highest 1D CNN model can assist the testing set of the intelligent sock to
when the sample length is 4 s (Supplementary Fig. 11). The sample achieve 96% accuracy of identity recognition, and the confusion
triboelectric outputs for the five participants are shown with the map of the classified result is shown in Fig. 4c. Although the sock
featured patterns, including some important features (Fig. 4b), i.e., with single sensor demonstrates a relatively good accuracy for
amplitude, frequency, interval of events in gait cycle, etc. The differentiating the gait patterns of five participants, it could be
proposed network architecture of the intelligent sock system largely reduced as the number of participants increases for it lacks
(Fig. 4b) consists of four convolutional layers, four max-pooling spatiotemporal detecting of foot pressure. Using a small piece of
layers, and one fully connected layer that outputs the predicted 3 cm × 3 cm of the textile sensor to test the different outputs of
identification of five participants. The accuracy of training set is various places around the foot is shown in Supplementary Fig. 12.
100% after 500 epochs training of the neural network and the The larger voltage means the better sensitivity to distinguish the
accuracy of validation set can be up to 98.4%. Then the trained gait pattern of different people; thus, we chose the three locations

npj Flexible Electronics (2020) 29 Published in partnership with Nanjing Tech University
 Z. Zhang et al.
7
with the largest pressure for further research. By attaching a multi- woods path, and gravel road until the following barrier. When they
pixel sensor array (i.e., three pieces) on the intelligent sock with are facing the rock and the stone cave, they should take the jump
the locations as shown in Fig. 4d, the intelligent sock should be and slide motion, respectively. This VR demonstration well
able to collect more information from foot motions for more indicates the potential of the intelligent sock for the realization
complicated gait analysis, and potentially raise the accuracy for a of the advanced multifunctional HMIs toward diversified applica-
larger group of participants. It should be noted that the sensing tions. In the future, the intelligent system based on the proposed
outputs show negligible interference from shoe wearing (Supple- socks could be further integrated with personal fitness monitoring
mentary Fig. 13), which demonstrates the good stability and system, which measures and analyses data on physical perfor-
adaptability of the intelligent sock in gait monitoring. In terms of mance during the VR gaming.
the triple-sensor intelligent sock, after applying the same method Since the deep learning-enabled sock is capable of recognizing
of data collection, the triboelectric signals from three channels can identification and monitoring activity, it has a great aspect in the
form a whole spectrum and enter the input layer of same CNN development of the intelligent wearable system, where the whole-
structure (Fig. 4b) to classify the different users. The final accuracy body movement and the physical signals can be continuously
of the triple-sensor intelligent sock can reach 100% (Fig. 4e), which monitored by the integration of the sock and other wearable
is a 4% improvement over the single-sensor sock. In general, the sensors. Such a wearable system would facilitate the rapid
larger the number of samples, the higher the classification advancement of the future digital human, where a realistic digital
accuracy of the trained model. To increase the accuracy of the replicate of a human could be created in the virtual space. For a
recognition system, here we present a method of 4 s sliding simple demonstration toward the future application of the digital
window with 2 s overlap. It will extend the limited training set human, a smart home system is developed to enable the
twice than the previous dataset. As shown in Supplementary Fig. automatic distinguishing of the family members from strangers,
14, with the increase of participants, models trained with as well as the real-time monitoring of family member’s indoor
overlapping window datasets have higher accuracy. As a result, activities in a no-camera environment, where the recognition
a dataset of 13 individual gait patterns was collected and a high results in the real space can be projected into a virtual space.
accuracy of 93.54% was also achieved with the aforementioned Figure 6a summarizes the structure of the whole system, including
sliding window method, as shown in Fig. 4f. The detailed the first stage of identity recognition, second stage of activity
parameters used for training a 13-class CNN are provided in monitoring, and real-time VR projection. As shown in Fig. 6b, the
Supplementary Table 1. Due to the limitation of the number of server side of the smart home can establish wireless communica-
datasets, the accuracy rate decreases slightly as the number of tion protocols with multiple users’ intelligent socks at the same
people increases. This problem can be solved by expanding the time. The middle segment of the collected gait signals from the
training set samples, e.g., collecting more gait data from multiple users will be sent to the first stage of smart home system
participants. separately (Fig. 6c) to recognize the user’s identity. It is known that
The rapid development of VR and augmented reality (AR) triboelectric output can be susceptible to environmental varia-
technologies paves a way for the diversified application in social tions, such as humidity, which will affect the recognition accuracy
media, personal engagement, surgical training, gaming, and so on. of the user’s identity as well. To eliminate the interferences from
The deep learning-enabled sock as a wearable HMI can provide the environment, here we tried to collect the datasets of the users
the gait information of the users to create an immersive from multiple days to form a comprehensive training set so as to
experience for VR/AR scenarios, and here we demonstrate a VR generalize the data. And it is observed that as the number of
fitness game with the proposed intelligent sock as the control training days increases, the accuracy is improved significantly,
interface (Supplementary Video 2). As depicted by the schematics showing the excellent stability of the deep learning-enabled sock
in Fig. 5a, the complete real-time control system includes the in gait pattern identification (Supplementary Note 3 and
triple-sensor sock, signal preprocessing circuit, MCU with wireless Supplementary Fig. 16). These results also indicate a reliable and
transmitter module, PC, and the virtual space in Unity software. general method to improve the reliability of the triboelectric
First, when the user moves, the triboelectric output signal from sensors, which is considered as one of the great challenges for
the sock will be generated and then enter the MCU after going them to be used for practical applications. After the identity
through the preprocessing circuit. The preprocessing circuit is recognition of first stage, the gait signals of family members will
used for filtering out the ambient noise and removing the enter second stage of real-time activity monitoring to output the
crosstalk between different channels, which is mainly composed final individual activity. Here, three common actions (running,
of circuit blocks with several functions, i.e., bias circuit, amplifier walking, and jumping) are extracted to build individual classifiers
circuit, and low-pass filter (further described in Supplementary for second stage. The corresponding signals from the triple-sensor
Note 2 and Supplementary Fig. 15). Then the MCU will detect the sock of each family member under walking states are shown in
equivalent analog signal of output voltage and send the data of Fig. 6c. Integrating the trained identification model and motion
the whole spectrum to the PC through wireless transmission. recognition model, the laptop screen displays the signal waveform
Based on the received spectrum of signals, the trained machine of gait and the corresponding prediction result. Simultaneously,
learning model will then send respective motion commands to the virtual space will generate the corresponding human model
Unity. In our demonstrated VR fitness game, there are five default who will perform the same actions as the predicted motion of the
movements available for detection to control the virtual character, gait in the second stage of the system. The proposed system
including the leaping, running, sliding, jumping, and walking. The provides a no-camera environment for the privacy and security
corresponding triboelectric outputs are shown in Fig. 5b, with aspect of the smart home, as shown in Supplementary Video 3.
respect to the five different motions in virtual space. After the This integrated system of different deep learning models can
training model of 1D CNN, the recognition accuracy of human also be used for future smart classroom setting. From Fig. 7a, b, a
activities reaches 96.67% with 100 training samples from each check-in system is demonstrated. All the signals obtained from the
motion as shown in the confusion map (Fig. 5c). The program of students who have entered the classroom will be identified and
unity can receive the command of the predicted motion and displayed in the system. As shown in Fig. 7c, when an unknown
convert it into the motion of the virtual character, as show in student moves from outside classroom to the classroom, the real-
Fig. 5d. For example, when the wood pile appears, the user should time gait signal will be collected and analyzed by the system to
take the leap motion to pass it, which will be simultaneously identify the student. And the students entering the classroom will
synchronized onto the virtual character’s movement in virtual be logged into the system via a trigger signal, as shown in Fig. 7d,
space. Next, users can choose run motion or walk motion in the i.e., stamping foot twice, and it will be synchronously displayed in

Published in partnership with Nanjing Tech University npj Flexible Electronics (2020) 29
 Z. Zhang et al.
8

Fig. 5 Human activities recognition of deep learning-enabled socks. a The process flow from sensory information collection to the real-time
prediction in VR fitness game. b 3D plots of the deep learning sock outputs responding to different motions (leap, run, slide, jump, and walk).
c The confusion map for deep learning outcome. d The motion of the virtual character corresponding to real motion in a proposed digital
human system.

the virtual space. After that, it can be detected whether the the detectable symptom. As a result, by leveraging the deep
student is sitting on the seat or keeping moving around in Fig. 7e. learning techniques, this intelligent sock can eventually turn into a
The system not only can record the number of people solution for supplying a safer and more intelligent environment,
participating in classes, but also can monitor the records of regarding the smart building in no need for supports from
students entering and leaving the classroom (Supplementary cameras and microphones.
Video 4). In addition, the user’s body temperature could be
detected with an embedded temperature sensor and uploaded to
the server together with the gait information, in which case the DISCUSSION
body temperature of all check-in students in the classroom will In summary, a deep learning-enabled sock based on the
also be monitored, providing a more comprehensive personal triboelectric textile sensors has been developed with great
status monitoring. Body temperature monitoring is not only potential of providing a low cost and low-power consumption
beneficial for the user himself, but also helpful in controlling the solution to the future digital human. The sensing range and
wide spread of the contagious diseases that may cause a fever, as sensitivity of the triboelectric textile sensor are highly improved

npj Flexible Electronics (2020) 29 Published in partnership with Nanjing Tech University
 Z. Zhang et al.
9

Fig. 6 Integrated demonstration of smart home application. a Overview of the two-stage recognition system. b Schematics of multiple
users with their gait signals collected by wireless communication. c The gait signal of two family members for gait identification in the first
stage of smart home system. d The gait signals of recorded family members responding to different motions (run, walk, and jump) in the
second stage of the smart home system.

with the aid of the mm-scale frustum structures patterned on the METHODS
contact surface, making it more suitable for gait sensing. By Fabrication of the triboelectric textile sensor
scavenging energy from normal walking, this sock can charge up a The triboelectric textile sensor contains two layers: a positive charge
27 µF capacitor in 3–4 min, which is sufficient to support a generation layer and a negative charge generation layer. Firstly, the
Bluetooth IoT sensor module to transmit temperature and conductive textile is cut into the desired size and shape, which is made of
humidity data. Furthermore, 96% accuracy of differentiating gait metalized fabric (polyester Cu) coated with an adhesive. To fabricate the
patterns among five participants has been achieved with the positive charge generation layer, a thin nitrile film is attached to one side
single-sensor sock by 1D CNN-based deep learning analytics. To of a conductive textile. Another conductive textile is coated with silicone
rubber film on the one side as well. The coating process was firstly
further boost the gait patterns sensing accuracy of the intelligent dispensing required amounts of parts A and B of the EcoFlexTM 00-30 into
sock, a triple-sensor sock can enhance the identification accuracy a mixing container (1A:1B by volume or weight), followed by mixing the
of 5 and 13 participants to 100% and 93.54%, respectively. More blend thoroughly for 3 min, and then the mixed solution was poured into a
importantly, the proposed sock is capable of detecting different 3D-printed mold followed by 20-min baking at 70 °C for curing. For the
human activities with 96.67% accuracy among five predefined textile sensor without surface structures, the uncured mixture was directly
motions for an identified user. A VR fitness game with it as a pasted onto the conductive textile to form a flat surface. Lastly, the silicone
control interface has been demonstrated. Combining the identi- rubber-coated textile was stitched to the nitrile-coated textile with two
fication and activity monitoring capabilities, we demonstrate the nonconductive textiles attached to the outer sides for encapsulation, the
detailed fabrication sees the Supplementary Information (Supplementary
deep learning-enabled sock as a functional part in both smart Figs. 1 and 2).
home and smart classroom applications by creating a virtual figure
with replicated motions of the user, showing its great prospect in
the development of the digital human in near future. In general, Experiment measurement and characterization
the proposed socks are equipped with diversified functionalities The signal outputs in the characterization of the T-TENG sensor were
measured by an oscilloscope (DSO-X3034A, Agilent), using a high
for the application of energy harvesting in IoT framework, and gait
impedance probe of 100 MΩ. Calibrations of output voltage against force
sensing for walking pattern recognition and human activity for triboelectric sensors were conducted by force gauge (Mecmesin,
monitoring. Further explorations would obtain more comprehen- MultiTest 2.5-i) with the speed of 600 mm min−1. Finite element method
sive information of users ranging from entertainment, social simulation of electric potential between two triboelectric layers (Eco-flex
network, healthcare, sports monitoring, and smart home, etc. and Nitrile) were numerically calculated using the commercial software

Published in partnership with Nanjing Tech University npj Flexible Electronics (2020) 29
 Z. Zhang et al.
10

Fig. 7 The implementation of deep learning-enabled socks for smart classroom application. a, b The photograph of real classroom, the
corresponding screenshot of virtual classroom in Unity and the corresponding demonstration of check-in system interface. c The gait signal
from the man outside the classroom for gait identification. d The check-in signal from the woman entering the classroom. e The static sitting
signals from the students sitting in the classroom.

COMSOL (Supplementary Fig. 5). The open-circuit voltages, transferred DATA AVAILABILITY
charges, and short-circuit currents were measured by an electrometer The data that support the findings of this study are available from the corresponding
(Model 6514, Keithley), and the signals were displayed and recorded by an author upon reasonable request.
oscilloscope (DSO-X3034A, Agilent). The Bluetooth module for transmitting
the data of temperature and humility sensors were made from commercial Received: 18 April 2020; Accepted: 30 September 2020;
BLE sensors (CYALKIT-E02), with an integrated temperature and humility
sensors (Si7020-A20). Analog voltage signals generated from intelligent
socks for real-time monitoring, HMIs, and digital human system are
collected and processed by the hardware circuit, consisting of a
conditioner PCB and an MCU (ESP32-PICO-KIT V4). After the measure- REFERENCES
ments, MATLAB®2019a was used for data analysis. The CNN models were
1. Bariya, M., Nyein, H. Y. Y. & Javey, A. Wearable sweat sensors. Nat. Electron. 1,
developed in Python with Keras and Tensorflow backend.
160–171 (2018).

npj Flexible Electronics (2020) 29 Published in partnership with Nanjing Tech University
 Z. Zhang et al.
11
2. Shi, Q., He, T. & Lee, C. More than energy harvesting – combining triboelectric 33. Sun, N. et al. All flexible electrospun papers based self-charging power system.
nanogenerator and flexible electronics technology for enabling novel micro-/ Nano Energy 38, 210–217 (2017).
nano-systems. Nano Energy 57, 851–871 (2019). 34. Jiang, D. et al. A wearable noncontact free-rotating hybrid nanogenerator for self-
3. Tai, L.-C. et al. Methylxanthine drug monitoring with wearable sweat sensors. Adv. powered electronics. InfoMat 2, 1191–1200 (2020).
Mater. 30, 1707442 (2018). 35. Wen, Z. et al. Self-powered textile for wearable electronics by hybridizing fiber-
4. Ouyang, H. et al. Self-powered pulse sensor for antidiastole of cardiovascular shaped nanogenerators, solar cells, and supercapacitors. Sci. Adv. 2, e1600097 (2016).
disease. Adv. Mater. 29, 1703456 (2017). 36. Dong, B. et al. Wearable triboelectric/aluminum nitride nano‐energy‐nano‐system
5. Liu, Y. et al. Epidermal mechano-acoustic sensing electronics for cardiovascular with self‐sustainable photonic modulation and continuous force sensing. Adv. Sci.
diagnostics and human-machine interfaces. Sci. Adv. 2, e1601185 (2016). 7, 1903636 (2020).
6. Trung, T. Q. et al. Freestanding, fiber-based, wearable temperature sensor with 37. He, T. et al. Beyond energy harvesting - multi-functional triboelectric nanosensors
tunable thermal index for healthcare monitoring. Adv. Healthc. Mater. 7, 1800074 on a textile. Nano Energy 57, 338–352 (2019).
(2018). 38. Chen, L. et al. Controlling surface charge generated by contact electrification:
7. Khan, Y., Ostfeld, A. E., Lochner, C. M., Pierre, A. & Arias, A. C. Monitoring of vital strategies and applications. Adv. Mater. 30, 1802405 (2018).
signs with flexible and wearable medical devices. Adv. Mater. 28, 4373–4395 39. Cai, G. et al. Extremely stretchable strain sensors based on conductive self-
(2016). healing dynamic cross-links hydrogels for human-motion detection. Adv. Sci. 4,
8. Zhang, Y.-Z. et al. MXenes stretch hydrogel sensor performance to new limits. Sci. 1600190 (2017).
Adv. 4, eaat0098 (2018). 40. Huang, Q. et al. Additive functionalization and embroidery for manufacturing
9. Li, L. et al. Moisture-driven power generation for multifunctional flexible sensing wearable and washable textile supercapacitors. Adv. Funct. Mater. 30, 1910541
systems. Nano Lett. 19, 5544–5552 (2019). (2020).
10. Lu, Q. et al. Bio-inspired flexible artificial synapses for pain perception and nerve 41. Zhao, Z. et al. Machine-washable and breathable pressure sensors based on
injuries. npj Flex. Electron. 4, 3 (2020). triboelectric nanogenerators enabled by textile technologies. Nano Energy 70,
11. Dehzangi, O., Taherisadr, M. & ChangalVala, R. IMU-based gait recognition using 104528 (2020).
convolutional neural networks and multi-sensor fusion. Sensors 17, 2735 (2017). 42. Zhao, Z. et al. Machine-washable textile triboelectric nanogenerators for effective
12. Liang, X. et al. Fusion of wearable and contactless sensors for intelligent gesture human respiratory monitoring through loom weaving of metallic yarns. Adv.
recognition. Adv. Intell. Syst. 1, 1900088 (2019). Mater. 28, 10267–10274 (2016).
13. Wen, F. et al. Battery-free short-range self-powered wireless sensor network (SS- 43. Hu, Y. & Zheng, Z. Progress in textile-based triboelectric nanogenerators for smart
WSN) using TENG based direct sensory transmission (TDST) mechanism. Nano fabrics. Nano Energy 56, 16–24 (2019).
Energy 67, 104266 (2020). 44. Dong, K., Peng, X. & Wang, Z. L. Fiber/fabric‐based piezoelectric and triboelectric
14. Tian, X. et al. Wireless body sensor networks based on metamaterial textiles. Nat. nanogenerators for flexible/stretchable and wearable electronics and artificial
Electron. 2, 243–251 (2019). intelligence. Adv. Mater. 32, 1902549 (2020).
15. Lee, S., Shi, Q. & Lee, C. From flexible electronics technology in the era of IoT and 45. He, T. et al. Self-sustainable wearable textile nano-energy nano-system (NENS) for
artificial intelligence toward future implanted body sensor networks. APL Mater. next-generation healthcare applications. Adv. Sci. 6, 1901437 (2019).
7, 031302 (2019). 46. Zhu, M. et al. Self-powered and self-functional cotton sock using piezoelectric
16. Li, S., Ni, Q., Sun, Y., Min, G. & Al-Rubaye, S. Energy-efficient resource allocation for and triboelectric hybrid mechanism for healthcare and sports monitoring. ACS
industrial cyber-physical IoT systems in 5G era. IEEE Trans. Ind. Inform. 14, Nano 13, 1940–1952 (2019).
2618–2628 (2018). 47. Choi, S. I. L., Moon, J., Park, H. C. & Choi, S. T. User identification from gait analysis
17. Liu, G. et al. Wireless electric energy transmission through various isolated solid using multi-modal sensors in smart insole. Sensors 19, 3785 (2019).
media based on triboelectric nanogenerator. Adv. Energy Mater. 8, 1703086 (2018). 48. Farah, J. D., Baddour, N. & Lemaire, E. D. Gait phase detection from thigh kine-
18. Hassani, F. A. & Lee, C. A triboelectric energy harvester using low-cost, flexible, matics using machine learning techniques. In Proc. 2017 IEEE International Sym-
and biocompatible ethylene vinyl acetate (EVA). J. Microelectromech. Syst. 24, posium on Medical Measurements and Applications (MeMeA), 263–268 (IEEE,
1338–1345 (2015). Rochester, MN, 2017).
19. Dong, B. et al. Wearable triboelectric–human–machine interface (THMI) using 49. Han, Y. et al. Self-powered gait pattern-based identity recognition by a soft and
robust nanophotonic readout. ACS Nano 14, 8915–8930 (2020). stretchable triboelectric band. Nano Energy 56, 516–523 (2019).
20. Khan, A. A., Mahmud, A. & Ban, D. Evolution from single to hybrid nanogenerator: 50. Lee, S.-S., Choi, S. T. & Choi, S.-I. Classification of gait type based on deep learning
a contemporary review on multimode energy harvesting for self-powered elec- using various sensors with smart insole. Sensors 19, 1757 (2019).
tronics. IEEE Trans. Nanotechnol. 18, 21–36 (2019). 51. Lin, Z. et al. A triboelectric nanogenerator-based smart insole for multifunctional
21. Liu, Y. et al. Quantifying contact status and the air-breakdown model of charge- gait monitoring. Adv. Mater. Technol. 4, 1800360 (2019).
excitation triboelectric nanogenerators to maximize charge density. Nat. Com- 52. Sorrentino, I. et al. A novel sensorised insole for sensing feet pressure distribu-
mun. 11, 1599 (2020). tions. Sensors 20, 747 (2020).
22. Smirnov, J. R. C. et al. Flexible distributed feedback lasers based on nanoim- 53. Song, K., Zhao, R., Wang, Z. L. & Yang, Y. Conjuncted pyro-piezoelectric effect for
printed cellulose diacetate with efficient multiple wavelength lasing. npj Flex. self-powered simultaneous temperature and pressure sensing. Adv. Mater. 31,
Electron. 3, 17 (2019). 1902831 (2019).
23. Lund, A. et al. Energy harvesting textiles for a rainy day: woven piezoelectrics 54. Guo, H. et al. A highly sensitive, self-powered triboelectric auditory sensor for
based on melt-spun PVDF microfibres with a conducting core. npj Flex. Electron. social robotics and hearing aids. Sci. Robot. 3, eaat2516 (2018).
2, 9 (2018). 55. Yang, Y. et al. Liquid-metal-based super-stretchable and structure-designable tri-
24. Torres Alonso, E. et al. Graphene electronic fibres with touch-sensing and light- boelectric nanogenerator for wearable electronics. ACS Nano 12, 2027–2034 (2018).
emitting functionalities for smart textiles. npj Flex. Electron. 2, 25 (2018). 56. He, Q. et al. Triboelectric vibration sensor for a human-machine interface built on
25. Arab Hassani, F. et al. Smart materials for smart healthcare–moving from sensors ubiquitous surfaces. Nano Energy 59, 689–696 (2019).
and actuators to self-sustained nanoenergy nanosystems. Smart Mater. Med. 1, 57. Zhu, M. et al. Haptic-feedback smart glove as a creative human-machine interface
92–124 (2020). (HMI) for virtual/augmented reality applications. Sci. Adv. 6, eaaz8693 (2020).
26. Lee, J. et al. Conductive fiber-based ultrasensitive textile pressure sensor for 58. Wang, J., Chen, Y., Hao, S., Peng, X. & Hu, L. Deep learning for sensor-based
wearable electronics. Adv. Mater. 27, 2433–2439 (2015). activity recognition: a survey. Pattern Recognit. Lett. 119, 3–11 (2019).
27. Liu, M. et al. Large-area all-textile pressure sensors for monitoring human motion 59. LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
and physiological signals. Adv. Mater. 29, 1703700 (2017). 60. Zhu, M., He, T. & Lee, C. Technologies toward next generation human machine
28. Cao, X., Jie, Y., Wang, N. & Wang, Z. L. Triboelectric nanogenerators driven self- interfaces: From machine learning enhanced tactile sensing to neuromorphic
powered electrochemical processes for energy and environmental science. Adv. sensory systems. Appl. Phys. Rev. 7, 031305 (2020).
Energy Mater. 6, 1600665 (2016). 61. Sundaram, S. et al. Learning the signatures of the human grasp using a scalable
29. Cheng, P. et al. Self-powered active spherical triboelectric sensor for fluid velocity tactile glove. Nature 569, 698–702 (2019).
detection. IEEE Trans. Nanotechnol. 19, 230–235 (2020). 62. Qiu, C., Wu, F., Shi, Q., Lee, C. & Yuce, M. R. Sensors and control interface methods
30. Zhou, C. et al. Flexible self-charging power units for portable electronics based on based on triboelectric nanogenerator in IoT applications. IEEE Access 7, 92745 (2019).
folded carbon paper. Nano Res. 11, 4313–4322 (2018). 63. Lee, G. et al. Parallel signal processing of a wireless pressure‐sensing platform
31. Shen, Q. et al. Self-powered vehicle emission testing system based on coupling of combined with machine‐learning‐based cognition, inspired by the human
triboelectric and chemoresistive effects. Adv. Funct. Mater. 28, 1703420 (2018). somatosensory system. Adv. Mater. 32, 1906269 (2019).
32. Liao, J. et al. Nestable arched triboelectric nanogenerator for large deflection 64. Shi, Q. et al. Progress in wearable electronics/photonics-moving toward the era of
biomechanical sensing and energy harvesting. Nano Energy 69, 104417 (2020). artificial intelligence and internet of things. InfoMat 2, 1131–1162 (2020).

Published in partnership with Nanjing Tech University npj Flexible Electronics (2020) 29
 Z. Zhang et al.
12
65. Mezghani, E., Exposito, E., Drira, K., Da Silveira, M. & Pruski, C. A semantic big data ACKNOWLEDGEMENTS
platform for integrating heterogeneous wearable data in healthcare. J. Med. Syst. This research is supported by the National Research Foundation Singapore under its
39, 185 (2015). AI Singapore Programme (Award Number: AISG-GC-2019-002) and National Key
66. Madni, A., Madni, C. & Lucero, S. Leveraging digital twin technology in model- Research and Development Program of China (Grant No. 2019YFB2004800 and
based systems engineering. Systems 7, 7 (2019). Project No. R-2020-S-002). Any opinions, findings and conclusions or recommenda-
67. Tao, F. et al. Digital twin-driven product design, manufacturing and service with tions expressed in this material are those of the author(s) and do not reflect the views
big data. Int. J. Adv. Manuf. Technol. 94, 3563–3576 (2018). of National Research Foundation, Singapore.
68. Yu, X. et al. Skin-integrated wireless haptic interfaces for virtual and augmented
reality. Nature 575, 473–479 (2019).
69. Wang, J., He, T. & Lee, C. Development of neural interfaces and energy harvesters AUTHOR CONTRIBUTIONS
towards self-powered implantable systems for healthcare monitoring and reha-
Z.Z., T.H., and C.L. conceived the idea and planned the experiments. Z.Z., M.Z., Z.S.,
bilitation purposes. Nano Energy 65, 104039 (2019).
and C.L. designed the hardware, and performed the experiments. Z.Z., Z.S., and C.L.
70. Jiang, D. et al. A 25-year bibliometric study of implantable energy harvesters and
wrote the control programs and algorithms for demonstration. Z.Z., M.Z., and C.L.
self-powered implantable medical electronics researches. Mater. Today Energy 16,
contributed to the data analysis. Z.Z., T.H., M.Z., and C.L. wrote the paper. All authors
100386 (2020).
discussed the results and commented on the manuscript.
71. Cheng, J., Amft, O., Bahle, G. & Lukowicz, P. Designing sensitive wearable capa-
citive sensors for activity recognition. IEEE Sens. J. 13, 3935–3947 (2013).
72. Meyer, J., Lukowicz, P. & Troster, G. Textile pressure sensor for muscle activity and
motion detection. In 2006 10th IEEE International Symposium on Wearable Com- COMPETING INTERESTS
puters, 69–72 (IEEE, Montreux, Switzerland, 2006). The authors declare no competing interests.
73. Niu, S. et al. A wireless body area sensor network based on stretchable passive
tags. Nat. Electron. 2, 361–368 (2019).
74. Son, D. et al. An integrated self-healable electronic skin system fabricated via ADDITIONAL INFORMATION
dynamic reconstruction of a nanostructured conducting network. Nat. Nano- Supplementary information is available for this paper at https://doi.org/10.1038/
technol. 13, 1057–1065 (2018). s41528-020-00092-7.
75. Mackanic, D. G., Kao, M. & Bao, Z. Enabling deformable and stretchable batteries.
Adv. Energy Mater. 2001424, 2001424 (2020). Correspondence and requests for materials should be addressed to C.L.
76. Linghu, C., Zhang, S., Wang, C. & Song, J. Transfer printing techniques for flexible
and stretchable inorganic electronics. npj Flex. Electron. 2, 26 (2018). Reprints and permission information is available at http://www.nature.com/
77. Aqueveque, P., Germany, E., Osorio, R. & Pastene, F. Gait segmentation method reprints
using a plantar pressure measurement system with custom-made capacitive
sensors. Sensors 20, 656 (2020). Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims
78. Zhang, R. et al. Sensing body motions based on charges generated on the body. in published maps and institutional affiliations.
Nano Energy 63, 103842 (2019).
79. Wang, C., Dong, L., Peng, D. & Pan, C. Tactile sensors for advanced intelligent
systems. Adv. Intell. Syst. 1, 1900090 (2019).
80. Li, J. et al. Skin‐inspired electronics and its applications in advanced intelligent
systems. Adv. Intell. Syst. 1, 1970060 (2019). Open Access This article is licensed under a Creative Commons
81. Zhao, G. et al. Keystroke dynamics identification based on triboelectric nano- Attribution 4.0 International License, which permits use, sharing,
generator for intelligent keyboard using deep learning method. Adv. Mater. adaptation, distribution and reproduction in any medium or format, as long as you give
Technol. 4, 1800167 (2019). appropriate credit to the original author(s) and the source, provide a link to the Creative
82. Shi, Q. et al. Deep learning enabled smart mats as a scalable floor monitoring Commons license, and indicate if changes were made. The images or other third party
system. Nat. Commun. 11, 4609 (2020). material in this article are included in the article’s Creative Commons license, unless
83. Jin, T. et al. Triboelectric nanogenerator sensors for soft robotics aiming at digital indicated otherwise in a credit line to the material. If material is not included in the
twin applications. Nat. Commun. (2020, in press) https://doi.org/10.1038/s41467- article’s Creative Commons license and your intended use is not permitted by statutory
020-19059-3. regulation or exceeds the permitted use, you will need to obtain permission directly
84. Cho, H. & Yoon, S. M. Divide and conquer-based 1D CNN human activity from the copyright holder. To view a copy of this license, visit http://creativecommons.
recognition using test data sharpening. Sensors 18, 1055 (2018). org/licenses/by/4.0/.
85. Baloglu, U. B., Talo, M., Yildirim, O., Tan, R. S. & Acharya, U. R. Classification of
myocardial infarction with multi-lead ECG signals and deep CNN. Pattern
Recognit. Lett. 122, 23–30 (2019). © The Author(s) 2020

npj Flexible Electronics (2020) 29 Published in partnership with Nanjing Tech University