Yoo 2022 Ieeeaccess DTS-SNN Spiking Neural Networks With Dynamic Time-Surfaces
Yoo 2022 Ieeeaccess DTS-SNN Spiking Neural Networks With Dynamic Time-Surfaces
1 ABSTRACT Convolution helps spiking neural networks (SNNs) capture the spatio-temporal structures
2 of neuromorphic (event) data as evident in the convolution-based SNNs (C-SNNs) with the state-of-the-
3 art classification-accuracies on various datasets. However, the efficacy aside, the efficiency of C-SNN is
4 questionable. In this regard, we propose SNNs with novel trainable dynamic time-surfaces (DTS-SNNs)
5 as efficient alternatives to convolution. The novel dynamic time-surface proposed in this work features its
6 high responsiveness to moving objects given the use of the zero-sum temporal kernel that is motivated
7 by the simple cells’ receptive fields in the early stage visual pathway. We evaluated the performance and
8 computational complexity of our DTS-SNNs on three real-world event-based datasets (DVS128 Gesture,
9 Spiking Heidelberg dataset, N-Cars). The results highlight high classification accuracies and significant
10 improvements in computational efficiency, e.g., merely 1.51% behind of the state-of-the-art result on
11 DVS128 Gesture but a ×18 improvement in efficiency. The code is available online (https://github.com/
12 dooseokjeong/DTS-SNN).
13 INDEX TERMS Lightweight spiking neural network, spiking neural network, dynamic time-surfaces, event-
14 based data.
16 Convolution-based methods are pervasive in a variety of chronous spikes, unlike layer-wise sequential operations in 33
17 deep learning application domains given their high efficacy DNNs which impose forward locking constraints [5], [6]. 34
18 across different domains when implemented in convolutional To leverage this advantage, it is required to implement SNNs 35
19 neural networks (CNNs). The same holds for spiking neural in dedicated hardware, which is referred to as neuromorphic 36
20 networks (SNNs) in that convolution-based SNNs (C-SNNs) hardware [7], [8], [9], [10], [11]. Generally, a neuromorphic 37
21 hold the state-of-the-art classification accuracies on a variety processor consists of multiple cores supporting asynchronous 38
22 of datasets [1], [2], [3]. Convolution is an operation-intensive event-based operations across them. The consequent power 39
23 method that involves a large number of multiply-accumulate efficiency is the key feature of neuromorphic hardware. 40
24 operations over 3D feature maps. Therefore, convolution Time-surface (TS) analyses are effective methods to pro- 41
25 generally results in high computational complexity and high cess asynchronous events (spikes) for various tasks [12], [13], 42
26 power consumption, which is a daunting challenge, partic- [14]. A TS for a given event is a 2D map of the event 43
27 ularly for C-SNNs, because power efficiency is supposed timestamps prior to the event in the spatial vicinity of the 44
28 to be one of the key advantages of SNNs over deep neural event. Therefore, the TS can capture the spatio-temporal local 45
30 SNNs are time-dependent hypotheses consisting of spik- the previous TSs are not tailored to SNNs and hardly support 47
31 ing units and unidirectional synapses [4]. One of the end-to-end learning. 48
The associate editor coordinating the review of this manuscript and volution, to extract the features of event data in a highly 50
approving it for publication was Fu-Kwun Wang . operation-efficient manner to leverage the key advantage, 51
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 10, 2022 102659
D. Yoo, D. S. Jeong: DTS-SNN: Spiking Neural Networks With Dynamic Time-Surfaces
they partitioned the input field into grid-cells. For each grid- 94
cell, the TSs for several recent events in the grid-cell are 95
response is denoted by r . multiple stages. The feature histogram from the last stage 104
52 i.e., power efficiency, of SNNs. To the best of our knowledge, category. HATS uses grid-cell-wise averaged TSs as a dic- 107
53 this work is the first attempt to integrate TSs into SNNs to tionary. Similar to HOTS, the feature map is built based on 108
54 process event-based data efficiently. Moreover, we signifi- matching frequency but uses a support vector machine as a 109
55 cantly modified the conventional TSs to better capture the classifier. Notably, both HOTS and HATS use time-invariant 110
56 dynamic features of event-data by using a zero-sum temporal (static) TSs as inputs to their classifiers. 111
57 kernel motivated by the temporal kernels of simple cells in the With the development of training algorithms for SNNs, 112
58 early stage visual pathways [15]. Additionally, our TSs are there have been attempts to process event data by exploiting 113
59 dynamic insomuch as they are calculated for every timestep the spatio-temporal processing ability of SNNs [1], [2], [18], 114
60 to encode event dynamics unlike the previous TSs, which are [19], [20]. Yet, to achieve high classification accuracies, most 115
61 referred to as dynamic time-surfaces (DTSs). The primary of them used large C-SNNs with multiple hidden layers, 116
62 contributions of our work include the following: which cause significant computational complexity. 117
63 • We propose DTS-SNN to replace C-SNN, which is This lets us revisit the initial motivation of SNNs, energy- 118
64 remarkably lightweight but exhibits high classification efficiency, and consequently rethink of efficient methods to 119
65 accuracy on event-based data. extract the spatio-temporal features of event data using TSs 120
66 • We propose trainable DTSs that are susceptible to mov- as alternatives to convolution. To this end, the prerequisites 121
67 ing objects and fully support end-to-end learning. include (i) the modification of the conventional time-invariant 122
68 • We evaluate the classification accuracy and computa- TSs to time-dependent (dynamic) forms with a noise-robust 123
69 tional efficiency of DTS-SNNs on various event-based temporal kernel and (ii) development of a DTS builder sup- 124
70 datasets, including DVS128 Gesture [16], Spiking Hei- porting end-to-end batch learning. 125
73 In the early stage of visual processing, simple cells (lin- DTS-SNNs consists of a DTS builder and SNN classifier. 128
74 ear neurons) exhibit linear responses to visual inputs in The builder constructs DTSs for the events at every timestep, 129
75 their receptive fields of particular spatial and temporal struc- which are subsequently fed into the SNN as inputs. To val- 130
76 tures [15]. The temporal structure (kernel) of receptive fields idate the feature extraction ability of the proposed DTS 131
77 features alternating positive and negative contributions of builder and the importance of well-defined features for SNNs, 132
78 input to the simple cell’s response in time such that the input we used a simple dense SNN with a single hidden layer, which 133
79 at a particular point (t0 ) causes a positive contribution within a was trained using a surrogate gradient-based backpropagation 134
80 time window (tc in width, i.e., if t − t0 ≤ tc ), which turns into algorithm [21]. This section elucidates the DTS in compari- 135
81 a negative contribution when the time exceeds the window. son with the previous TSs and a method to build DTSs in 136
82 A schematic of the temporal kernel is illustrated in Figure 1. parallel for the samples in a single batch. 137
85 A comparison between simple cell’s responses to static and TEMPORAL KERNELS 139
86 moving objects is depicted in Figure 1. For an event stream from an event camera, the ith event (ei ) 140
87 As the early research of TS analysis, HOTS considers only is encoded as ei = (pi , ti , Xi ), where pi , ti , and Xi denote 141
88 the last timestamps of the pixels in a given TS. Although its polarity pi ∈ {−1, 1}, timestamp, and location on a 142
89 HOTS successfully introduced the concept of TS to process 2D pixel array Xi = (xi , yi ), respectively. The DTS for the 143
90 event data, the TS was prone to noises that are events irrele- ith event Tei only considers the previous or simultaneous 144
91 vant to the objects under consideration [12]. events ej (j ≤ i) of the same polarity (pj = pi ), which are 145
149 1yij = yj − yi , Rx = 2rx + 1, and Ry = 2ry + 1. An example rate. This can clearly be differentiated from the timestamp 186
150 of building DTSs for three consecutive events is illustrated encoding using the single-exponential kernel kt as compared 187
152 The DTS at timestep ti is encoded with the novel zero-sum Similar to HATS, the input field is partitioned into grid- 189
153 temporal kernel ktzs as cells, and a single grid-cell-wise representative DTS is built 190
155 The event stream ρ for each location Xj is described by weighted sum of the DTSs of simultaneous events Tei . 193
X X
156 ρ t; Xj = δ (t − tk ) , (2) T c (t) = ai Tei , (5) 194
tk ∈tk ei ∈et,c
158 previous timestamps tk = tk |k ≤ j, Xk = Xj . The zero- time-surface Tei is denoted by ai which is a trainable param- 196
159 sum temporal kernel ktzs is given by eter. This set of weights is shared among all grid-cells. Note 197
164 Eq. (3) is termed zero-sum temporal kernel because the con- The key to training SNNs using DTSs on a given dataset 202
165 volution with this kernel over an event stream constant firing is the parallel calculations of DTSs for all samples in a 203
166 rate yields zero due to the balance between the positive and batch. Additionally, the compatibility of parallel calculations 204
167 negative sub-kernels in Eq. (3). with readily available deep learning frameworks enhances 205
168 Lemma 1: Consider the convolution of a train of Poisson efficiency. To this end, we propose pixel-wise timestamp- 206
169 spikes ρ at a constant firing rate r using the zero-sum tempo- encoding banks E (t) that are updated once for every timestep. 207
170 ral kernel ktzs , y (t) = (ktzs ∗ ρ) (t). The result converges to The timestamp encodings in the bank can readily be retrieved 208
171 zero as time t increases, i.e., y (∞) = 0. when events at particular pixels occur. This bank is subse- 209
172 The derivation of Lemma 1 is shown in Appendix A. Accord- quently unfolded to endow each pixel with an element time- 210
173 ing to Lemma 1, the kernel yields high responsiveness to surface. At a given timestep, the element time-surfaces for 211
174 spike trains at time-dependent firing rates by filtering out the the simultaneous events are retrieved and summed with their 212
175 spikes at constant firing rates. Consequently, the zero-sum weights to calculate the DTS for a given grid-cell. 213
176 temporal kernel endows the timestamp encoding with high We consider periodically distributed grid-cells over 214
177 responsiveness to moving objects compared with the single- a H × W input field for a given polarity; each grid-cell is 215
178 exponential kernel. hc × wc in size so that there exist H /hc × W /wc grid-cells on 216
FIGURE 4. Procedure for building DTSs for a given sample at a given timestep t . Rx and Ry define the size of a DTS such that
Rx = 2rx + 1 and Ry = 2ry + 1.
217 the input field. As such, the spatial domain of each DTS Dei is reshaped into a P × Rx Ry × H × W map. This reshaping 247
219 The procedure is detailed in the following subsections. The the DTSs using 3D convolution. This process is depicted in 249
222 A timestamp-encoding bank E (t) is a bank of timestamp- The reshaped preliminary time-surface map T (t) is read out 252
223 encodings for all pixels so that its dimension is identical to to acquire the DTSs of the events occurring at timestep t. 253
224 the input field. E (t) ∈ RP×H ×W for a P × H × W input field. To this end, we reuse the event-map 1A (p, X) in Eq. (7). 254
225 Each element Epxy (t) is calculated by convolving an event- This event-map is expanded by repeating the map along 255
226 stream (polarity p) at a location (x, y), ρ (t; p, x, y) with the the flattened time-surface axis. This expanded event-map is 256
227 zero-sum temporal kernel ktzs , element-wise multiplied by the map T (t), resulting in the 257
229 For efficient computation, we transform this convolution into by ‘‘Multiplied by expanded event-map’’ in Figure 4. The 260
230 a recursive form as follows: elements in each flattened Tei (t) are L2-normalized. 261
231 E (t + 1) = E1 (t + 1) − E2 (t + 1) ,
5) GRID-CELL-WISE WEIGHTED SUM OF TIME-SURFACES 262
232 E1 (t + 1) = E1 (t) e−1/τ1 + 1A (p, X) , The input field is a H /hc × W /hw grid, and each grid-cell 263
233 E2 (t + 1) = E2 (t) e−1/τ2 + τ1 /τ2 1A (p, X) . (7) is hc × wc in size. For a given grid-cell, the DTS T c (t) 264
242 Tpxy (t) ← Ep,(x−rx ):(x+rx ),(y−ry ):(y+ry ) (t) . (8) the convolution methods in the deep learning frameworks. 275
(
(l)
(l) 1 if ui ≥ ϑ,
299 Sϑ ui = (11) in Python using the Pytorch’s Autograd framework [22]. 327
0 otherwise.
We trained the networks using Adam [23] without weight 328
300 When the potential in Eq. (10) crosses a threshold for decay and learning rate scheduling. 329
309 where a is a positive constant. Table 1 shows the performance and efficiency of 339
310 IV. EXPERIMENTS ous methods using CNN-based SNNs. It highlights (i) high 341
311 We evaluated the performance of DTS-SNNs on three real- classification accuracy (1.51% lower than the state-of-the- 342
312 world datasets, DVS128 Gesture, SHD datasets, N-Cars. For art result though) and (ii) extremely high computational effi- 343
313 all datasets, we reduced the input event sampling rate to ciency (×18 of that of the state-of-the-art result). The high 344
314 reduce the computational complexity, which is equivalent to computational efficiency arises from the use of a small FCN 345
315 the reciprocal timestep 1t −1 . The hyper-parameters used for instead of a CNN and the high efficiency of the DTS builder. 346
316 each dataset are listed in Appendix E, which were found using The evolution of test accuracy with training epoch is plotted 347
317 manual searches. We used the raw event datasets without any in Figure 5. 348
319 To identify the effect of the zero-sum temporal kernel ment by using the zero-sum temporal kernel ktzs instead 350
320 ktzs on performance, we compared the classification accu- of the single-exponential kernel kt , which indicates the 351
321 racy for the zero-sum temporal kernel ktzs with that for the higher temporal responsiveness of the zero-sum temporal 352
322 single-exponential kernel kt . We evaluated the computational kernel than the conventional single-exponent temporal ker- 353
323 efficiency of DTS-SNN in terms of the number of OPs per nel. We visualize the DTSs T c of the two temporal ker- 354
324 timestep. Note that the number of OPs includes #OPDTS in nels at five timesteps (200 – 240) in Figure 6. A detailed 355
325 Eq. (9). All experiments were conducted on a GPU work- comparison between the two time-surfaces is addressed 356
326 station (GPU: RTX 2080 TI). DTS-SNNs were implemented in Appendix B. 357
FIGURE 6. DTSs for (upper panel) the zero-sum temporal kernel and (lower panel) the single-exponential kernel. Each grid-cell in an
8 × 8 grid is a 7 × 7 DTS. (Right hand clockwise sample from DVS128 Gesture).
358 B. SHD the input data dimension. Each sample varies in length 363
359 SHD is an audio classification dataset. It consists of 10,420 (0.24 – 1.17 s). We set the input sampling time 1t to 1 ms. The 364
360 samples of spoken digits (0 – 9) in English and German, other hyper-parameters are listed in Table 4. We considered 365
361 and thus labeled as 20 classes. The recorded samples were the original 700-long 1D sample at a given timestep as a 2D 366
362 analyzed using 700 channels as bases which determine sample (H = 1, W = 700) and mapped it onto a 1 × 35 grid, 367
FIGURE 7. Distribution of timestamp encoding values for the zero-sum temporal kernel ktzs and single-exponential kernel kt (Right hand
clockwise sample from DVS128 Gesture).
368 i.e., hc = 1 and wc = 20. The size of each time-surface was TABLE 2. Classification (best) accuracy for different time-surface sizes
(R × R for DVS128 Gesture and N-Cars, and R for SHD) and grid-cell sizes
369 set to 1 × 3, i.e., Rx = 1 and Ry = 3. The 105-long flattened (C × C for DVS128 Gesture and N-Cars, and C for SHD).
370 DTS was fed into a 105-128-20 SNN classifier.
371 Table 1 shows the performance and efficiency on SHD
372 compared with previous works. Note that all models
373 except [26] on SHD in Table 1 have 128 neurons in each
374 hidden layer. The use of the zero-sum temporal kernel real-
375 ized a significant improvement in classification accuracy
376 (by 15.96%) compared with the single-exponential kernel.
377 The learning kinetics for both cases is plotted in Figure 5.
378 C. N-CARS
379 N-Cars is an event-based dataset that was directly recorded
380 using an event camera for car detection task. This dataset
381 aims to binary classification task (car or background) with enhances the responsiveness to time-varying events. To show 399
382 event data of static objects (rather than dynamic objects as this, we address the distribution of timestamp encoding val- 400
384 and we set the input sampling time 1t to 1 ms so that the given timesteps. We plot the distribution for the zero-sum 402
385 number of timesteps was 100. temporal kernel ktzs and single-exponential kernel kt at given 403
386 We mapped each sample at a given timestep (H = 100, timesteps in Figure 7. We used a sample from DVS128 Ges- 404
387 W = 120) onto a 10 × 12 grid, i.e., hc = wc = 10. We used ture. The comparison evidently indicates the larger dispersion 405
388 the same size of time-surfaces as for DVS128 Gesture and of encoding values for the single-exponential kernel, and 406
389 SHD, i.e., Rx = Ry = 5. The 3,000-long flattened DTS thus the larger standard deviation than the zero-sum temporal 407
390 was fed into a 3000-400-2 SNN classifier. The results are kernel. The larger encoding values for the single-exponential 408
391 shown in Table 1 and compared with several state-of-the- kernel are likely attributed to persistent events. The zero-sum 409
392 art methods. The results indicate an accuracy improvement temporal kernel filters out such large encoding values and 410
393 by 0.81% by using the zero-sum temporal kernel instead of consequently allows the SNN classifier to pay attention to 411
394 the single-exponential kernel. The evolution of test accuracy time-varying events. 412
395 with training epoch is plotted in Figure 5. The dimensions of each time-surface and grid-cell are 413
396 V. DISCUSSION C × C) for 2D data and (R and C) for 1D data. The depen- 415
397 The zero-sum temporal kernel ktzs avoids large timestamp dency of classification accuracy on these hyper-parameters 416
398 encoding values caused by persistent events so that it is shown in Table 2. The larger the value R, the further 417
418 events are considered to build time-surfaces, capturing the probabilities are given by 467
424 take into account coherent events by filtering out incoherent ering the nontrivial cases only. 470
k=1
427 VI. CONCLUSION
428 We proposed DTS-SNN that merges the time-surface analysis Because Ps (k) = r (k) 1t, we have 472
τ1
where k is a single exponential kernel k = e−t/τ . The y (t) = y(1) (t) − y(2) (t) , 487
τ
449
2
450 expected value y is approximated to the convolution of the
(i)
451 firing rate r using the same kernel, y (t) = ktzs ∗ r (t) (t) , i ∈ {1, 2} .
(i)
(15) 488
452 y (t) = (k ∗ r (t)) (t) . We consider a Poisson-spike train whose firing rate r is given 489
453 Proof: The probability of a particular pattern of proba- by a boxcar function with constant nonzero firing rate r0 in 490
454 bilistic spikes in a period is calculated using the probability the range t0 < t < t1 . 491
455 of spiking Ps (t) and not spiking 1−Ps (t). Consider a pattern
456 of spikes at timesteps Ts and no spikes at timesteps Tns . The r (t) = r0 (H (t − t0 ) − H (t − t1 )) , 492
458 P= Ps (t) (1 − Ps (t)) . (13) we have the result of the convolution in Eq. (15) as follows. 494
t∈Ts t∈Tns
y (t) = 0 if 0 ≤ t < t0 , 495
466 the entire period T , Thus, there are T nontrivial cases whose 500
523 The pseudocode for constructing dynamic time-surfaces in The hyper-parmeters used for evaluation are shown in 526
528 REFERENCES [22] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, 602
529 [1] H. Zheng, Y. Wu, L. Deng, Y. Hu, and G. Li, ‘‘Going deeper with directly- Z. Lin, N. Gimelshein, L. Antiga, and A. Desmaison, ‘‘Pytorch: An imper- 603
530 trained larger spiking neural networks,’’ in Proc. AAAI Conf. Artif. Intell., ative style, high-performance deep learning library,’’ in Proc. Adv. Neural 604
531 2021, pp. 11062–11070. Inf. Process. Syst., 2019, pp. 8026–8037. 605
532 [2] A. Kugele, T. Pfeil, M. Pfeiffer, and E. Chicca, ‘‘Efficient processing [23] D. P. Kingma and J. Ba, ‘‘Adam: A method for stochastic optimization,’’ 606
533 of spatio-temporal data streams with spiking neural networks,’’ Frontiers 2014, arXiv:1412.6980. 607
534 Neurosci., vol. 14, p. 439, May 2020. [24] S. B. Shrestha and G. Orchard, ‘‘Slayer: Spike layer error reassignment in 608
535 [3] B. Yin, F. Corradi, and S. M. Bohté, ‘‘Accurate and efficient time-domain time,’’ in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 1412–1421. 609
536 classification with adaptive spiking recurrent neural networks,’’ Nature [25] W. Fang, Z. Yu, Y. Chen, T. Masquelier, T. Huang, and Y. Tian, ‘‘Incorpo- 610
537 Mach. Intell., vol. 3, no. 10, pp. 905–913, Oct. 2021. rating learnable membrane time constant to enhance learning of spiking 611
538 [4] D. S. Jeong, ‘‘Tutorial: Neuromorphic spiking neural networks for tempo- neural networks,’’ in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 612
539 ral learning,’’ J. Appl. Phys., vol. 124, no. 15, Oct. 2018, Art. no. 152002. Oct. 2021, pp. 2661–2671. 613
540 [5] M. Pfeiffer and T. Pfeil, ‘‘Deep learning with spiking neurons: Opportuni- [26] F. Zenke and T. P. Vogels, ‘‘The remarkable robustness of surrogate gra- 614
541 ties and challenges,’’ Frontiers Neurosci., vol. 12, p. 774, Oct. 2018. dient learning for instilling complex function in spiking neural networks,’’ 615
542 [6] M. Jaderberg, W. M. Czarnecki, S. Osindero, O. Vinyals, A. Graves, Neural Comput., vol. 33, no. 4, pp. 899–925, 2021. 616
543 D. Silver, and K. Kavukcuoglu, ‘‘Decoupled neural interfaces using syn- [27] N. Perez-Nieves, V. C. H. Leung, P. L. Dragotti, and D. F. M. Goodman, 617
544 thetic gradients,’’ in Proc. Int. Conf. Mach. Learn., 2017, p. 1627–1635. ‘‘Neural heterogeneity promotes robust learning,’’ Nature Commun., 618
545 [7] P. A. Merolla, J. V. Arthur, R. Alvarez-Icaza, A. S. Cassidy, J. Sawada, vol. 12, no. 1, pp. 1–9, Dec. 2021. 619
546 F. Akopyan, B. L. Jackson, N. Imam, C. Guo, Y. Nakamura, B. Brezzo, [28] A. Viale, A. Marchisio, M. Martina, G. Masera, and M. Shafique, 620
547 I. Vo, S. K. Esser, R. Appuswamy, B. Taba, A. Amir, M. D. Flickner, ‘‘CarSNN: An efficient spiking neural network for event-based 621
548 W. P. Risk, R. Manohar, and D. S. Modha, ‘‘A million spiking-neuron autonomous cars on the loihi neuromorphic research processor,’’ in 622
549 integrated circuit with a scalable communication network and interface,’’ Proc. Int. Joint Conf. Neural Netw. (IJCNN), Jul. 2021, pp. 1–10. 623
550 Science, vol. 345, no. 6197, pp. 668–673, Aug. 2014.
551 [8] M. Davies, N. Srinivasa, T. H. Lin, G. Chinya, Y. Cao, S. H. Choday,
552 G. Dimou, P. Joshi, N. Imam, S. Jain, and Y. Liao, ‘‘Loihi: A neuromorphic
553 manycore processor with on-chip learning,’’ IEEE Micro, vol. 38, no. 1,
554 pp. 82–99, Jan. 2018.
555 [9] A. Neckar, S. Fok, B. V. Benjamin, T. C. Stewart, N. N. Oza, A. R. Voelker,
556 C. Eliasmith, R. Manohar, and K. Boahen, ‘‘Braindrop: A mixed-signal
557 neuromorphic architecture with a dynamical systems-based programming
558 model,’’ Proc. IEEE, vol. 107, no. 1, pp. 144–164, Jan. 2019.
559 [10] S. Moradi, N. Qiao, F. Stefanini, and G. Indiveri, ‘‘A scalable multicore
560 architecture with heterogeneous memory structures for dynamic neuromor-
561 phic asynchronous processors (DYNAPs),’’ IEEE Trans. Biomed. Circuits
562 Syst., vol. 12, no. 1, pp. 106–122, Feb. 2018.
563 [11] V. Kornijcuk and D. S. Jeong, ‘‘Recent progress in real-time adaptable
564 digital neuromorphic hardware,’’ Adv. Intell. Syst., vol. 1, no. 6, Oct. 2019, DONGHYUNG YOO received the B.S. degree in 624
565 Art. no. 1900030. materials science and engineering from Hanyang 625
566 [12] X. Lagorce, G. Orchard, F. Gallupi, B. E. Shi, and R. Benosman, ‘‘HOTS: University, Seoul, South Korea, in 2017, where he 626
567 A hierarchy of event-based time-surfaces for pattern recognition,’’ IEEE is currently pursuing the Ph.D. degree in materi- 627
568 Trans. Pattern Anal. Mach. Intell., vol. 39, no. 7, pp. 1346–1359, Jan. 2017.
als science and engineering. His current research 628
569 [13] A. Sironi, M. Brambilla, N. Bourdis, X. Lagorce, and R. Benosman,
interests include learning algorithms for spiking 629
570 ‘‘HATS: Histograms of averaged time surfaces for robust event-based
571 object classification,’’ in Proc. IEEE/CVF Conf. Comput. Vis. Pattern neural networks and neuromorphic vision sensor. 630
596 Y. Xie, ‘‘Comparing SNNs and RNNs on neuromorphic vision datasets: an Associate Professor with Hanyang University, 638
597 Similarities and differences,’’ Neural Netw., vol. 132, pp. 108–120, South Korea. His research interests include spik- 639
598 Dec. 2020. ing neural networks for sequence learning, future 640
599 [21] Y. Wu, L. Deng, G. Li, J. Zhu, and L. Shi, ‘‘Spatio-temporal backpropa- prediction, learning algorithms, spiking neural network design, and digital 641
600 gation for training high-performance spiking neural networks,’’ Frontiers neuromorphic processor design. 642
601 Neurosci., vol. 12, p. 331, May 2018. 643