Strel streaming
Download as PPTX, PDF•0 likes•55 views
The document proposes translating rules expressed in Spatio-Temporal Reach and Escape Logic (STREL) to streaming-based monitoring applications. STREL allows expressing properties over attributes that vary in space and time. The contribution is defining streaming operators whose semantics enforce STREL rules by composing base streaming operators. An evaluation on Apache Flink shows the approach can achieve throughput of 1000-500 tuples/second and sub-millisecond latency depending on spatial and temporal resolution of the data. Future work includes further evaluation, additional temporal operators, path-based spatial analysis, and compilation optimizations.
Strel streaming
- 2. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 2
- 3. 95 95 95 • Monitoring rules are essential to detect anomalies / dangerous situations / broken agents... • Challenges: the # of devices, their heterogeneity, their complex interactions... • Wishlist: • easy-to-express rules “for the masses” • efficient analysis for timely detection UAVs should never get closer than 10 meters to cars
- 4. easy-to-express rules + efficient analysis • Ingredients: • High-level language Spatio-temporal logics • Continuous analysis paradigm Data streaming • Distributed / Parallel analysis framework Stream Processing Engines How do we take his knowledge... ... and transfer it here?
- 5. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 5
- 6. 7 data stream: unbounded sequence of tuples sharing the same schema 95 A 8:00 20 Ff Ff Ff A 8:03 15 Ff Ff Ff <UAV id, ts, speed, dest>
- 7. 8 95 A 8:00 20 Ff Ff Ff A 8:03 15 Ff Ff Ff Join Aggregate stateful continuous query: Directed Acyclic Graph of streams and operators Map Filter Union Multiplex stateless Compute the average speed in the last 10 minutes (group by UAV) Add to each UAV report its average speed in the last 10 minutes Report UAVs close to the destination that are not decelerating Stream Processing Engine (SPE)
- 8. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 9
- 9. ifo A d REET pg FEI f if a o A d REET FEI f if a o A d REET pg FEI f if a o A FEI f if a o A d REET STREL (Spatio-Temporal Reach and Escape Logic) • Spatio-temporal modal logic • Allows expressing properties over attributes which vary over space and time 𝕊2 -Operators (streaming-STREL operators) can be ported to the streaming paradigm by means of Base streaming operator whose semantics can be enforced composing FEI f if a o A FEI f if a o A d REET = 𝑓( ) = 𝑔( )
- 10. STREL (Spatio-Temporal Reach and Escape Logic) • Spatio-temporal modal logic • Allows expressing properties over attributes which vary over space and time 𝕊2 -Operators (streaming-STREL operators) can be ported to the streaming paradigm by means of Base streaming operators whose semantics can be enforced composing What’s the novelty? Can be automated!(1) Any of the existing SPEs can be used (2) Their distribution / parallelization / adaptiveness / fault-tolerance solutions can be leveraged! (3)
- 11. FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg STREL (Spatio-Temporal Reach and Escape Logic) • Atomic propositions (True/False/Undefined) • Logical operations (AND/OR/NOT) • Temporal operators: • Since • Sometimes in the past • Always in the past • Spatial operators: • Somewhere • Everywhere Join Aggregate stateful Map Filter Union Multiplex stateless d REET pg = 𝑔( ) Base streaming operators 𝕊2 -MAP 𝕊2 -MERGE 𝕊2 -TAGGR (𝕊2 -time-aggregate) 𝕊2-LAGGR (𝕊2-location-aggregate) 𝕊2-Operators (streaming-STREL operators) FEI f if a o A d REET pg FEI f if a o A d REET pg = 𝑓( )
- 12. FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg = 𝑓( ) d REET pg = 𝑔( ) STREL (Spatio-Temporal Reach and Escape Logic) Any combination of STREL operators... Base streaming operators 𝕊2 -Operators (streaming-STREL operators) A tree of 𝕊2 -Operators Each node of a tree of 𝕊2 -Operators... A DAG of base operators
- 13. FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg FEI f if a o A d REET pg = 𝑓( ) d REET pg = 𝑔( ) STREL (Spatio-Temporal Reach and Escape Logic) Base streaming operators 𝕊2 -Operators (streaming-STREL operators) the UAV should not have remained farther than 20 meters from its destination for longer than 10 minutes 95
- 14. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 15
- 15. Evaluation • Performance evaluation of the monitoring query: • Implemented using Apache Flink • Testbed: Odroid-XU4 • Synthetic data, discretized over • Time: sampling frequency f (Hz) • 2D space: number of cells c the UAV should not have remained farther than 20 meters from its destination for longer than 10 minutes 95
- 16. Throughput, latency and CPU consumption for increasing f values and fixed c values Evaluation
- 17. Evaluation Throughput, latency and CPU consumption for increasing c values and fixed f values
- 18. Evaluation For small to medium spatial resolution, throughput ~ 1000 t/s (max f = 8), for high spatial resolution, throughput ~ 500 t/s 0
- 19. Evaluation If the system is not overloaded, sub-millisecond latency
- 20. Agenda • Motivation • Preliminaries: data streaming in a nutshell • Contribution: translation of Spatio-Temporal Logics to Streaming-Based Monitoring Applications • Evaluation • Conclusion / Future work 21
- 21. Conclusions / future work What’s next? • Extended experimental evaluation • Future temporal operators • Spatial analysis based on paths • Compilation optimizations STREL (Spatio-Temporal Reach and Escape Logic) • Spatio-temporal modal logic • Allows expressing properties over attributes which vary over space and time 𝕊2 -Operators (streaming-STREL operators) can be ported to the streaming paradigm by means of Base streaming operator whose semantics can be enforced composing
Editor's Notes
- #4: In this work we focus on environments in which autonomous agents, Unmanned Arial Vehicles (UAVs) or cars, for instance, can interact with humans. /// In this environments, monitoring rules are used to check if everything is working correctly (simply put) /// Challenges [...] we usually have large / complex set of rules What we really want are easy-to-express rules (for experts that are not programmers) and efficient analysis (for timely detection)
- #5: to achieve these goals we know the “ingredients” of the recipe: [...] /// The problem is who should take care of mixing them. Our research question is: How do we [...]?
- #7: Differently from databases, which are designed to essentially store information and later query it, streaming applications (run by Stream Processing Engines) are running continuously (a streaming application is usually referred to as a continuous query), processing new incoming data and producing results accordingly.
- #8: Very briefly, in data streaming we have streams, unbounded [...], like this one {explain example}
- #9: Which are fed to DAGs of operators. The basic operators are divided into stateless {explain} and stateful {explain}. We could do something like this for instance: {explain}
- #11: We start by taking STREL: a Spatio-temporal [...] and show it can be ported to the streaming paradigm [...]. Then, we show that each one of these S2 operators [...]. In a LEGO-parallelism: we show that ...
- #12: Why is this important? What’s the novelty? [...] Let me show you a bit more in detail (but without too many formalisms, because of time)
- #13: With STREL, monitoring rules can be composed using propositions, [...]. When it comes to porting STREL to the streaming paradigm, we actually need 4 “rich” operators. And each one of these operators is actually a small composition of the basic operators I already mentioned before
- #14: As we discuss in the paper [...] Let me show you an example.
- #15: Rombo is somewhere negate always in the past negate