Scribd
Upload
11 views

Efficient Load Shedding Techniques For Distributed Stream Processing

The document discusses efficient load shedding techniques for distributed stream processing. When servers become overloaded due to bursty data, excessive load must be shed to maintain low query latency. Load shedding decisions must be well-coordinated across servers to control output quality as servers are interdependent. The paper models distributed load shedding as a linear optimization problem and proposes a centralized solver-based approach and a distributed metadata aggregation and propagation approach, both using pre-generated load shedding plans. Experimental results from a prototype in Borealis show the techniques' performance under different workloads.

Uploaded by

hafizrahimmit
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
11 views

Efficient Load Shedding Techniques For Distributed Stream Processing

The document discusses efficient load shedding techniques for distributed stream processing. When servers become overloaded due to bursty data, excessive load must be shed to maintain low query latency. Load shedding decisions must be well-coordinated across servers to control output quality as servers are interdependent. The paper models distributed load shedding as a linear optimization problem and proposes a centralized solver-based approach and a distributed metadata aggregation and propagation approach, both using pre-generated load shedding plans. Experimental results from a prototype in Borealis show the techniques' performance under different workloads.

Uploaded by

hafizrahimmit
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 1

efficient load shedding techniques for

distributed stream processing



In distributed stream processing environments, large numbers of continuous queries are distributed
onto multiple servers. When one or more of these servers become overloaded due to bursty data
arrival, excessive load needs to be shed in order to preserve low latency for the query results. Because
of the load dependencies among the servers, load shedding decisions on these servers must be well-
coordinated to achieve end-to-end control on the output quality. In this paper, we model the distributed
load shedding problem as a linear optimization problem, for which we propose two alternative solution
approaches: a solver-based centralized approach, and a distributed approach based on metadata
aggregation and propagation, whose centralized implementation is also available. Both of our solutions
are based on generating a series of load shedding plans in advance, to be used under certain input load
conditions. We have implemented our techniques as part of the Borealis distributed stream processing
system. We present experimental results from our prototype implementation showing the performance
of these techniques under different input and query workloads.
AUTHORS:-
Hesime tatbul,
ugur centintemel,
stanzdonik..

You might also like