SlideShare a Scribd company logo
Energy-Price-Driven Query Processing 
in 
Multi-center Web 
Search Engines 
Enver Kayaaslan (Bliken University) 
B. Barla Cambazoglu (Yahoo! Research) 
Roi Blanco (Yahoo! Research) 
Flavio Junqueira (Yahoo! Research) 
Cevdet Aykanat (Bilken University)
Overview 
• Large scale web search engines 
• Is it possible to decrease the energy financial costs? 
– Reducing the electric bill (>>35M$ annual) 
– Shifting workload between datacenters 
- 2 - 
• Agenda 
– Motivation 
– Problem Definition 
– Algorithm 
– Experiments
Centralized Web Search Engines 
- 3 -
- 4 - 
Query 
Central 
Broker 
Cluster 
Cluster 
Cluster 
Cluster 
Query Execution
In-side cluster processing 
- 5 - 
Query 
Cluster 
Result
- 6 - 
Multi-center WSE
- 7 - 
Metrics 
• Query Response Time 
– Under a satisfactory amount (400ms) 
• User to data center latency 
• Query processing operations 
– Query degradation 
• Peak Sustainable Throughput 
– Query overflows 
– Number of clusters 
• Total Energy Consumption 
– Query processing operations 
• Total Energy Cost (electric bill) 
– Energy price
Space/temporal variation in query workload 
- 8 -
Space/Temporal variation in energy price 
Spatial Temporal 
- 9 -
• To decrease the total energy cost, exploiting 
- Spatio-temporal variation in energy prices 
- Spatio-temporal variation in query workloads 
- 10 - 
• Constraints: 
– Limited hardware 
– Bounded response times 
Goal
- 11 - 
Query forwarding
- 12 - 
Result 
Query forwarding
- 13 - 
Result 
Query forwarding
0 Timeline 
- 14 - 
Problem definition
- 15 - 
Problem definition 
Minimize Financial Cost 
Response Time Constraint 
Workload Constraint
Query degradation rate 
Response Time Constraint 
- 16 - 
Query Degradation Rate
- 17 - 
Query overflow rate 
Workload Constraint 
Query Overflow Rate 
Realized workload 
=
Online Workload shifting algorithm 
- 18 - 
Remote Centers 
? 
Local Center 
User query
Workload estimation (no forwarding) 
0 Timeline 
Current time 
0 Timeline 
Current time 
0 Timeline 
Current time 
- 19 -
- 20 - 
 Estimates future 
workloads 
 Current energy prices 
 Capacities 
 Shifts workload 
evenly assuming 
every expensive data-center 
will forward 
queries evenly 
 Probability is the ratio 
of the forwarding rate 
to the estimated 
workload 
 Sort of conservative 
Generating probabilities
- 21 - 
Generating probabilities 
 Estimates future 
workloads 
 Current energy prices 
 Capacities 
 Shifts workload 
evenly assuming 
every expensive data-center 
will forward 
queries evenly 
 Probability is the ratio 
of the forwarding rate 
to the estimated 
workload 
 Sort of conservative
- 22 - 
Generating probabilities 
 Estimates future 
workloads 
 Current energy prices 
 Capacities 
 Shifts workload 
evenly assuming 
every expensive data-center 
will forward 
queries evenly 
 Probability is the ratio 
of the forwarding rate 
to the estimated 
workload 
 Sort of conservative
- 23 - 
A 
B 
C 
D 
Remote Centers 
Picking a data-center 
Local Center 
User query 
w.p. pA(A)
Query Workload Price Configuration 
- 24 - 
U T 
S ST 
Energy price configuration
- 25 - 
Set-up 
• 5 Datacenters 
• 38M queries, over 4 days 
• Turn caching on (lowers PST of the back-end) 
• Tuned capacities to a low query overflow rate (<0.005)
- 26 - 
Results (I) 
• 63% of the queries served by the cache 
– Reduces the PST of the backend 
• Average query response time increases (from 66ms to 
around 100ms) 
• Query degradation rate increases but is kept <5% for a 
budget of 400ms (none if budget > 800ms) 
• Overflow rate is the same as in the non-forwarding scenario 
• Forwarding rate is proportional to price variation 
• Despite network latencies forwarding is possible
Aggregate hourly query forwarding rate 
- 27 - 
 Global Time 
 Forwarding depends on ordinal 
ranking of current prices 
 In spatial it dominates price 
ordinal ranking more than intra-day 
variations (S correlates 
with ST) 
 Traffic and forwarding inverse 
relation 
 Takes into account energy 
prices
- 28 - 
 Relative to U setup 
 Increasing saving with larger 
response time limit 
 ST > S > T (r = ∞) 
 About 35% saving at ST 
Savings in electric cost
Savings vs electric price vs forwarding rate 
- 29 - 
 Local Time (where the query 
is processed) 
 T setup, correlates with 
temporal effects 
 r = 800 (~ ∞) 
 Query forwarding rate: 
positive correlation 
 Electric price: 
negative correlation 
 What happens at 17:00?
- 30 - 
Conclusions 
• Presented the reduction of the electric bill in distributed 
search engines as an online optimization framework 
• A practical algorithm, based on shifting query workloads 
• Evaluation of potential savings via realistic simulations 
• Depending on electric price distribution, energy costs can be 
significantly reduced by shifting query workloads to energy-cheap 
data centers. 
– By maintaining the overflow rate equal to that of a centralized 
engine 
– By keeping the query degradation rate < 5% 
• The higher the variability in the configuration the higher the 
savings
- 31 - 
Future work 
• Price-aware crawling 
• Price-aware indexer 
• Energy-aware caches 
• Green search engines!
Reducing the financial cost per query 
- 32 -
- 33 - 
 No bound on response time, 
r = ∞ 
 Almost all can be answered 
under 800ms, 5% under 
400ms 
 U does not forward 
 T forwards more 
Query forwarding vs budget
- 34 - 
PST 
(for each datacenter) 
Window Size 
(for each price configuration) 
PST and window size
Query processing time estimation 
- 35 - 
Query q = {t1, t2, t3} 
LIST 1 
LIST2 
LIST3 
t1 
t2 
t3 
results = {r11,r15, r22, r26 r31} 
#operations ~ c + w(|LIST1| + |LIST2| + |LIST3|)

More Related Content

What's hot (16)

Power Comparison Power Comparison of Cloud Data of Cloud Data Center Architec...
Power Comparison Power Comparison of Cloud Data of Cloud Data Center Architec...Power Comparison Power Comparison of Cloud Data of Cloud Data Center Architec...
Power Comparison Power Comparison of Cloud Data of Cloud Data Center Architec...
Paolo Giaccone
 
Bond Graph of a One Stage Reduction Gearbox
Bond Graph of a One Stage Reduction GearboxBond Graph of a One Stage Reduction Gearbox
Bond Graph of a One Stage Reduction Gearbox
Gehendra Sharma
 
A framework for dynamic pricing electricity consumption patterns via time ser...
A framework for dynamic pricing electricity consumption patterns via time ser...A framework for dynamic pricing electricity consumption patterns via time ser...
A framework for dynamic pricing electricity consumption patterns via time ser...
Asoka Korale
 
POWER SYSTEM PLANNING
POWER SYSTEM PLANNINGPOWER SYSTEM PLANNING
POWER SYSTEM PLANNING
NishaDangi99
 
Optimization for-power-sy-8631549
Optimization for-power-sy-8631549Optimization for-power-sy-8631549
Optimization for-power-sy-8631549
Kannan Kathiravan
 
Bradley Glenn: Holomorphic Embedding Load Flow Method (helmtm) Algorithm Deve...
Bradley Glenn: Holomorphic Embedding Load Flow Method (helmtm) Algorithm Deve...Bradley Glenn: Holomorphic Embedding Load Flow Method (helmtm) Algorithm Deve...
Bradley Glenn: Holomorphic Embedding Load Flow Method (helmtm) Algorithm Deve...
EnergyTech2015
 
Short-Term Load Forecasting of Australian National Electricity Market by Hier...
Short-Term Load Forecasting of Australian National Electricity Market by Hier...Short-Term Load Forecasting of Australian National Electricity Market by Hier...
Short-Term Load Forecasting of Australian National Electricity Market by Hier...
JEE HYUN PARK
 
Benjamin Loop: Simulation Environment for Power Management and Distribution D...
Benjamin Loop: Simulation Environment for Power Management and Distribution D...Benjamin Loop: Simulation Environment for Power Management and Distribution D...
Benjamin Loop: Simulation Environment for Power Management and Distribution D...
EnergyTech2015
 
Modern power system planning new
Modern power system planning newModern power system planning new
Modern power system planning new
Bayu imadul Bilad
 
Anne McNelis: Intelligent Power Controller Development for Human Deep Space ...
 Anne McNelis: Intelligent Power Controller Development for Human Deep Space ... Anne McNelis: Intelligent Power Controller Development for Human Deep Space ...
Anne McNelis: Intelligent Power Controller Development for Human Deep Space ...
EnergyTech2015
 
Power Quality Analysis and Enhancement of Grid Connected Solar Energy System
Power Quality Analysis and Enhancement of Grid Connected Solar Energy SystemPower Quality Analysis and Enhancement of Grid Connected Solar Energy System
Power Quality Analysis and Enhancement of Grid Connected Solar Energy System
Asoka Technologies
 
Optimal multiserver configuration for profit
Optimal multiserver configuration for profitOptimal multiserver configuration for profit
Optimal multiserver configuration for profit
IMPULSE_TECHNOLOGY
 
Optimization in power system
Optimization in power systemOptimization in power system
Optimization in power system
Hassan Khalid Uppal
 
AC-Based Differential Evolution Algorithm for Dynamic Transmission Expansion ...
AC-Based Differential Evolution Algorithm for Dynamic Transmission Expansion ...AC-Based Differential Evolution Algorithm for Dynamic Transmission Expansion ...
AC-Based Differential Evolution Algorithm for Dynamic Transmission Expansion ...
TELKOMNIKA JOURNAL
 
Application of Optimal Control theory to HEV
Application of Optimal Control theory to HEVApplication of Optimal Control theory to HEV
Application of Optimal Control theory to HEV
Raja Sekhar
 
Reliability Constrained Unit Commitment Considering the Effect of DG and DR P...
Reliability Constrained Unit Commitment Considering the Effect of DG and DR P...Reliability Constrained Unit Commitment Considering the Effect of DG and DR P...
Reliability Constrained Unit Commitment Considering the Effect of DG and DR P...
IJECEIAES
 
Power Comparison Power Comparison of Cloud Data of Cloud Data Center Architec...
Power Comparison Power Comparison of Cloud Data of Cloud Data Center Architec...Power Comparison Power Comparison of Cloud Data of Cloud Data Center Architec...
Power Comparison Power Comparison of Cloud Data of Cloud Data Center Architec...
Paolo Giaccone
 
Bond Graph of a One Stage Reduction Gearbox
Bond Graph of a One Stage Reduction GearboxBond Graph of a One Stage Reduction Gearbox
Bond Graph of a One Stage Reduction Gearbox
Gehendra Sharma
 
A framework for dynamic pricing electricity consumption patterns via time ser...
A framework for dynamic pricing electricity consumption patterns via time ser...A framework for dynamic pricing electricity consumption patterns via time ser...
A framework for dynamic pricing electricity consumption patterns via time ser...
Asoka Korale
 
POWER SYSTEM PLANNING
POWER SYSTEM PLANNINGPOWER SYSTEM PLANNING
POWER SYSTEM PLANNING
NishaDangi99
 
Optimization for-power-sy-8631549
Optimization for-power-sy-8631549Optimization for-power-sy-8631549
Optimization for-power-sy-8631549
Kannan Kathiravan
 
Bradley Glenn: Holomorphic Embedding Load Flow Method (helmtm) Algorithm Deve...
Bradley Glenn: Holomorphic Embedding Load Flow Method (helmtm) Algorithm Deve...Bradley Glenn: Holomorphic Embedding Load Flow Method (helmtm) Algorithm Deve...
Bradley Glenn: Holomorphic Embedding Load Flow Method (helmtm) Algorithm Deve...
EnergyTech2015
 
Short-Term Load Forecasting of Australian National Electricity Market by Hier...
Short-Term Load Forecasting of Australian National Electricity Market by Hier...Short-Term Load Forecasting of Australian National Electricity Market by Hier...
Short-Term Load Forecasting of Australian National Electricity Market by Hier...
JEE HYUN PARK
 
Benjamin Loop: Simulation Environment for Power Management and Distribution D...
Benjamin Loop: Simulation Environment for Power Management and Distribution D...Benjamin Loop: Simulation Environment for Power Management and Distribution D...
Benjamin Loop: Simulation Environment for Power Management and Distribution D...
EnergyTech2015
 
Modern power system planning new
Modern power system planning newModern power system planning new
Modern power system planning new
Bayu imadul Bilad
 
Anne McNelis: Intelligent Power Controller Development for Human Deep Space ...
 Anne McNelis: Intelligent Power Controller Development for Human Deep Space ... Anne McNelis: Intelligent Power Controller Development for Human Deep Space ...
Anne McNelis: Intelligent Power Controller Development for Human Deep Space ...
EnergyTech2015
 
Power Quality Analysis and Enhancement of Grid Connected Solar Energy System
Power Quality Analysis and Enhancement of Grid Connected Solar Energy SystemPower Quality Analysis and Enhancement of Grid Connected Solar Energy System
Power Quality Analysis and Enhancement of Grid Connected Solar Energy System
Asoka Technologies
 
Optimal multiserver configuration for profit
Optimal multiserver configuration for profitOptimal multiserver configuration for profit
Optimal multiserver configuration for profit
IMPULSE_TECHNOLOGY
 
AC-Based Differential Evolution Algorithm for Dynamic Transmission Expansion ...
AC-Based Differential Evolution Algorithm for Dynamic Transmission Expansion ...AC-Based Differential Evolution Algorithm for Dynamic Transmission Expansion ...
AC-Based Differential Evolution Algorithm for Dynamic Transmission Expansion ...
TELKOMNIKA JOURNAL
 
Application of Optimal Control theory to HEV
Application of Optimal Control theory to HEVApplication of Optimal Control theory to HEV
Application of Optimal Control theory to HEV
Raja Sekhar
 
Reliability Constrained Unit Commitment Considering the Effect of DG and DR P...
Reliability Constrained Unit Commitment Considering the Effect of DG and DR P...Reliability Constrained Unit Commitment Considering the Effect of DG and DR P...
Reliability Constrained Unit Commitment Considering the Effect of DG and DR P...
IJECEIAES
 

Viewers also liked (20)

Gender in NTFP value chains in the Congo Basin
Gender in NTFP value chains in the Congo BasinGender in NTFP value chains in the Congo Basin
Gender in NTFP value chains in the Congo Basin
Verina Ingram
 
Around the garden
Around the gardenAround the garden
Around the garden
Victoria Jensen
 
Physical Science Ch 2, sec 1 tools in science
Physical Science Ch 2, sec 1 tools in sciencePhysical Science Ch 2, sec 1 tools in science
Physical Science Ch 2, sec 1 tools in science
mshenry
 
PRBS - Where YOU can make a difference
PRBS - Where YOU can make a differencePRBS - Where YOU can make a difference
PRBS - Where YOU can make a difference
PRBS
 
Basicgrammar1
Basicgrammar1Basicgrammar1
Basicgrammar1
Vicky
 
Web 2.0..Business Friend or Foe?
Web 2.0..Business Friend or Foe?Web 2.0..Business Friend or Foe?
Web 2.0..Business Friend or Foe?
Stites & Harbison
 
D:\งานส่ง\G48 53011810070
D:\งานส่ง\G48 53011810070D:\งานส่ง\G48 53011810070
D:\งานส่ง\G48 53011810070
BenjamasS
 
Nupd 400 chapter 3 culture
Nupd 400 chapter 3 cultureNupd 400 chapter 3 culture
Nupd 400 chapter 3 culture
4161
 
Pkbm ekonomi 11 01
Pkbm ekonomi 11 01Pkbm ekonomi 11 01
Pkbm ekonomi 11 01
Ridwan Gucci
 
Economic potential of future growth at the halifax shipyard
Economic potential of future growth at the halifax shipyardEconomic potential of future growth at the halifax shipyard
Economic potential of future growth at the halifax shipyard
Halifax Partnership
 
Responsible Disclosure - For Dutch ISACA chapter
Responsible Disclosure - For Dutch ISACA chapterResponsible Disclosure - For Dutch ISACA chapter
Responsible Disclosure - For Dutch ISACA chapter
Frank Breedijk
 
Effective and Efficient Entity Search in RDF data
Effective and Efficient Entity Search in RDF dataEffective and Efficient Entity Search in RDF data
Effective and Efficient Entity Search in RDF data
Roi Blanco
 
PA EI Leadership Meeting: Technology to Suppoty EI Leaders
PA EI Leadership Meeting: Technology to Suppoty EI LeadersPA EI Leadership Meeting: Technology to Suppoty EI Leaders
PA EI Leadership Meeting: Technology to Suppoty EI Leaders
Jenny Brown
 
Tmh5 rahasia-kitab-tujuh
Tmh5 rahasia-kitab-tujuhTmh5 rahasia-kitab-tujuh
Tmh5 rahasia-kitab-tujuh
Ridwan Gucci
 
本当は怖いオープンデータ・ビッグデータ
本当は怖いオープンデータ・ビッグデータ本当は怖いオープンデータ・ビッグデータ
本当は怖いオープンデータ・ビッグデータ
pgcafe
 
Dream's graphic
Dream's graphicDream's graphic
Dream's graphic
Walter
 
SD92 Nisga'a Language & Culture Presentation
SD92 Nisga'a Language & Culture PresentationSD92 Nisga'a Language & Culture Presentation
SD92 Nisga'a Language & Culture Presentation
School District 92, Nisga'a
 
Andy warhol
Andy warholAndy warhol
Andy warhol
iLoveGeorgeStr8
 
Computing - Delivering Innovative Research
Computing - Delivering Innovative ResearchComputing - Delivering Innovative Research
Computing - Delivering Innovative Research
Peter Lancaster
 
AABM - September 2012 Newsletter
AABM - September 2012 NewsletterAABM - September 2012 Newsletter
AABM - September 2012 Newsletter
Felix Ortiz
 
Gender in NTFP value chains in the Congo Basin
Gender in NTFP value chains in the Congo BasinGender in NTFP value chains in the Congo Basin
Gender in NTFP value chains in the Congo Basin
Verina Ingram
 
Physical Science Ch 2, sec 1 tools in science
Physical Science Ch 2, sec 1 tools in sciencePhysical Science Ch 2, sec 1 tools in science
Physical Science Ch 2, sec 1 tools in science
mshenry
 
PRBS - Where YOU can make a difference
PRBS - Where YOU can make a differencePRBS - Where YOU can make a difference
PRBS - Where YOU can make a difference
PRBS
 
Basicgrammar1
Basicgrammar1Basicgrammar1
Basicgrammar1
Vicky
 
Web 2.0..Business Friend or Foe?
Web 2.0..Business Friend or Foe?Web 2.0..Business Friend or Foe?
Web 2.0..Business Friend or Foe?
Stites & Harbison
 
D:\งานส่ง\G48 53011810070
D:\งานส่ง\G48 53011810070D:\งานส่ง\G48 53011810070
D:\งานส่ง\G48 53011810070
BenjamasS
 
Nupd 400 chapter 3 culture
Nupd 400 chapter 3 cultureNupd 400 chapter 3 culture
Nupd 400 chapter 3 culture
4161
 
Pkbm ekonomi 11 01
Pkbm ekonomi 11 01Pkbm ekonomi 11 01
Pkbm ekonomi 11 01
Ridwan Gucci
 
Economic potential of future growth at the halifax shipyard
Economic potential of future growth at the halifax shipyardEconomic potential of future growth at the halifax shipyard
Economic potential of future growth at the halifax shipyard
Halifax Partnership
 
Responsible Disclosure - For Dutch ISACA chapter
Responsible Disclosure - For Dutch ISACA chapterResponsible Disclosure - For Dutch ISACA chapter
Responsible Disclosure - For Dutch ISACA chapter
Frank Breedijk
 
Effective and Efficient Entity Search in RDF data
Effective and Efficient Entity Search in RDF dataEffective and Efficient Entity Search in RDF data
Effective and Efficient Entity Search in RDF data
Roi Blanco
 
PA EI Leadership Meeting: Technology to Suppoty EI Leaders
PA EI Leadership Meeting: Technology to Suppoty EI LeadersPA EI Leadership Meeting: Technology to Suppoty EI Leaders
PA EI Leadership Meeting: Technology to Suppoty EI Leaders
Jenny Brown
 
Tmh5 rahasia-kitab-tujuh
Tmh5 rahasia-kitab-tujuhTmh5 rahasia-kitab-tujuh
Tmh5 rahasia-kitab-tujuh
Ridwan Gucci
 
本当は怖いオープンデータ・ビッグデータ
本当は怖いオープンデータ・ビッグデータ本当は怖いオープンデータ・ビッグデータ
本当は怖いオープンデータ・ビッグデータ
pgcafe
 
Dream's graphic
Dream's graphicDream's graphic
Dream's graphic
Walter
 
Computing - Delivering Innovative Research
Computing - Delivering Innovative ResearchComputing - Delivering Innovative Research
Computing - Delivering Innovative Research
Peter Lancaster
 
AABM - September 2012 Newsletter
AABM - September 2012 NewsletterAABM - September 2012 Newsletter
AABM - September 2012 Newsletter
Felix Ortiz
 

Similar to Energy-Price-Driven Query Processing in Multi-center Web Search Engines (20)

SUSTAINABLE HIGH-SPEED DATA TRANSFER TECHNIQUES IN SHARED NETWORK ENVIRONMENTS
SUSTAINABLE HIGH-SPEED DATA TRANSFER TECHNIQUES IN SHARED NETWORK ENVIRONMENTSSUSTAINABLE HIGH-SPEED DATA TRANSFER TECHNIQUES IN SHARED NETWORK ENVIRONMENTS
SUSTAINABLE HIGH-SPEED DATA TRANSFER TECHNIQUES IN SHARED NETWORK ENVIRONMENTS
hasibjamil
 
An empirical evaluation of cost-based federated SPARQL query Processing Engines
An empirical evaluation of cost-based federated SPARQL query Processing EnginesAn empirical evaluation of cost-based federated SPARQL query Processing Engines
An empirical evaluation of cost-based federated SPARQL query Processing Engines
Umair Qudus
 
MRI Energy-Efficient Cloud Computing
MRI Energy-Efficient Cloud ComputingMRI Energy-Efficient Cloud Computing
MRI Energy-Efficient Cloud Computing
Roger Rafanell Mas
 
Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Cen...
Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Cen...Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Cen...
Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Cen...
Papitha Velumani
 
Distributed Convex Optimization Thesis - Behroz Sikander
Distributed Convex Optimization Thesis - Behroz SikanderDistributed Convex Optimization Thesis - Behroz Sikander
Distributed Convex Optimization Thesis - Behroz Sikander
rogerz1234567
 
Energy Efficiency in Data Centers
Energy Efficiency in Data CentersEnergy Efficiency in Data Centers
Energy Efficiency in Data Centers
GreenLSI Team, LSI, UPM
 
Metaheuristics-based Optimal Reactive Power Management in Offshore Wind Farms...
Metaheuristics-based Optimal Reactive Power Management in Offshore Wind Farms...Metaheuristics-based Optimal Reactive Power Management in Offshore Wind Farms...
Metaheuristics-based Optimal Reactive Power Management in Offshore Wind Farms...
Aimilia-Myrsini Theologi
 
Thermal-aware scheduling of batch jobs in Geographically distributed data cen...
Thermal-aware scheduling of batch jobs in Geographically distributed data cen...Thermal-aware scheduling of batch jobs in Geographically distributed data cen...
Thermal-aware scheduling of batch jobs in Geographically distributed data cen...
Papitha Velumani
 
IRJET- An Optimal Algorithm for Data Centres to Minimize the Power Supply
IRJET-  	  An Optimal Algorithm for Data Centres to Minimize the Power SupplyIRJET-  	  An Optimal Algorithm for Data Centres to Minimize the Power Supply
IRJET- An Optimal Algorithm for Data Centres to Minimize the Power Supply
IRJET Journal
 
Valuing demand response pub
Valuing demand response pubValuing demand response pub
Valuing demand response pub
Lucas Finco
 
Architecting for Sustainability
Architecting for SustainabilityArchitecting for Sustainability
Architecting for Sustainability
ssuserd4e0d2
 
A Cyber Physical Approach to a Combined Hardware-Software
A Cyber Physical Approach to a Combined Hardware-Software A Cyber Physical Approach to a Combined Hardware-Software
A Cyber Physical Approach to a Combined Hardware-Software
GreenLSI Team, LSI, UPM
 
RT15 Berkeley | Optimized Power Flow Control in Microgrids - Sandia Laboratory
RT15 Berkeley | Optimized Power Flow Control in Microgrids - Sandia LaboratoryRT15 Berkeley | Optimized Power Flow Control in Microgrids - Sandia Laboratory
RT15 Berkeley | Optimized Power Flow Control in Microgrids - Sandia Laboratory
OPAL-RT TECHNOLOGIES
 
Smart optimization techniques for virtual power plants
Smart optimization techniques for virtual power plants Smart optimization techniques for virtual power plants
Smart optimization techniques for virtual power plants
Babatunde Odetayo, PhD, P.Eng, PMP
 
Vldb14
Vldb14Vldb14
Vldb14
hdbtracker
 
Energy efficient computing & computational services
Energy efficient computing & computational services Energy efficient computing & computational services
Energy efficient computing & computational services
David Wallom
 
SG Data analytics.pptx
SG Data analytics.pptxSG Data analytics.pptx
SG Data analytics.pptx
Danish Mahmood
 
Parallel Programming for Multi- Core and Cluster Systems - Performance Analysis
Parallel Programming for Multi- Core and Cluster Systems - Performance AnalysisParallel Programming for Multi- Core and Cluster Systems - Performance Analysis
Parallel Programming for Multi- Core and Cluster Systems - Performance Analysis
Shah Zaib
 
Is accurate system-level power measurement challenging? Check this out!
Is accurate system-level power measurement challenging? Check this out!Is accurate system-level power measurement challenging? Check this out!
Is accurate system-level power measurement challenging? Check this out!
Deepak Shankar
 
IRJET- Reducing electricity usage in Internet using transactional data
IRJET-  	  Reducing electricity usage in Internet using transactional dataIRJET-  	  Reducing electricity usage in Internet using transactional data
IRJET- Reducing electricity usage in Internet using transactional data
IRJET Journal
 
SUSTAINABLE HIGH-SPEED DATA TRANSFER TECHNIQUES IN SHARED NETWORK ENVIRONMENTS
SUSTAINABLE HIGH-SPEED DATA TRANSFER TECHNIQUES IN SHARED NETWORK ENVIRONMENTSSUSTAINABLE HIGH-SPEED DATA TRANSFER TECHNIQUES IN SHARED NETWORK ENVIRONMENTS
SUSTAINABLE HIGH-SPEED DATA TRANSFER TECHNIQUES IN SHARED NETWORK ENVIRONMENTS
hasibjamil
 
An empirical evaluation of cost-based federated SPARQL query Processing Engines
An empirical evaluation of cost-based federated SPARQL query Processing EnginesAn empirical evaluation of cost-based federated SPARQL query Processing Engines
An empirical evaluation of cost-based federated SPARQL query Processing Engines
Umair Qudus
 
MRI Energy-Efficient Cloud Computing
MRI Energy-Efficient Cloud ComputingMRI Energy-Efficient Cloud Computing
MRI Energy-Efficient Cloud Computing
Roger Rafanell Mas
 
Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Cen...
Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Cen...Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Cen...
Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Cen...
Papitha Velumani
 
Distributed Convex Optimization Thesis - Behroz Sikander
Distributed Convex Optimization Thesis - Behroz SikanderDistributed Convex Optimization Thesis - Behroz Sikander
Distributed Convex Optimization Thesis - Behroz Sikander
rogerz1234567
 
Metaheuristics-based Optimal Reactive Power Management in Offshore Wind Farms...
Metaheuristics-based Optimal Reactive Power Management in Offshore Wind Farms...Metaheuristics-based Optimal Reactive Power Management in Offshore Wind Farms...
Metaheuristics-based Optimal Reactive Power Management in Offshore Wind Farms...
Aimilia-Myrsini Theologi
 
Thermal-aware scheduling of batch jobs in Geographically distributed data cen...
Thermal-aware scheduling of batch jobs in Geographically distributed data cen...Thermal-aware scheduling of batch jobs in Geographically distributed data cen...
Thermal-aware scheduling of batch jobs in Geographically distributed data cen...
Papitha Velumani
 
IRJET- An Optimal Algorithm for Data Centres to Minimize the Power Supply
IRJET-  	  An Optimal Algorithm for Data Centres to Minimize the Power SupplyIRJET-  	  An Optimal Algorithm for Data Centres to Minimize the Power Supply
IRJET- An Optimal Algorithm for Data Centres to Minimize the Power Supply
IRJET Journal
 
Valuing demand response pub
Valuing demand response pubValuing demand response pub
Valuing demand response pub
Lucas Finco
 
Architecting for Sustainability
Architecting for SustainabilityArchitecting for Sustainability
Architecting for Sustainability
ssuserd4e0d2
 
A Cyber Physical Approach to a Combined Hardware-Software
A Cyber Physical Approach to a Combined Hardware-Software A Cyber Physical Approach to a Combined Hardware-Software
A Cyber Physical Approach to a Combined Hardware-Software
GreenLSI Team, LSI, UPM
 
RT15 Berkeley | Optimized Power Flow Control in Microgrids - Sandia Laboratory
RT15 Berkeley | Optimized Power Flow Control in Microgrids - Sandia LaboratoryRT15 Berkeley | Optimized Power Flow Control in Microgrids - Sandia Laboratory
RT15 Berkeley | Optimized Power Flow Control in Microgrids - Sandia Laboratory
OPAL-RT TECHNOLOGIES
 
Energy efficient computing & computational services
Energy efficient computing & computational services Energy efficient computing & computational services
Energy efficient computing & computational services
David Wallom
 
SG Data analytics.pptx
SG Data analytics.pptxSG Data analytics.pptx
SG Data analytics.pptx
Danish Mahmood
 
Parallel Programming for Multi- Core and Cluster Systems - Performance Analysis
Parallel Programming for Multi- Core and Cluster Systems - Performance AnalysisParallel Programming for Multi- Core and Cluster Systems - Performance Analysis
Parallel Programming for Multi- Core and Cluster Systems - Performance Analysis
Shah Zaib
 
Is accurate system-level power measurement challenging? Check this out!
Is accurate system-level power measurement challenging? Check this out!Is accurate system-level power measurement challenging? Check this out!
Is accurate system-level power measurement challenging? Check this out!
Deepak Shankar
 
IRJET- Reducing electricity usage in Internet using transactional data
IRJET-  	  Reducing electricity usage in Internet using transactional dataIRJET-  	  Reducing electricity usage in Internet using transactional data
IRJET- Reducing electricity usage in Internet using transactional data
IRJET Journal
 

More from Roi Blanco (13)

From Queries to Answers in the Web
From Queries to Answers in the WebFrom Queries to Answers in the Web
From Queries to Answers in the Web
Roi Blanco
 
Entity Linking via Graph-Distance Minimization
Entity Linking via Graph-Distance MinimizationEntity Linking via Graph-Distance Minimization
Entity Linking via Graph-Distance Minimization
Roi Blanco
 
Introduction to Big Data
Introduction to Big DataIntroduction to Big Data
Introduction to Big Data
Roi Blanco
 
Mining Web content for Enhanced Search
Mining Web content for Enhanced Search Mining Web content for Enhanced Search
Mining Web content for Enhanced Search
Roi Blanco
 
Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement
Roi Blanco
 
Introduction to Information Retrieval
Introduction to Information RetrievalIntroduction to Information Retrieval
Introduction to Information Retrieval
Roi Blanco
 
Searching over the past, present and future
Searching over the past, present and futureSearching over the past, present and future
Searching over the past, present and future
Roi Blanco
 
Beyond document retrieval using semantic annotations
Beyond document retrieval using semantic annotations Beyond document retrieval using semantic annotations
Beyond document retrieval using semantic annotations
Roi Blanco
 
Keyword Search over RDF Graphs
Keyword Search over RDF GraphsKeyword Search over RDF Graphs
Keyword Search over RDF Graphs
Roi Blanco
 
Large-Scale Semantic Search
Large-Scale Semantic SearchLarge-Scale Semantic Search
Large-Scale Semantic Search
Roi Blanco
 
Extending BM25 with multiple query operators
Extending BM25 with multiple query operatorsExtending BM25 with multiple query operators
Extending BM25 with multiple query operators
Roi Blanco
 
Caching Search Engine Results over Incremental Indices
Caching Search Engine Results over Incremental IndicesCaching Search Engine Results over Incremental Indices
Caching Search Engine Results over Incremental Indices
Roi Blanco
 
Finding support sentences for entities
Finding support sentences for entitiesFinding support sentences for entities
Finding support sentences for entities
Roi Blanco
 
From Queries to Answers in the Web
From Queries to Answers in the WebFrom Queries to Answers in the Web
From Queries to Answers in the Web
Roi Blanco
 
Entity Linking via Graph-Distance Minimization
Entity Linking via Graph-Distance MinimizationEntity Linking via Graph-Distance Minimization
Entity Linking via Graph-Distance Minimization
Roi Blanco
 
Introduction to Big Data
Introduction to Big DataIntroduction to Big Data
Introduction to Big Data
Roi Blanco
 
Mining Web content for Enhanced Search
Mining Web content for Enhanced Search Mining Web content for Enhanced Search
Mining Web content for Enhanced Search
Roi Blanco
 
Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement Influence of Timeline and Named-entity Components on User Engagement
Influence of Timeline and Named-entity Components on User Engagement
Roi Blanco
 
Introduction to Information Retrieval
Introduction to Information RetrievalIntroduction to Information Retrieval
Introduction to Information Retrieval
Roi Blanco
 
Searching over the past, present and future
Searching over the past, present and futureSearching over the past, present and future
Searching over the past, present and future
Roi Blanco
 
Beyond document retrieval using semantic annotations
Beyond document retrieval using semantic annotations Beyond document retrieval using semantic annotations
Beyond document retrieval using semantic annotations
Roi Blanco
 
Keyword Search over RDF Graphs
Keyword Search over RDF GraphsKeyword Search over RDF Graphs
Keyword Search over RDF Graphs
Roi Blanco
 
Large-Scale Semantic Search
Large-Scale Semantic SearchLarge-Scale Semantic Search
Large-Scale Semantic Search
Roi Blanco
 
Extending BM25 with multiple query operators
Extending BM25 with multiple query operatorsExtending BM25 with multiple query operators
Extending BM25 with multiple query operators
Roi Blanco
 
Caching Search Engine Results over Incremental Indices
Caching Search Engine Results over Incremental IndicesCaching Search Engine Results over Incremental Indices
Caching Search Engine Results over Incremental Indices
Roi Blanco
 
Finding support sentences for entities
Finding support sentences for entitiesFinding support sentences for entities
Finding support sentences for entities
Roi Blanco
 

Recently uploaded (20)

tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?
Daniel Lehner
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsAI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global Trends
InData Labs
 
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxDevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
Justin Reock
 
Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)
Ortus Solutions, Corp
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
How analogue intelligence complements AI
How analogue intelligence complements AIHow analogue intelligence complements AI
How analogue intelligence complements AI
Paul Rowe
 
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
SOFTTECHHUB
 
Mobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi ArabiaMobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi Arabia
Steve Jonas
 
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
BookNet Canada
 
Role of Data Annotation Services in AI-Powered Manufacturing
Role of Data Annotation Services in AI-Powered ManufacturingRole of Data Annotation Services in AI-Powered Manufacturing
Role of Data Annotation Services in AI-Powered Manufacturing
Andrew Leo
 
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul
 
Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.
hpbmnnxrvb
 
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfThe Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
Abi john
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 
tecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdftecnologias de las primeras civilizaciones.pdf
tecnologias de las primeras civilizaciones.pdf
fjgm517
 
How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?How Can I use the AI Hype in my Business Context?
How Can I use the AI Hype in my Business Context?
Daniel Lehner
 
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptxIncreasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Increasing Retail Store Efficiency How can Planograms Save Time and Money.pptx
Anoop Ashok
 
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdfSAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
SAP Modernization: Maximizing the Value of Your SAP S/4HANA Migration.pdf
Precisely
 
AI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global TrendsAI and Data Privacy in 2025: Global Trends
AI and Data Privacy in 2025: Global Trends
InData Labs
 
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptxDevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
DevOpsDays Atlanta 2025 - Building 10x Development Organizations.pptx
Justin Reock
 
Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)Into The Box Conference Keynote Day 1 (ITB2025)
Into The Box Conference Keynote Day 1 (ITB2025)
Ortus Solutions, Corp
 
HCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser EnvironmentsHCL Nomad Web – Best Practices and Managing Multiuser Environments
HCL Nomad Web – Best Practices and Managing Multiuser Environments
panagenda
 
Quantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur MorganQuantum Computing Quick Research Guide by Arthur Morgan
Quantum Computing Quick Research Guide by Arthur Morgan
Arthur Morgan
 
Big Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur MorganBig Data Analytics Quick Research Guide by Arthur Morgan
Big Data Analytics Quick Research Guide by Arthur Morgan
Arthur Morgan
 
How analogue intelligence complements AI
How analogue intelligence complements AIHow analogue intelligence complements AI
How analogue intelligence complements AI
Paul Rowe
 
Heap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and DeletionHeap, Types of Heap, Insertion and Deletion
Heap, Types of Heap, Insertion and Deletion
Jaydeep Kale
 
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
AI EngineHost Review: Revolutionary USA Datacenter-Based Hosting with NVIDIA ...
SOFTTECHHUB
 
Mobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi ArabiaMobile App Development Company in Saudi Arabia
Mobile App Development Company in Saudi Arabia
Steve Jonas
 
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
#StandardsGoals for 2025: Standards & certification roundup - Tech Forum 2025
BookNet Canada
 
Role of Data Annotation Services in AI-Powered Manufacturing
Role of Data Annotation Services in AI-Powered ManufacturingRole of Data Annotation Services in AI-Powered Manufacturing
Role of Data Annotation Services in AI-Powered Manufacturing
Andrew Leo
 
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul Shares 5 Steps to Implement AI Agents for Maximum Business Efficien...
Noah Loul
 
Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.Greenhouse_Monitoring_Presentation.pptx.
Greenhouse_Monitoring_Presentation.pptx.
hpbmnnxrvb
 
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdfThe Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
The Evolution of Meme Coins A New Era for Digital Currency ppt.pdf
Abi john
 
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven InsightsAndrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell: Transforming Business Strategy Through Data-Driven Insights
Andrew Marnell
 

Energy-Price-Driven Query Processing in Multi-center Web Search Engines

  • 1. Energy-Price-Driven Query Processing in Multi-center Web Search Engines Enver Kayaaslan (Bliken University) B. Barla Cambazoglu (Yahoo! Research) Roi Blanco (Yahoo! Research) Flavio Junqueira (Yahoo! Research) Cevdet Aykanat (Bilken University)
  • 2. Overview • Large scale web search engines • Is it possible to decrease the energy financial costs? – Reducing the electric bill (>>35M$ annual) – Shifting workload between datacenters - 2 - • Agenda – Motivation – Problem Definition – Algorithm – Experiments
  • 3. Centralized Web Search Engines - 3 -
  • 4. - 4 - Query Central Broker Cluster Cluster Cluster Cluster Query Execution
  • 5. In-side cluster processing - 5 - Query Cluster Result
  • 6. - 6 - Multi-center WSE
  • 7. - 7 - Metrics • Query Response Time – Under a satisfactory amount (400ms) • User to data center latency • Query processing operations – Query degradation • Peak Sustainable Throughput – Query overflows – Number of clusters • Total Energy Consumption – Query processing operations • Total Energy Cost (electric bill) – Energy price
  • 8. Space/temporal variation in query workload - 8 -
  • 9. Space/Temporal variation in energy price Spatial Temporal - 9 -
  • 10. • To decrease the total energy cost, exploiting - Spatio-temporal variation in energy prices - Spatio-temporal variation in query workloads - 10 - • Constraints: – Limited hardware – Bounded response times Goal
  • 11. - 11 - Query forwarding
  • 12. - 12 - Result Query forwarding
  • 13. - 13 - Result Query forwarding
  • 14. 0 Timeline - 14 - Problem definition
  • 15. - 15 - Problem definition Minimize Financial Cost Response Time Constraint Workload Constraint
  • 16. Query degradation rate Response Time Constraint - 16 - Query Degradation Rate
  • 17. - 17 - Query overflow rate Workload Constraint Query Overflow Rate Realized workload =
  • 18. Online Workload shifting algorithm - 18 - Remote Centers ? Local Center User query
  • 19. Workload estimation (no forwarding) 0 Timeline Current time 0 Timeline Current time 0 Timeline Current time - 19 -
  • 20. - 20 -  Estimates future workloads  Current energy prices  Capacities  Shifts workload evenly assuming every expensive data-center will forward queries evenly  Probability is the ratio of the forwarding rate to the estimated workload  Sort of conservative Generating probabilities
  • 21. - 21 - Generating probabilities  Estimates future workloads  Current energy prices  Capacities  Shifts workload evenly assuming every expensive data-center will forward queries evenly  Probability is the ratio of the forwarding rate to the estimated workload  Sort of conservative
  • 22. - 22 - Generating probabilities  Estimates future workloads  Current energy prices  Capacities  Shifts workload evenly assuming every expensive data-center will forward queries evenly  Probability is the ratio of the forwarding rate to the estimated workload  Sort of conservative
  • 23. - 23 - A B C D Remote Centers Picking a data-center Local Center User query w.p. pA(A)
  • 24. Query Workload Price Configuration - 24 - U T S ST Energy price configuration
  • 25. - 25 - Set-up • 5 Datacenters • 38M queries, over 4 days • Turn caching on (lowers PST of the back-end) • Tuned capacities to a low query overflow rate (<0.005)
  • 26. - 26 - Results (I) • 63% of the queries served by the cache – Reduces the PST of the backend • Average query response time increases (from 66ms to around 100ms) • Query degradation rate increases but is kept <5% for a budget of 400ms (none if budget > 800ms) • Overflow rate is the same as in the non-forwarding scenario • Forwarding rate is proportional to price variation • Despite network latencies forwarding is possible
  • 27. Aggregate hourly query forwarding rate - 27 -  Global Time  Forwarding depends on ordinal ranking of current prices  In spatial it dominates price ordinal ranking more than intra-day variations (S correlates with ST)  Traffic and forwarding inverse relation  Takes into account energy prices
  • 28. - 28 -  Relative to U setup  Increasing saving with larger response time limit  ST > S > T (r = ∞)  About 35% saving at ST Savings in electric cost
  • 29. Savings vs electric price vs forwarding rate - 29 -  Local Time (where the query is processed)  T setup, correlates with temporal effects  r = 800 (~ ∞)  Query forwarding rate: positive correlation  Electric price: negative correlation  What happens at 17:00?
  • 30. - 30 - Conclusions • Presented the reduction of the electric bill in distributed search engines as an online optimization framework • A practical algorithm, based on shifting query workloads • Evaluation of potential savings via realistic simulations • Depending on electric price distribution, energy costs can be significantly reduced by shifting query workloads to energy-cheap data centers. – By maintaining the overflow rate equal to that of a centralized engine – By keeping the query degradation rate < 5% • The higher the variability in the configuration the higher the savings
  • 31. - 31 - Future work • Price-aware crawling • Price-aware indexer • Energy-aware caches • Green search engines!
  • 32. Reducing the financial cost per query - 32 -
  • 33. - 33 -  No bound on response time, r = ∞  Almost all can be answered under 800ms, 5% under 400ms  U does not forward  T forwards more Query forwarding vs budget
  • 34. - 34 - PST (for each datacenter) Window Size (for each price configuration) PST and window size
  • 35. Query processing time estimation - 35 - Query q = {t1, t2, t3} LIST 1 LIST2 LIST3 t1 t2 t3 results = {r11,r15, r22, r26 r31} #operations ~ c + w(|LIST1| + |LIST2| + |LIST3|)

Editor's Notes

  • #7: Each center keeps a replica of its index