Efficient Primary-Backup replication on top of consensus
Download as pptx, pdf1 like607 views
This document discusses efficient passive replication techniques using consensus algorithms like Paxos. It presents two options: 1) using standard consensus with sequential instances, or 2) modifying Paxos to allow parallel instances when the primary fails. The key contribution is realizing option 1 is sufficient by having the new primary pipeline consensus instances, avoiding blocking or extra state transfers. This achieves the same stable performance as systems like Zab that modify consensus, while keeping the consensus algorithm as a black box.
1 of 9
Download to read offline
Ad
Recommended
Prometheus 2.0
Prometheus 2.0Kausal Prometheus 2.0 provides significantly improved performance for highly dynamic environments like Kubernetes with a brand new storage engine. It offers around 3x reduction in CPU usage, 2x less disk space usage, and 100x reduction in I/O compared to previous versions. Additionally, Prometheus 2.0 handles target staleness differently by stopping returning values after a single scrape interval if a target goes down, rather than continuing for 5 minutes, making it simpler to understand.
Social Piggybacking: Leveraging Common Friends to Generate Event Streams
Social Piggybacking: Leveraging Common Friends to Generate Event StreamsMarco Serafini This document describes optimizations for generating social event streams. It proposes using materialized views containing a user's events and the events of other users they follow. It evaluates different approaches like push-all, pull-all, and hybrid to optimize throughput. It then introduces the idea of social piggybacking, where two connected users likely share common friends, allowing their views to be used as hubs to prune edges. An algorithm called Nosy is presented as a heuristic to find hub graphs and determine when to use pushing or piggybacking based on production and consumption rates to minimize costs. Experimental results on Twitter and Flickr graphs show Nosy can improve throughput over baseline hybrid optimizations by up to 2.4 times by exploiting community structure
Réplication de base de données oracle avec Golden Gate
Réplication de base de données oracle avec Golden GateMor THIAM Le document traite de la réplication de bases de données Oracle, en définissant les concepts clés, les types de réplication (basique et avancée), ainsi que les avantages et inconvénients associés. Il présente également une comparaison des outils de réplication et les étapes d'installation et de test de Goldengate. L'objectif principal est d'assurer la haute disponibilité et la cohérence des données à travers différents serveurs.
Réplication des bases de données
Réplication des bases de donnéessie92 La réplication des bases de données est une solution destinée à améliorer la disponibilité, les performances et la tolérance aux pannes des systèmes de gestion de bases de données. Elle se décline en plusieurs types, tels que la réplication synchrone et asynchrone, chaque méthode ayant ses avantages et inconvénients en matière de gestion des données. Ce document traite des principes, des objectifs, des différents types de réplication et de leur mise en œuvre dans divers scénarios.
Introduction au datamining, concepts et techniques
Introduction au datamining, concepts et techniquesIsmail CHAIB Le document présente une introduction au datamining, mettant en avant ses concepts, techniques et applications dans divers domaines tels que les télécommunications et la segmentation des clients. Il aborde également le processus itératif du datamining, les étapes de collecte et de prétraitement des données, ainsi que les différentes méthodes d'estimation et d'interprétation des modèles. Enfin, il souligne les perspectives d'évolution du datamining et les défis associés, notamment en ce qui concerne la gestion des données.
Vertex cover Problem
Vertex cover ProblemGajanand Sharma The document discusses the vertex cover problem in graph theory, which involves finding the minimum size vertex cover that covers every edge of a graph. It outlines approximation algorithms, specifically greedy algorithms, highlighting their limitations in always securing optimal solutions. The goal of such algorithms is to yield feasible solutions that are close to optimal, despite not guaranteeing the best result.
Np complete
Np completeDr. C.V. Suresh Babu This document discusses NP-complete problems and their properties. Some key points:
- NP-complete problems have an exponential upper bound on runtime but only a polynomial lower bound, making them appear intractable. However, their intractability cannot be proven.
- NP-complete problems are reducible to each other in polynomial time. Solving one would solve all NP-complete problems.
- NP refers to problems that can be verified in polynomial time. P refers to problems that can be solved in polynomial time.
- A problem is NP-complete if it is in NP and all other NP problems can be reduced to it in polynomial time. Proving a problem is NP-complete involves showing
the Paxos Commit algorithm
the Paxos Commit algorithmpaolos84 The document provides a detailed overview of the Paxos Commit Algorithm, highlighting its functioning, participants, and optimizations. It contrasts Paxos with the two-phase commit mechanism, emphasizing its fault tolerance and performance advantages in distributed transaction commits managed by resource managers. Additionally, the text discusses failure scenarios, leader coordination, and the significance of acceptors in achieving consensus among a majority for successful transaction commitments.
Basic Paxos Implementation in Orc
Basic Paxos Implementation in OrcHemanth Kumar Mantri This document summarizes the implementation of the classic Paxos consensus algorithm in Orc. It describes the key aspects of Paxos including phases 1 and 2, properties like uniqueness of proposal numbers, and how consensus is achieved even with failures of proposers or acceptors. It also discusses optimizations that can be made for efficiency but are not required for correctness.
Papers We Love / Kyiv : PAXOS (and little about other consensuses )
Papers We Love / Kyiv : PAXOS (and little about other consensuses )Ruslan Shevchenko The document discusses Paxos, a consensus algorithm, along with a brief overview of its development and related works on distributed systems and fault tolerance. It includes references to various academic papers and explains key components of the algorithm such as roles, messages, safety, and progress, as well as related concepts like multi-Paxos and Byzantine fault tolerance (BFT). Additionally, it touches on alternative algorithms like Raft, highlighting their differences and similarities to Paxos.
Reaching reliable agreement in an unreliable world
Reaching reliable agreement in an unreliable worldHeidi Howard The document discusses the challenges of reaching consensus in distributed systems, highlighting protocols like Paxos and Raft as solutions. It examines the trade-offs between consistency, availability, and partition tolerance as per the CAP theorem, and presents case studies on Raft and Tango, emphasizing scalability and fault tolerance. The conclusion suggests that while consensus is achievable, there is still room for improvement in distributed system protocols.
genpaxospublic-090703114743-phpapp01.pdf
genpaxospublic-090703114743-phpapp01.pdfHiroshi Ono This document summarizes a presentation on Paxos and gen_paxos. It introduces Paxos as a distributed consensus algorithm that is robust to network failures and allows data replication across multiple nodes. It then describes the gen_paxos Erlang implementation of Paxos, including its data model, state machine approach, and messaging between nodes. Key aspects of Paxos like the prepare and propose phases are explained through examples. The document also provides context on applications of Paxos and references for further reading.
genpaxospublic-090703114743-phpapp01.pdf
genpaxospublic-090703114743-phpapp01.pdfHiroshi Ono This document summarizes a presentation on the PAXOS consensus algorithm and the gen_paxos Erlang implementation. The presentation introduces PAXOS as a solution to the split brain problem in distributed systems and discusses its key aspects, including phases, data model, and state machine implementation in gen_paxos. It also covers why Erlang is suitable, references related work, and includes an appendix on bugs and FAQs.
genpaxospublic-090703114743-phpapp01.pdf
genpaxospublic-090703114743-phpapp01.pdfHiroshi Ono This document summarizes an Erlang meeting held on July 3, 2009 in Tokyo. It discusses the gen_paxos Erlang module, which implements the Paxos consensus algorithm. Paxos is needed to solve problems like split-brains where data could become inconsistent without coordination between nodes. The document explains the key aspects of Paxos like its phases, data model in gen_paxos, and how nodes communicate through message passing in Erlang. It also provides references to related works and papers about Paxos.
genpaxospublic-090703114743-phpapp01.pdf
genpaxospublic-090703114743-phpapp01.pdfHiroshi Ono This document summarizes an Erlang meeting held on July 3, 2009 in Tokyo. It discusses the gen_paxos Erlang module, which implements the Paxos consensus algorithm. Paxos is explained as solving the split brain problem in distributed systems through coordination between nodes. The implementation details in gen_paxos are covered, including how it uses finite state machines. The document also discusses where Paxos gets its name and includes references for further reading.
Distributed Consensus: Making the Impossible Possible
Distributed Consensus: Making the Impossible PossibleC4Media The document outlines the evolution of distributed consensus in computer systems, highlighting key algorithms such as Paxos, Raft, and Fast Paxos, and addressing the challenges of achieving consensus in asynchronous, unreliable environments. It discusses historical results on consensus impossibility, various replication techniques, and future directions for algorithm development. The aim is to encourage system designers to find suitable consensus algorithms for their specific applications despite theoretical difficulties.
Cornelia Davis, Meaghan Kjelland, Erin Schnabel, Therese Stowell, and Mathang...
Cornelia Davis, Meaghan Kjelland, Erin Schnabel, Therese Stowell, and Mathang...VMware Tanzu The document discusses the complexities of distributed systems, focusing on leader election, the Paxos consensus protocol, and the CAP theorem, which highlights the trade-offs between consistency, availability, and partition tolerance. It explains how Paxos can help achieve consensus in distributed systems while addressing challenges like failures and isolation. The document also touches on Google's Spanner and its approach to achieving global consistency and availability using TrueTime technology.
Distributed Consensus: Making Impossible Possible [Revised]
Distributed Consensus: Making Impossible Possible [Revised]Heidi Howard The document discusses the evolution of distributed consensus mechanisms over three decades, covering foundational theories such as the FLP impossibility result and various algorithms, including Paxos and Raft. It emphasizes the importance of achieving consensus in unreliable systems while balancing factors such as consistency, availability, and partition tolerance, encapsulated in the CAP theorem. Additionally, it highlights practical implementations and variations of consensus algorithms, stressing the importance of selecting the right algorithm for specific applications.
Distributed Consensus: Making Impossible Possible
Distributed Consensus: Making Impossible PossibleHeidi Howard Distributed consensus algorithms allow different computer systems to agree on data values in an unreliable distributed network. The document summarizes the history and developments of consensus algorithms over three decades, including early impossibility results, Paxos, Multi-Paxos, Fast Paxos, Egalitarian Paxos, and Raft. It also discusses challenges in implementing consensus algorithms and potential future directions like Ios and Hydra that improve throughput and allow consensus across geographically distributed data centers.
Distributed Consensus: Making Impossible Possible by Heidi howard
Distributed Consensus: Making Impossible Possible by Heidi howardJ On The Beach The document discusses the evolution of distributed consensus over three decades, addressing critical questions about how consensus can be achieved among unreliable machines in asynchronous networks. It outlines major concepts such as the FLP impossibility result, various consensus algorithms like Paxos and Raft, and their implications for practical applications. The exploration emphasizes the ongoing relevance and potential for advancing consensus methods despite theoretical limitations.
Paxos building-reliable-system
Paxos building-reliable-systemYanpo Zhang The document discusses the Paxos algorithm as a framework for achieving consensus in distributed systems, focusing on its necessity for data durability and availability. It outlines various replication techniques and the core phases of the Paxos algorithm, explaining how it manages to store values reliably by requiring a quorum of acceptors. Additionally, it describes optimizations such as Multi-Paxos and Fast Paxos to reduce latency while maintaining consensus across distributed nodes.
The paxos commit algorithm
The paxos commit algorithmahmed hamza The document summarizes the Paxos Commit algorithm, which uses the Paxos consensus algorithm to achieve fault-tolerant distributed transaction commit. It describes the key participants in Paxos Commit - the resource managers, leader, and acceptors. The base version of Paxos Commit is explained along with optimizations. Performance is analyzed in comparison to two-phase commit. Failure scenarios are discussed, showing how Paxos Commit can handle leader failure or missing resource managers. The relationship between Paxos Commit and two-phase commit is also clarified.
Paxos vs Raft Have we reached consensus on distributed consensus.pptx
Paxos vs Raft Have we reached consensus on distributed consensus.pptxmahdiaghaei19 Paxos vs Raft Have we reached consensus on distributed consensus paper
Distributed Systems Theory for Mere Mortals
Distributed Systems Theory for Mere MortalsEnsar Basri Kahveci The presentation discusses distributed systems theory, emphasizing their complexity and the challenges in achieving consensus among processes that might fail independently. Key topics include system models, consensus algorithms like Paxos and Raft, and issues related to time, order, and failure detection in distributed environments. The document highlights the importance of understanding the properties of distributed systems and the various algorithms designed to address their inherent challenges.
9X5u87KWa267pP7aGX3K
9X5u87KWa267pP7aGX3KCapitolPunishment This document discusses patterns in distributed computing. It outlines some of the key hurdles in distributed systems like asynchrony, locality, failure, and Byzantine faults. It then summarizes common distributed algorithms like leader election, group membership, and consensus. It presents two Ruby libraries, TokenWorker and StaticQueueWorker, that implement leader election and work distribution patterns. It also briefly outlines the Paxos consensus algorithm but notes that a full implementation is very challenging. The document advocates for practical approaches over purely formal solutions and emphasizes starting with reasonable reliability requirements.
Distributed System by Pratik Tambekar
Distributed System by Pratik TambekarPratik Tambekar The document summarizes concepts related to fault tolerance including:
- Dependability includes availability, reliability, safety and maintainability.
- Different types of failures like crash, omission, timing and arbitrary failures.
- Redundancy can be used to mask failures through techniques like triple modular redundancy.
- Agreement in faulty systems looks at problems like the Byzantine generals problem.
- Reliable multicasting schemes use techniques such as hierarchical feedback control and virtual synchrony.
- Commit protocols like two-phase commit and three-phase commit coordinate transactions between processes.
- Recovery techniques involve stable storage, checkpointing to avoid domino effect, and message logging to prevent orphans.
Checkpointing with Synchronized Clocks in Distributed Systems
Checkpointing with Synchronized Clocks in Distributed Systemsijujournal ABSTRACT
The processes of the distributed system considered in this paper use loosely synchronized clocks. The paper describes a method of taking checkpoints by such processes in a truly distributed manner, that is, in the absence of a global checkpoint coordinator. The constituent processes take checkpoints according to their
own clocks at predetermined checkpoint instants. A global consistent set of such asynchronous checkpoints needs to be formed to avoid the domino effect. This is achieved by adding suitable information to the existing clock synchronization messages looking at which the processes synchronize their checkpoints to form a global consistent checkpoint. Communication in this system is synchronous, so, processes may be blocked for communication at checkpointing instants. The blocked processes save the state they were in just before being blocked. It is shown here that the set of such i-th checkpoints is consistent and hence the rollback required by the system in case of failure is only up to the last saved state.
KEYWORDS
Fault tolerance, Checkpointing, Rollback recovery, Synchronized clock,Clock synchronization message
PDF LINK: https://www.airccse.org/journal/iju/papers/0410iju5.pdf
VOLUME LINK: https://www.airccse.org/journal/iju/vol1.html
MORE INFO: https://www.airccse.org/journal/iju/index.html
Consensus in distributed computing
Consensus in distributed computingRuben Tan The document provides a comprehensive overview of consensus in distributed computing, highlighting key concepts such as consensus protocols, failure modes (including Byzantine failures), and the significance of the Byzantine General's problem. It discusses various consensus methods like the 2-phase commit and Paxos, explaining their workings and challenges, particularly in achieving safety and liveness. Lastly, it emphasizes the ongoing challenges in developing Byzantine fault tolerant consensus systems and mentions practical applications of these concepts in modern distributed systems.
More Related Content
Similar to Efficient Primary-Backup replication on top of consensus (20)
Basic Paxos Implementation in Orc
Basic Paxos Implementation in OrcHemanth Kumar Mantri This document summarizes the implementation of the classic Paxos consensus algorithm in Orc. It describes the key aspects of Paxos including phases 1 and 2, properties like uniqueness of proposal numbers, and how consensus is achieved even with failures of proposers or acceptors. It also discusses optimizations that can be made for efficiency but are not required for correctness.
Papers We Love / Kyiv : PAXOS (and little about other consensuses )
Papers We Love / Kyiv : PAXOS (and little about other consensuses )Ruslan Shevchenko The document discusses Paxos, a consensus algorithm, along with a brief overview of its development and related works on distributed systems and fault tolerance. It includes references to various academic papers and explains key components of the algorithm such as roles, messages, safety, and progress, as well as related concepts like multi-Paxos and Byzantine fault tolerance (BFT). Additionally, it touches on alternative algorithms like Raft, highlighting their differences and similarities to Paxos.
Reaching reliable agreement in an unreliable world
Reaching reliable agreement in an unreliable worldHeidi Howard The document discusses the challenges of reaching consensus in distributed systems, highlighting protocols like Paxos and Raft as solutions. It examines the trade-offs between consistency, availability, and partition tolerance as per the CAP theorem, and presents case studies on Raft and Tango, emphasizing scalability and fault tolerance. The conclusion suggests that while consensus is achievable, there is still room for improvement in distributed system protocols.
genpaxospublic-090703114743-phpapp01.pdf
genpaxospublic-090703114743-phpapp01.pdfHiroshi Ono This document summarizes a presentation on Paxos and gen_paxos. It introduces Paxos as a distributed consensus algorithm that is robust to network failures and allows data replication across multiple nodes. It then describes the gen_paxos Erlang implementation of Paxos, including its data model, state machine approach, and messaging between nodes. Key aspects of Paxos like the prepare and propose phases are explained through examples. The document also provides context on applications of Paxos and references for further reading.
genpaxospublic-090703114743-phpapp01.pdf
genpaxospublic-090703114743-phpapp01.pdfHiroshi Ono This document summarizes a presentation on the PAXOS consensus algorithm and the gen_paxos Erlang implementation. The presentation introduces PAXOS as a solution to the split brain problem in distributed systems and discusses its key aspects, including phases, data model, and state machine implementation in gen_paxos. It also covers why Erlang is suitable, references related work, and includes an appendix on bugs and FAQs.
genpaxospublic-090703114743-phpapp01.pdf
genpaxospublic-090703114743-phpapp01.pdfHiroshi Ono This document summarizes an Erlang meeting held on July 3, 2009 in Tokyo. It discusses the gen_paxos Erlang module, which implements the Paxos consensus algorithm. Paxos is needed to solve problems like split-brains where data could become inconsistent without coordination between nodes. The document explains the key aspects of Paxos like its phases, data model in gen_paxos, and how nodes communicate through message passing in Erlang. It also provides references to related works and papers about Paxos.
genpaxospublic-090703114743-phpapp01.pdf
genpaxospublic-090703114743-phpapp01.pdfHiroshi Ono This document summarizes an Erlang meeting held on July 3, 2009 in Tokyo. It discusses the gen_paxos Erlang module, which implements the Paxos consensus algorithm. Paxos is explained as solving the split brain problem in distributed systems through coordination between nodes. The implementation details in gen_paxos are covered, including how it uses finite state machines. The document also discusses where Paxos gets its name and includes references for further reading.
Distributed Consensus: Making the Impossible Possible
Distributed Consensus: Making the Impossible PossibleC4Media The document outlines the evolution of distributed consensus in computer systems, highlighting key algorithms such as Paxos, Raft, and Fast Paxos, and addressing the challenges of achieving consensus in asynchronous, unreliable environments. It discusses historical results on consensus impossibility, various replication techniques, and future directions for algorithm development. The aim is to encourage system designers to find suitable consensus algorithms for their specific applications despite theoretical difficulties.
Cornelia Davis, Meaghan Kjelland, Erin Schnabel, Therese Stowell, and Mathang...
Cornelia Davis, Meaghan Kjelland, Erin Schnabel, Therese Stowell, and Mathang...VMware Tanzu The document discusses the complexities of distributed systems, focusing on leader election, the Paxos consensus protocol, and the CAP theorem, which highlights the trade-offs between consistency, availability, and partition tolerance. It explains how Paxos can help achieve consensus in distributed systems while addressing challenges like failures and isolation. The document also touches on Google's Spanner and its approach to achieving global consistency and availability using TrueTime technology.
Distributed Consensus: Making Impossible Possible [Revised]
Distributed Consensus: Making Impossible Possible [Revised]Heidi Howard The document discusses the evolution of distributed consensus mechanisms over three decades, covering foundational theories such as the FLP impossibility result and various algorithms, including Paxos and Raft. It emphasizes the importance of achieving consensus in unreliable systems while balancing factors such as consistency, availability, and partition tolerance, encapsulated in the CAP theorem. Additionally, it highlights practical implementations and variations of consensus algorithms, stressing the importance of selecting the right algorithm for specific applications.
Distributed Consensus: Making Impossible Possible
Distributed Consensus: Making Impossible PossibleHeidi Howard Distributed consensus algorithms allow different computer systems to agree on data values in an unreliable distributed network. The document summarizes the history and developments of consensus algorithms over three decades, including early impossibility results, Paxos, Multi-Paxos, Fast Paxos, Egalitarian Paxos, and Raft. It also discusses challenges in implementing consensus algorithms and potential future directions like Ios and Hydra that improve throughput and allow consensus across geographically distributed data centers.
Distributed Consensus: Making Impossible Possible by Heidi howard
Distributed Consensus: Making Impossible Possible by Heidi howardJ On The Beach The document discusses the evolution of distributed consensus over three decades, addressing critical questions about how consensus can be achieved among unreliable machines in asynchronous networks. It outlines major concepts such as the FLP impossibility result, various consensus algorithms like Paxos and Raft, and their implications for practical applications. The exploration emphasizes the ongoing relevance and potential for advancing consensus methods despite theoretical limitations.
Paxos building-reliable-system
Paxos building-reliable-systemYanpo Zhang The document discusses the Paxos algorithm as a framework for achieving consensus in distributed systems, focusing on its necessity for data durability and availability. It outlines various replication techniques and the core phases of the Paxos algorithm, explaining how it manages to store values reliably by requiring a quorum of acceptors. Additionally, it describes optimizations such as Multi-Paxos and Fast Paxos to reduce latency while maintaining consensus across distributed nodes.
The paxos commit algorithm
The paxos commit algorithmahmed hamza The document summarizes the Paxos Commit algorithm, which uses the Paxos consensus algorithm to achieve fault-tolerant distributed transaction commit. It describes the key participants in Paxos Commit - the resource managers, leader, and acceptors. The base version of Paxos Commit is explained along with optimizations. Performance is analyzed in comparison to two-phase commit. Failure scenarios are discussed, showing how Paxos Commit can handle leader failure or missing resource managers. The relationship between Paxos Commit and two-phase commit is also clarified.
Paxos vs Raft Have we reached consensus on distributed consensus.pptx
Paxos vs Raft Have we reached consensus on distributed consensus.pptxmahdiaghaei19 Paxos vs Raft Have we reached consensus on distributed consensus paper
Distributed Systems Theory for Mere Mortals
Distributed Systems Theory for Mere MortalsEnsar Basri Kahveci The presentation discusses distributed systems theory, emphasizing their complexity and the challenges in achieving consensus among processes that might fail independently. Key topics include system models, consensus algorithms like Paxos and Raft, and issues related to time, order, and failure detection in distributed environments. The document highlights the importance of understanding the properties of distributed systems and the various algorithms designed to address their inherent challenges.
9X5u87KWa267pP7aGX3K
9X5u87KWa267pP7aGX3KCapitolPunishment This document discusses patterns in distributed computing. It outlines some of the key hurdles in distributed systems like asynchrony, locality, failure, and Byzantine faults. It then summarizes common distributed algorithms like leader election, group membership, and consensus. It presents two Ruby libraries, TokenWorker and StaticQueueWorker, that implement leader election and work distribution patterns. It also briefly outlines the Paxos consensus algorithm but notes that a full implementation is very challenging. The document advocates for practical approaches over purely formal solutions and emphasizes starting with reasonable reliability requirements.
Distributed System by Pratik Tambekar
Distributed System by Pratik TambekarPratik Tambekar The document summarizes concepts related to fault tolerance including:
- Dependability includes availability, reliability, safety and maintainability.
- Different types of failures like crash, omission, timing and arbitrary failures.
- Redundancy can be used to mask failures through techniques like triple modular redundancy.
- Agreement in faulty systems looks at problems like the Byzantine generals problem.
- Reliable multicasting schemes use techniques such as hierarchical feedback control and virtual synchrony.
- Commit protocols like two-phase commit and three-phase commit coordinate transactions between processes.
- Recovery techniques involve stable storage, checkpointing to avoid domino effect, and message logging to prevent orphans.
Checkpointing with Synchronized Clocks in Distributed Systems
Checkpointing with Synchronized Clocks in Distributed Systemsijujournal ABSTRACT
The processes of the distributed system considered in this paper use loosely synchronized clocks. The paper describes a method of taking checkpoints by such processes in a truly distributed manner, that is, in the absence of a global checkpoint coordinator. The constituent processes take checkpoints according to their
own clocks at predetermined checkpoint instants. A global consistent set of such asynchronous checkpoints needs to be formed to avoid the domino effect. This is achieved by adding suitable information to the existing clock synchronization messages looking at which the processes synchronize their checkpoints to form a global consistent checkpoint. Communication in this system is synchronous, so, processes may be blocked for communication at checkpointing instants. The blocked processes save the state they were in just before being blocked. It is shown here that the set of such i-th checkpoints is consistent and hence the rollback required by the system in case of failure is only up to the last saved state.
KEYWORDS
Fault tolerance, Checkpointing, Rollback recovery, Synchronized clock,Clock synchronization message
PDF LINK: https://www.airccse.org/journal/iju/papers/0410iju5.pdf
VOLUME LINK: https://www.airccse.org/journal/iju/vol1.html
MORE INFO: https://www.airccse.org/journal/iju/index.html
Consensus in distributed computing
Consensus in distributed computingRuben Tan The document provides a comprehensive overview of consensus in distributed computing, highlighting key concepts such as consensus protocols, failure modes (including Byzantine failures), and the significance of the Byzantine General's problem. It discusses various consensus methods like the 2-phase commit and Paxos, explaining their workings and challenges, particularly in achieving safety and liveness. Lastly, it emphasizes the ongoing challenges in developing Byzantine fault tolerant consensus systems and mentions practical applications of these concepts in modern distributed systems.
Efficient Primary-Backup replication on top of consensus
- 1. Consensus and Efficient Passive Replication Marco Serafini joint work with Flavio Junqueira Yahoo! Research Barcelona
- 2. Passive Replication 1) Receive request 4) Reply Primary 2) Speculative execution Produce state update (Δ) Δ PO Atomic Broadcast 3) Broadcast state update Δ Δ Followers
- 3. Option 1: Standard Consensus Sequential consensus instances: PO atomic broadcast decide (v, i) < propose (v’, i+1) Semi-passive replication Consensus interface: Consensus • propose (val, inst) • decide (val, inst) Ω Asynch. links
- 4. Option 2: Modify Paxos PO atomic broadcast Comparison to Paxos • Stable leader: similar to Paxos Zab / • New leader: synch primary and Virtually Synchronous Paxos followers • Extra state transfer steps • Block consensus until state Ω Asynch. links transfer completes 0 Pro: can parallelize instances (like Paxos) 0 Cons: less modular, dedicated solutions
- 5. Option 2: Modify Paxos PO atomic broadcast Comparison to Paxos • Stable leader: similar to Paxos Zab / • New leader: synch primary and Virtually Synchronous Paxos followers Contribution: • Extra state transfer steps • Block consensus until state Use standard consensus Ω Asynch. links transfer completes with parallel(like Paxos) 0 Pro: can parallelize instances instances 0 Cons: less modular, dedicated solutions
- 6. Primary Failure Old Primary 122 123 124 125 126 127 128 Consensus Wi 122 1 2 126 4 3 5 instances n New-Primary 1 2 3 4 3 5 New Primary
- 7. Synch without extra state transfer + Primary Failure No need for blocking consensus = No need for modifying consensus Old Primary 122 123 124 125 126 127 128 Consensus Wi 122 1 2 126 4 3 5 instances n New-Primary 1 2 3 4 3 5 New Primary
- 8. Synch without extra state transfer + Primary Failure No need for blocking consensus = No need for modifying consensus Old Primary 122 123 124 125 126 127 128 Consensus instances Pipelining 3 5 Wi 122 n 1 2 126 4 Same stable performance as Zab New-Primary 1 2 3 4 3 5 New Primary using Paxos as black box
- 9. Questions? Marco Serafini [email protected]