Distributed systems might be complicated…luckily, the concept is easy to understand!

A distributed system is simply any environment where multiple computers or devices are
working on a variety of tasks and components, all spread across a network. Components within
distributed systems split up the work, coordinating efforts to complete a given job more
efficiently than if only a single device ran it.

It makes sense that we’re seeing more and more distributed systems: the internet enables all of
us to work remotely, and many compute jobs today are too complex for a single computer to
handle them solo. This is the huge advantage — working efficiently, across geographies and
teams. We wouldn’t be able to do most of this without distributed systems.

In this article, we’ll explore the operation of such systems, the challenges and risks of these
platforms, and the myriad benefits of distributed computing.

What are distributed systems?

Historically, distributed computing was expensive, complex to configure and difficult to manage.
Thanks to SaaS solutions, however, distributed computing has become more streamlined and
affordable for businesses of all stripes and sizes.

Today, all types of computing jobs — from database management to video games — use
distributed computing. In fact, many types of software, such as cryptocurrency systems,
scientific simulations, blockchain technologies and AI platforms, wouldn’t be possible at all
without these platforms.

Distributed systems are used when a workload is too great for a single computer or device to
handle. Distributed systems are essential in situations when the workload is subject to change,
such as e-commerce traffic on Cyber Monday or lots of web traffic in response to news about
your organization.
Because they draw on the capabilities of other computing devices and processes, distributed
systems can offer features that would be difficult or impossible to develop on a single system.

This includes things like performing an off-site server and application backup — if the master
catalog doesn’t see the segment bits it needs for a restore, it can ask the other off-site node or
nodes to send the segments. Virtually everything you do now with a computing device takes
advantage of the power of distributed systems, whether that’s sending an email, playing a game
or reading this article on the web.

Here are some very common examples of distributed systems:

Telecommunications networks that support mobile and internet networks

Graphical and video-rendering systems

Scientific computing, such as protein folding and genetic research

Airline and hotel reservation systems

Multiuser video conferencing systems

Cryptocurrency processing systems (e.g. Bitcoin)

Peer-to-peer file-sharing systems

Distributed community compute systems

Multiplayer video games

Global, distributed retailers and supply chain management

Examples of distributed systems at work

A distributed system begins with a task. Let’s pretend you need to render a video to create a
finished product.

The application, or distributed applications, managing this task — like a video editor on a client
computer — splits the job into pieces. In this simple example, the algorithm gives one frame of
the video to each of a dozen different computers (or nodes) to complete the rendering. Once the
frame is complete, the managing application gives the node a new frame to work on. This
process continues until the video is finished and all the pieces are put back together.

A system like this doesn’t have to stop at just 12 nodes: the job may be distributed among
hundreds or thousands of nodes, turning a task that might have taken days for a single
computer to complete into one that is finished in a matter of minutes.

When thinking about the challenges of a distributed computing platform, the trick is to break it
down into a series of interconnected patterns. Simplifying the system into smaller, more
manageable and more easily understood components helps abstract a complicated
architecture. Patterns are commonly used to describe distributed systems, such as:

Command and query responsibility segregation (CQRS)

Two-phase commit (2PC)

Different combinations of patterns are used to design distributed systems, and each approach
has unique benefits and drawbacks.

Types of distributed systems

There are many models and architectures of distributed systems in use today.

Client-server systems, the most traditional and simple type of distributed system, involve a
multitude of networked computers that interact with a central server for data storage,
processing or other common goal.

Peer-to-peer networks distribute workloads among hundreds or thousands of computers all

running the same software.

Cell phone networks are an advanced distributed system, sharing workloads among handsets,
switching systems and internet-based devices.

At this point you might realize this: The most common forms of distributed systems today
operate over the internet, handing off workloads to dozens of cloud-based virtual server
instances that are created as needed, then terminated when the task is complete.

Key characteristics of a distributed system

So now that we “get” what distributed systems are, we can start to assign key features to them.
Here’s what good distributed systems have in common:

Scalability. The ability to grow as the size of the workload increases is an essential feature of
distributed systems, accomplished by adding additional processing units or nodes to the
network as needed.

Concurrency. Distributed system components run simultaneously. They’re also characterized by

the lack of a “global clock,” when tasks occur out of sequence and at different rates.

Availability and fault tolerance. If one node fails, the remaining nodes can continue to operate
without disrupting the overall computation.

Heterogeneity. In most distributed systems, the nodes and components are often asynchronous,
with different hardware, middleware, software and operating systems. This allows the
distributed systems to be extended with the addition of new components.

Replication. Distributed systems enable shared information and messaging, ensuring

consistency between redundant resources, such as software or hardware components,
improving fault tolerance, reliability and accessibility.

Transparency. The end user sees a distributed system as a single computational unit rather
than as its underlying parts, allowing users to interact with a single logical device rather than
being concerned with the system’s architecture.

Benefits, challenges & risks of distributed systems

Distributed systems offer a number of advantages over monolithic, or single, systems:

Scalability & flexibility. It is easier to add computing power as the need for services grows. In
most cases today, you can spin up servers to a distributed system on the fly, increasing
performance and further reducing time to completion.

Fault tolerance. Distributed systems reduce the risks involved with having a single point of
failure, bolstering reliability and fault tolerance.

Reliability: A well-designed distributed system can withstand failures in one or more of its nodes
without severely impacting performance. In a monolithic system, the entire application goes
down if the server goes down.

Speed. Heavy traffic can bog down single servers when traffic gets heavy, impacting
performance for everyone. The scalability of distributed databases and other distributed
systems makes them easier to maintain and also sustain high-performance levels.

Geo-distribution. Distributed content delivery is both intuitive for any internet user, and vital for
global organizations.

(Know the differences between CDNs & load balancers.)

Distributed systems are considerably more complex than monolithic computing environments,
and raise a number of challenges around design, operations and maintenance. These include:

Increased opportunities for failure: The more systems added to a computing environment, the
more opportunity there is for failure. If a system is not carefully designed and a single node
crashes, the entire system can go down. While distributed systems are designed to be fault
tolerant, that fault tolerance isn’t automatic or foolproof.

Synchronization process challenges: Distributed systems work without a global clock, requiring
careful programming to ensure that processes are properly synchronized to avoid transmission
delays that result in errors and data corruption. In a complex system — such as a multiplayer
video game — synchronization can be challenging, especially on a public network that carries
data traffic.
Imperfect scalability: Doubling the number of nodes in a distributed system doesn’t necessarily
double performance. Architecting an effective distributed system that maximizes scalability is a
complex undertaking that needs to take into account load balancing, bandwidth management
and other issues.

More complex security: Managing a large number of nodes in a heterogeneous or globally

distributed environment creates numerous security challenges. A single weak link in a file
system or larger distributed system network can expose the entire system to attack.

Increased complexity: Distributed systems are more complex to design, manage and
understand than traditional computing environments.

The challenges of distributed systems create a number of correlating risks.

Security. Distributed systems are as vulnerable to attack as any other system, but their
distributed nature creates a much larger attack surface that exposes organizations to threats.

Risk of network failure. Distributed systems are beholden to public networks to transmit and
receive data. If one segment of the internet becomes unavailable or overloaded, distributed
system performance may decline.

Governance and control issues. Distributed systems lack the governability of monolithic, single-
server-based systems, creating auditing and adherence issues around data privacy laws.
Globally distributed environments are challenging when it comes to providing certain levels of
assurance and understanding exactly where data resides.

Cost control. Unlike centralized systems, the scalability of distributed systems allows
administrators to easily add additional capacity as needed, which can also increase costs.
Pricing for cloud-based distributed computing systems are based on usage (such as the
number of memory resources and CPU power consumed over time). If demand suddenly spikes,
you might face a massive bill.

(Check out the latest cloud cost trends.)

Real world guidance: How to set up a distributed system

Distributed deployments can range from tiny, single department deployments on local area
networks to large-scale, global deployments. In addition to their size and overall complexity,
organizations can consider deployments based on:

The size and capacity of their computer network

The amount of data they’ll consume

How frequently they run processes and whether they'll be scheduled or ad hoc

The number of users accessing the system

Capacity of their data center

The necessary data fidelity and availability requirements

Distributed deployments are categorized as departmental, small enterprise, medium enterprise

or large enterprise. By no means formal, these categories are a starting point for planning the
needed resources to implement a distributed computing system.

Distributed systems can also evolve over time, transitioning from departmental to small
enterprise as the enterprise grows and expands.

Tracking what goes on in distributed systems

We know clearly that, for all their benefits, distributed systems are complicated. Knowing what
goes on within — the observability of that system — is a distinct advantage. Luckily, it’s one you
can achieve with distributed tracing.

Without distributed tracing, a globally distributed system environment would be impossible to

monitor effectively.

Distributed tracing, sometimes called distributed request tracing, is a method for monitoring
applications — typically those built on a microservices architecture — which are commonly
deployed on distributed systems. Distributed tracing is essentially a form of distributed
computing in that it’s commonly used to monitor the operations of applications running on
distributed systems.

In software development and operations, tracing is used to follow the course of a transaction as
it travels through an application. An online credit card transaction as it winds its way from a
customer’s initial purchase to the verification and approval process to the completion of the
transaction, for example. A tracing system monitors this process step by step, helping a
developer to uncover bugs, bottlenecks, latency or other problems with the application.

Distributed tracing is necessary because of the considerable complexity of modern software

architectures. A distributed tracing system is designed to operate on a distributed services
infrastructure, where it can track multiple applications and processes simultaneously across
numerous concurrent nodes and computing environments.

(Learn about best practices for distributed tracing.)

Applying access control in distributed systems

Administrators use a variety of approaches to manage access control in distributed computing

environments, ranging from traditional access control lists (ACLs) to role-based access control
(RBAC).

One of the most promising access control mechanisms for distributed systems is attribute-
based access control (ABAC), which controls access to objects and processes using rules that
include information about the user, the action requested and the environment of that request.
Administrators can also refine these types of roles to restrict access to certain times of day or
certain locations.

Distributed systems aren’t going away

Distributed systems are well-positioned to dominate computing as we know it for the

foreseeable future, and almost any type of application or service will incorporate some form of
distributed computing. The need for always-on, available-anywhere computing isn’t
disappearing anytime soon.
This posting does not necessarily represent Splunk's position, strategies or opinion.

Chrissy Kidd Picture