Whitepaper

Making the Most of

Kubernetes with
Cloud Native Networking
An in-depth guide to advanced networking strategies for
microservice architectures at scale
Table of Content

Abstract 03

Kubernetes and the Challenges of Cloud Native 04

Getting Started with Kubernetes Networking 06

The Network Traffic Compass 06

North-South Traffic and Kubernetes 07

Ingress Controllers 08

East-West Traffic and Service Mesh 09

Security and Diagnostics 12

Encryption in All Directions 13

Authentication and Access Control 14

Visibility Into Microservices 14

Day 2 Challenges: Network-Centric Solutions for Common Scenarios 16

Managing Excess Traffic 17

Software Delivery Management 19

Conclusions 21

| 02
Abstract
Containers and cloud native infrastructure are key enablers of IT modernization, and Kubernetes has
emerged as the leading management tool for this new model. As a graduated project under the Cloud
Native Computing Foundation (CNCF), Kubernetes is not only open source software, but it is also a mature
effort that is maintained by a large and growing community of contributors.

Among its demonstrated advantages, Kubernetes can:

• Support the adoption of microservices architectures

• Encourage agile development practices and rapid deployment
• Extend standardization through its APIs, tools, and best practices
And yet, Kubernetes is complex. Organizations that adopt it will need to gain new knowledge and
embrace new practices to unlock its full value. In particular, understanding the vital role that
application-level networking plays in Kubernetes environments is essential to achieving the greatest
success.

This paper aims to aid organizations in their Kubernetes adoption journey by

exploring how to make the most of its capabilities. Specifically, it will:

• Explain the critical role that networking plays in application deployments

• Review key use cases and how Kubernetes can address them
• Discuss common “Day 2” challenges and how networking software, such as Traefik Labs products,
can help address them

| 03
 Kubernetes and the
Challenges of Cloud Native
Kubernetes and the Challenges
of Cloud Native
“Cloud native” has become the catchword for modern IT infrastructure. But what does a cloud native
environment look like? Public cloud providers only offer part of the answer. The reality for most
organizations is that going cloud native involves a variety of technologies and their accompanying
challenges:

• Heterogeneous Platforms. While some organizations may choose to go all-in on one or several
public clouds, the typical case is not so straightforward. It’s likely to include a mix of public, private,
and hybrid clouds, traditional on-premises hosting, and even bare-metal platforms.
• Diverse Applications. Not every application is equally suited to modern development models. Real-
world environments are likely to include a mix of cloud native and traditional applications and data
sources, sometimes even including such legacy technologies as mainframes and EDI.
• Containers. Containerization can ease software deployment by abstracting away the underlying
platforms. But it can also complicate issues, such as security and management. Not all software
development teams have sufficient expertise to deliver containerized applications effectively.

Microservices Architecture

Many organizations view microservices as the preferred model for new applications. Yet composing
applications from collections of network-interconnected services — many of which are instantiated as
ephemeral containers — presents unique challenges.

Kubernetes helps mitigate the complexity of the cloud native model. It abstracts away traditional
infrastructure concerns, above and beyond the container level. It also helps automate the provisioning,
de-provisioning, and management of applications across clusters of resources in consistent and
reproducible ways.

Another important characteristic of each of the aforementioned technologies, however, is that they
increase reliance on networking. Kubernetes provides basic tools for interconnecting assorted services
and components, but external networking software is almost always necessary to complete a production-
grade deployment.

| 05
Getting Started with Kubernetes Networking

Networking for Kubernetes clusters is unlike traditional, server-based networking. Within the cluster, the
control plane creates and destroys containers as load demands, resulting in a dynamic and ever-changing
network environment. Resources and the network routes between them are in constant flux.
For this reason, it behooves new Kubernetes administrators to familiarize themselves with Kubernetes’
approach to cluster networking and some of its related terms and concepts, including Pods, Services,
endpoints, and so on. This paper uses this language throughout, along with terms common to traditional
networking.

The Network Traffic Compass

INTERNET

W E

NODE 1 NODE 2 NODE N

It can help to think of a Kubernetes cluster as its own network microcosm, consisting of multiple nodes and
controllers. Network traffic moves in and around this microcosm along two axes, sometimes visualized in
terms of the four cardinal compass directions:

• North-South Traffic. This flows in and out of the cluster’s internal network, whether it’s from a
private, traditional network, or from the public internet. In Kubernetes environments, components
known as ingress controllers play an important role in managing this traffic.
• East-West Traffic. This flows between Pods and Services within the cluster’s internal network and is
not visible from outside. One emerging option for enhanced management of this traffic is known as a
service mesh.

Most microservices-based environments need to manage both types of traffic.

| 06
North-South Traffic and Kubernetes

Routing requests to applications and services running on traditional networks are relatively
straightforward. On the other hand, Kubernetes complicates this traffic flow in two main ways.

Pods and Dynamic Addressing

The fundamental deployable unit in Kubernetes is the Pod, an abstraction that represents one or more
application containers. Kubernetes automatically assigns IP addresses to Pods when they are created, but
these addresses may change over time based upon various lifecycle operations.

This dynamic address assignment presents a connectivity challenge; if you can’t be sure of the IP address
of any given Pod — or indeed, if the Pod exists at all — the traditional networking toolbox offers no reliable
means to route traffic to that Pod.

Services and Cluster Networking

Kubernetes’s solution to the dynamic address problem is the concept of a Service. Each Service represents
an application running on any number of Pods, and it allows Kubernetes to distribute requests to these
Pods, regardless of how many replicas of the Pods are running. This allows the system to ensure
accessibility to the service, even as Pod replicas are created and destroyed according to demand.

On their own, however, Services don’t provide a particularly effective way for external clients to access
applications running on Kubernetes. The model has a number of drawbacks, including:

• Services can only expose specific ports (ports 30000-32767, by default).

• A given port can only map to a single Service.
• Clients must connect through a cluster node, so if the corresponding IP of that host/VM changes, it
must be updated accordingly.

These handicaps can be particularly problematic for supporting external access to Services that need to
connect through standard ports.

| 07
Ingress Controllers

Kubernetes provides a solution to this dilemma through its native Ingress resource. When configured, the
Ingress resource creates HTTP and HTTPS endpoints and routes traffic based upon rules defined by the
administrator.

The Ingress resource can expose multiple underlying Services through a single external endpoint or load
balancer. Administrators can specify how traffic should be routed to defined Service resources using, for
example, host or prefix rules. Together, Ingress and Service resources provide a flexible mechanism for
enabling external access to applications.

INCOMING
REQUEST

CLUSTER
INGRESS

SERVICE 1 SERVICE N

POD POD POD POD

Note, however, that defining an Ingress resource doesn’t expose services outside of a Kubernetes cluster.
On its own, an Ingress resource simply conveys a request for networking configuration. A separate
component known as an ingress controller is responsible for consuming these requests and creating the
corresponding routes in a technology-specific manner.

There are many potential options available, but a few examples include:

• Traefik Proxy. A leading open source reverse proxy and endpoint router that can function as a
Kubernetes Ingress controller and makes setting up routes between Kubernetes services and the
outside world straightforward and reliable.
• AWS ALB. An example of an ingress controller tied to a specific public cloud; it satisfies inbound
ingress resource requests using AWS Application Load Balancers.
• NGINX. Implements Ingress resources using the popular NGINX open source web server and load
balancer software.
• HAProxy. Implements Ingress resources using the HAProxy open source web server and load
balancer software.

As an additional note, Kubernetes administrators have the flexibility to deploy multiple ingress controllers if
desired, based upon their specific needs and requirements.

| 08
Using External Load Balancers with Ingress Controllers

Operators are typically familiar with deploying external load balancers, either in the cloud or in on-premises
environments, to route traffic to application instances. Kubernetes defines a Service type to automate the
provisioning of external load balancers when running in cloud environments. However, in any real-life
scenario, directly using external load balancers for every application has significant drawbacks, including:

• Cost Overhead. External load balancers, whether cloud-managed or instantiated through physical
network appliances on-premises, can be expensive. For Kubernetes clusters with many externally
exposed applications, these costs will quickly add up.
• Operational Complexity. Load balancers require various resources (IP addresses, DNS, certificates,
etc.) that can be difficult to manage without accompanying automation.
• Monitoring and Logging. Given the importance of external load balancers in the traffic flow, being
able to effectively centralize monitoring and logging data is critical. However, this becomes
challenging when there are multiple load balancers involved.

It is important to understand that the Kubernetes architecture provides an alternative to this traditional
approach. It allows users to combine the functions of load balancers with an ingress controller. Instead of
provisioning a load balancer for every application service that needs external connectivity, users deploy
and configure a single instance that targets an ingress controller. The ingress controller serves as a single
entry point and can route traffic to multiple applications in the cluster. Key points to keep in mind with this
approach include:

• Ingress controllers are nothing more than standard Kubernetes applications, instantiated via Pods.
• Ingress controllers are exposed as Services. The application that constitutes an Ingress Controller is
exposed through the service type that deploys a load balancer.
• As an intermediary between the external load balancer and applications, the Ingress Controller
uses default Kubernetes Service types to route and balance traffic across application instances
based upon operator-provided configurations.

East-West Traffic and the Service Mesh

In a microservices architecture, in addition to north-south (ingress) traffic, most applications will also
generate east-west traffic between services or components within the cluster. In many cases, some of
these components will not be exposed to the external network but must be accessible to the components
that are. One example might be a data store that maintains state between multiple instances of a service.

Often, east-west traffic is handled directly by the services on the cluster through links defined by the
components’ developers. An increasingly popular option, however, is to deploy a service mesh, which is a
dedicated controller and a set of proxy instances that handle east-west communication in a more abstract
and manageable way.

Functions provided by a service mesh might include traffic encryption, service authentication, request
authorization, load balancing and rate limiting, service discovery, and so on. In many cases, operators can
insert these functions between services transparently. The services need not be aware that they are
communicating via the service mesh, rather than directly with each other.

| 09
Standardizing the Service Mesh

The service mesh is still an emerging software category, and several implementations are available. One
popular pattern is to deploy service mesh proxy instances as sidecar containers, a method exemplified by
Istio. Traefik Mesh, on the other hand, takes a less invasive approach that installs a single proxy per node.

SIDECAR PROXY DESIGN PATTERN

INCOMING
REQUEST

INGRESS
KUBERNETES CLUSTER

POD 1 POD 3 POD 5

PROXY PROXY PROXY

POD 2 POD 4 POD 6

PROXY PROXY PROXY

SERVICE MESH CONTROL PLANE

| 10
Another recent development is the Service Mesh Interface (SMI) specification, a standardization effort that
was launched by Microsoft in 2019 and which enjoys growing industry adΩoption. Several of the more
prominent service mesh software offerings now support SMI, which promises to make service mesh
functions more uniform across diverse implementations.

PER NODE PROXY DESIGN PATTERN

INCOMING
REQUEST

INGRESS
KUBERNETES CLUSTER

PROXY PROXY PROXY

POD 1 POD 3 POD 5

POD 2 POD 4 POD 6

SERVICE MESH CONTROL PLANE

| 11
Security and Diagnostics
Security and Diagnostics
Mechanisms such as user authentication and data encryption are fundamental to application security. In
Kubernetes environments, these measures should be employed for north-south and east-west traffic alike.
Furthermore, metrics and diagnostic information about the cluster play a vital role in maintaining overall
application health.

Kubernetes itself provides only rudimentary foundations for these features. Implementing them for real-
world use cases is among the most important functions provided by Kubernetes Ingress controllers and
service meshes.

Encryption in All Directions

Today’s best practice is to encrypt internet traffic, where possible. In some cases, regulatory compliance
requires it. At the minimum, this means Transport Layer Security (TLS). Configuring TLS properly, however,
isn’t always straightforward and can be error-prone. Environments based on dynamic, containerized
microservices, where the network topology is in constant flux, only exacerbate the problem.

Kubernetes ingress controllers can address these issues by acting as intermediaries between services.
They enforce TLS policies at the application networking layer and eliminate much of the complexity of
configuration through automation.

ACME and Let’s Encrypt

Besides configuration, another complexity associated with TLS is certificate management. Historically,
managing certificates has been a human-driven process. Administrators needed to purchase, provision,
renew, and revoke certificates manually. This approach could quickly become cumbersome even for
traditional, on-premises environments, but it is particularly undesirable for dynamic microservices
environments, where the cost and administrative overhead of manual certificate management can balloon
rapidly.

One emerging solution is the Automated Certificate Management Environment (ACME) protocol, which was
developed by the Internet Security Research Group and is now an IETF draft standard. ACME clients can
request, renew, and revoke certificates via a fully automated process from certificate authorities that
support the protocol, the most prominent example being the nonprofit Let’s Encrypt. Here again,
Kubernetes ingress controllers that include built-in support for ACME (such as Traefik Proxy and Traefik
Enterprise) can take advantage of this capability to manage certificates for Kubernetes clusters without
requiring any manual actions by administrators.

Mutual TLS

Mutual TLS (mTLS) extends the TLS model so that it is bidirectional, where the client and server each
authenticate the other’s identity for purposes of access control and authorization. This approach is widely
used for machine-to-machine communication, making it well suited for service mesh implementations,
where services must authenticate one another. Several service mesh implementations already support
mTLS for east-west network communications.

| 13
Authentication and Access Control

Another role for Kubernetes ingress controllers and service meshes is controlling access to services and
applications on the cluster. This involves two distinct, but closely related functions:

• Authentication establishes the identity of the entity making the request.

• Authorization determines whether an authenticated entity is allowed to do what it asks to do.
Similar to certificate management and TLS termination, ingress controllers can help integrate
authentication and authorization functions into a Kubernetes cluster seamlessly and transparently. For
example, enterprise-grade ingress controllers may support pluggable middleware that retrieves
authentication information from external providers. A provider might be a traditional identity system, such
as Active Directory (AD) or an LDAP server; a lightweight protocol, such as JSON web tokens (JWTs); or
even a literal authentication service provider on the public internet.

Ingress controllers may also support Open Policy Agent (OPA), an access controller. This integration allows
you to define sets of rules regarding who can access which services.

By delegating this function to the ingress controller, all services exposed in the Kubernetes cluster gain the
benefits of authentication management without incurring the complexity at the individual service layer.
Modern authentication methods can even be added to applications on a cluster without modifying any
application code.

OAuth and OpenID Connect

One authentication model is OpenID Connect (OIDC), which can be used to implement single sign-on (SSO)
authentication based on the OAuth 2.0 protocol. Modern Kubernetes ingress controllers now support this
protocol in addition to more traditional identity sources (such as LDAP).

OIDC can be particularly appealing when using internal directory services isn’t a viable option because
applications must support user identities from outside of an organization. In such cases, OIDC provides a
way for users to approve delegated access for applications by authenticating with their existing accounts
with providers that support the protocol, such as Google or Okta

Visibility into Microservices

A microservices architecture offers many advantages over legacy, monolithic application design. Among
these are the ability to better manage complexity, more flexibility for continuous delivery, and improved
fault isolation.

The implication of a microservices approach, however, is that instead of a single, large piece of software
there are many loosely coupled components that need to communicate with each other over the network.
These requests can potentially fail, and as the volume of such inter-component messages grows, it
significantly increases the number of issues and failure scenarios that may occur.

This cloud native architectural shift calls for new approaches to gaining visibility into the inner workings of
applications, and it’s another role for Kubernetes Ingress routers.

| 14
Logging and Metrics

Application logging is often the first step that developers take towards improving runtime diagnostics of
their code. The ability to access and query log data can be invaluable when teams need to investigate live-
site issues, performance anomalies, or functional bugs reported by users.

In the context of Kubernetes, the problem comes down to being able to aggregate and query data in one
system, rather than trying to manually pull logs across multiple locations and then attempt to piece
together a holistic picture.

Naturally, because the Kubernetes ingress controller can be configured to act as the master load balancer
and intermediary between all north-south traffic, it is uniquely positioned to collect the types of health and
performance data that operators need.

In other cases, Kubernetes distributions or the underlying cloud environments can also provide services for
consolidating and managing log data at scale. One such example is CloudWatch for AWS.

USERS

APPLICATION KUBERNETES
LOAD BALANCER
INGRESS

METRIC METRIC
EXPORTER EXPORTER

GRAFANA /
CLOUDWATCH DATADOG

Tracing

Most developers are already familiar with logging and monitoring as methods for understanding the
operational behavior of their applications. With microservices architectures, a third approach that teams
should incorporate into their overall arsenal is tracing.

Due to the distributed nature of these architectures, a single user request is likely to invoke multiple
underlying services. The relationship between incoming requests and inter-service invocations is often
dynamic, since it may vary based upon input values, system state, and versions of application code.

Distributed tracing allows developers to understand how an initial request to the application maps to
various underlying requests and provides the ability to identify service call paths on failures.

| 15
 Day 2 Challenges:
Network-Centric Solutions
for Common Scenarios
Day 2 Challenges: Network-Centric Solutions
for Common Scenarios
Once a Kubernetes cluster is running with selected applications, some challenges will naturally arise over
time. These must be addressed to ensure the continuous functioning of the applications. Examples
include:

• Network traffic management. There’s no guarantee that requests will arrive according to
specifications. Anomalous traffic — whether from DDoS attacks, inadequate resource provisioning, or
misconfiguration — can quickly lead to service disruption.
• Software delivery. The first version of an application deployed to Kubernetes is unlikely to be the
last. Application-level networking can offer new approaches to testing and deploying patches and
feature updates.

Managing Excess Traffic

As soon as an application is exposed to the internet, it is vulnerable to malicious, misbehaving, or

misconfigured clients that may gain privileged access to systems or simply result in a denial-of-service
(DoS) for legitimate users. When ingress controllers are used to routing external traffic, their capabilities
can be applied toward various mitigation strategies.

Allow/deny lists are perhaps the bluntest instrument in the network engineer’s toolbox. The source IP
address of an external request can be used to decide whether it should be forwarded to its destination or
discarded.

If an application has a stable set of known clients, the allow list approach ignores requests originating from
unknown IP addresses providing the strongest safety guarantees.

On the other hand, if valid traffic could potentially come from anywhere, a reactive deny list can track new
malicious traffic sources.

Rate Limiting

Another approach to mitigating denial-of-service is to constrain the number of connections that an

external client can create. This can limit the impact of misconfigured clients as well as malicious users.

For example, an ingress controller can be configured to only accept a certain number of requests in a given
time window (for example, requests per second). Such constraints can be enforced in a variety of ways.
Typical options include setting rate limits based upon client IP address or a value in the HTTP header. Rate
limits can also be configured so that a maximum request rate is enforced at service hosts.

One implementation challenge for this mechanism, however, is determining an appropriate value for the
connection limit. Many controller implementations require a global setting that impacts all sources of
traffic. Also, imposing connection rate limits by source IP address can be ineffective in scenarios where
multiple clients appear to have the same IP address (as with NAT).

| 17
Another dimension to consider is the number of concurrent requests, which may vary significantly
depending upon the relative processing time of each type of request. It may be ideal to limit the number of
simultaneous requests that are in flight so that services don’t become overwhelmed by a large number of
resource-intensive requests.

Circuit Breaker Pattern

A more sophisticated technique is the circuit breaker pattern, which is an adaptive type of rate limiting
that’s designed to mitigate against application failures due to unexpected events that might take longer
than usual to recover from.

For example, if a particular network connection becomes unexpectedly slow, it may cause requests to
begin to time out. In a microservices application, such repeated failures between two systems may lead to
a cascade of errors that could negatively impact the entire system. When circuit breaking, the network
automatically applies limits on the rate of requests that are likely to fail. This gives the system time to
recover (for example, to spin up a new instance of a failing service).

This pattern is often desirable for service mesh implementations, where the volume of automated east-to-
west, machine-to-machine requests is high.

| 18
Software Delivery Management

An application or service deployed on Kubernetes will inevitably require basic lifecycle operations,
including feature updates and bug fixes. Kubernetes supports the ability to quickly scale deployments by
changing the number of replica instances defined in a configuration, and it handles the orchestration of
rolling updates based upon user-specified settings. These capabilities make it possible to use the ingress
controller to route traffic in ways that support a number of advanced software deployment strategies.

Canary Releases

Once the development of feature enhancements or bug fixes is complete, the next step is usually to test
them in a staging environment. Yet staging environments do not always present an accurate picture of
application performance. It’s often helpful to see how a candidate release performs against real-world
requests in the production environment, albeit in a controlled way.

To achieve this, a Kubernetes Ingress controller can be configured to route some fraction of requests to the
candidate release to determine its viability, while still directing the bulk of traffic to the production release.
If no negative issues or side effects are observed with a canary release, the number of requests routed to it
can be gradually increased until it has effectively replaced the earlier version.

OLD VERSION

MOST USERS (95%)

APLICATION
WEB SERVER DATABASE
SERVER

INGRESS
USER
NEW VERSION

SOME USERS (5%)

APLICATION
WEB SERVER DATABASE
SERVER

| 19
A/B Testing

Another way to use application networking to aid software deployment is to configure the Kubernetes
ingress controller to split traffic between two versions of an application at once. This is usually done for
purposes of user experience testing, where the goal is to determine whether users prefer one version over
the other.

If, for example, users stay engaged with version B of the application, it will become the preferred version;
unless, of course, they are only spending more time with it because its UI is harder to understand. Much
like canary deployments, admins can use the ingress controller configuration to weigh the amount of traffic
in the desired direction.

VERSION A

50% USERS

APLICATION
WEB SERVER DATABASE
SERVER

INGRESS
USER
VERSION B

50% USERS
APLICATION
WEB SERVER DATABASE
SERVER

| 20
Conclusions
Conclusions
Kubernetes is the leading tool for creating and managing modern infrastructure today, in environments
ranging from on-premises to the public cloud. The automation of tedious, human-driven processes is key
to unlocking the value of Kubernetes, particularly as organizations move toward software delivery models
based on containers and microservices.

This paper has sought to demonstrate how application-layer networking is a critical component of a
successful Kubernetes infrastructure stack. And yet, networking can often be one of the more obscure
facets of running Kubernetes in production.

The Kubernetes Ingress resource is key to defining networking behavior for a Kubernetes cluster, and the
ingress controller is the engine that turns those definitions into a functioning reality. Whether enabling
external connectivity or access control mechanisms during early deployment phases or managing day-
two scenarios around lifecycle management, the ingress controller plays a critical and outsized role in the
Kubernetes architecture.

Not all ingress controllers provide the same capabilities or efficiencies, however. The importance of this
crucial component dictates that teams must spend time thoughtfully considering their options and choose
an implementation that can help address the breadth of their expected scenarios.

Ingress controllers, particularly for business-critical applications, should be expected to meet the needs of
organizations large and small when it comes to ease of configuration, flexibility, security, and support for a
variety of software lifecycle management models. By taking full advantage of such features, organizations
can gain the most value from their Kubernetes deployments and ensure that they are well-positioned for
success.

Unlock the Power

of Kubernetes
Simpler, automated cloud native
networking with Traefik Enterprise

REQUEST A DEMO

LEARN MORE
traefik.io