Category Archives: Azure Kubernetes

Azure Kubernetes Service in an Azure Stack HCI environment

The hyper-converged Azure Stack HCI solution allows you to activate the Azure Kubernetes Service orchestrator in an on-premises environment (AKS) for running containerized applications at scale. This article explores how Azure Kubernetes in Azure Stack HCI environment offers the possibility of hosting Linux and Windows containers in your datacenter, going to explore the main benefits of this solution.

Before going into the specifics of AKS in the Azure Stack environment, a summary of the solutions involved is reported.

What is Kubernetes?

Kubernetes, also known as "k8s", provides automated orchestration of containers, improving its reliability and reducing the time and resources required in the DevOps field, through:

  • Generally simpler deployments that allow automatic implementations and rollbacks.
  • Better application management with the ability to monitor the status of services to avoid implementation errors. In fact, the various features include service integrity checks, with the ability to restart containers that are not running or that are blocked, allowing to advertise to clients only the services that have started correctly.
  • Ability to scale automatically based on usage and, exactly the same as for containers, manage the cluster environment in a declarative manner, allowing version-controlled and easily replicable configuration.

Figure 1 – Kubernetes cluster with related architecture components

What is Azure Kubernetes Service (AKS)?

Azure Kubernetes Service (AKS) is the fully managed Azure service that allows the activation of a Kubernetes cluster, ideal for simplifying the deployment and management of microservices-based architectures. Thanks to the features offered by AKS it is possible to scale automatically according to the use, use controls to ensure the integrity of the services, implement load balancing policies and manage secrets. The use of this managed service is integrated with the container development and deployment pipelines.

Figure 2 - Azure Kubernetes Service architecture example (AKS)

What is Azure Stack HCI?

Azure Stack HCI is the solution that allows you to create a hyper-converged infrastructure (HCI) for the execution of workloads in an on-premises environment and which provides for a strategic connection to Azure services. This is a hyper-converged infrastructure (HCI), where different hardware components are removed, substitutes from the software, able to combine the layer of compute, storage and network in one solution. In this way there is a transition from a traditional "three tier" infrastructure, composed of network switches, appliance, physical systems with onboard hypervisors, storage fabric and SAN, toward hyper-converged infrastructure (HCI).

Figure 3 – "Three Tier" Infrastructure vs Hyper-Converged Infrastructure (HCI)

What is AKS in Azure Stack HCI?

AKS in the Azure Stack HCI environment is a Microsoft implementation of AKS, which automates the deployment and management of containerized applications.

Microsoft, after introducing AKS as a service in Azure, has extended its availability also to on-premises environments. However, there are some important differences:

  • In Azure, Microsoft manages the control plane of each AKS cluster. Furthermore, the cluster nodes (management node and worker node) run on Azure virtual machines or on Azure virtual machine scale sets.
  • In an on-premises environment , the customer manages the entire environment, where the AKS cluster nodes are running on virtual machines hosted on the hyper-converged infrastructure.

AKS architecture on Azure Stack HCI

The implementation of AKS in Azure Stack HCI consists of two types of clusters:

  • A management cluster of AKS. This cluster acts as a dedicated control plane for managing Kubernetes clusters running on the hyper-converged platform. This cluster consists of Linux virtual machines, that host Kubernetes system components such as API servers and load balancers.
  • One or more Kubernetes clusters. These clusters consist of control nodes and worker nodes. Control nodes are implemented as Linux virtual machines, with API server and load balancers that satisfy the requests of Azure Stack HCI users. Workloads are distributed on Linux or Windows OS-based worker nodes.

Figure 4 - AKS architecture on Azure Stack HCI

Each Kubernetes cluster runs on its own dedicated set of virtual machines, protected by hypervisor-based isolation, allowing you to securely share the same physical infrastructure even in scenarios that require workload isolation.

AKS on Azure Stack HCI supports both Linux-based and Windows-based containers. When you create a Kubernetes cluster you simply need to specify the type of container you intend to run and on the hyper-converged platform the installation procedure of the required operating system is automatically started on the nodes of the Kubernetes cluster .

Benefits of AKS on Azure Stack HCI

AKS simplifies the deployment of Kubernetes clusters by providing a layer of abstraction that can mask some of the more challenging implementation details.

Among the main benefits of AKS in the Azure Stack HCI environment we find:

  • Simplified deployments of containerized apps in a cluster environment. Using the Windows Admin Center you have a guided installation process of the AKS management cluster. Windows Admin Center also facilitates the installation of individual Kubernetes clusters that contain worker nodes, through an automatic installation process of all relevant software components, including management tools such as kubectl.
  • Ability to scale horizontally to manage computational resources, adding or removing Kubernetes cluster nodes.
  • Simplified management of cluster resource storage and network configurations.
  • Automatic updates of cluster nodes to the latest version of Kubernetes available. Microsoft manages the Windows Server and Linux images for the cluster nodes and updates them monthly.
  • Strategic connection, using Azure Arc, to Azure services such as: Microsoft Azure Monitor, Azure Policy, and Azure Role-Based Access Control (RBAC).
  • Centralized management of Kubernetes clusters and related workloads through the Azure portal, thanks to the adoption of Azure Arc for Kubernetes. Azure portal-based management also integrates traditional Kubernetes administration tools and interfaces, like the command line utility kubectl and the Kubernetes dashboard.
  • Managing the automatic failover of virtual machines acting as Kubernetes cluster nodes if there is a localized failure of the underlying physical components. This complements the high availability inherent in Kubernetes, able to automatically restart containers in failed state.

Conclusions

Thanks to Azure Stack HCI, the adoption of container-based application architectures can be hosted directly in your own datacenter, adopting the same Kubernetes management experience that you have with the managed service present in the Azure public cloud. The deployment process is also very simplified and intuitive. Furthermore, Azure Stack HCI allows you to further improve the agility and resilience of Kubernetes deployments in an on-premises environment.

Secure network architecture design for Azure Kubernetes Service (AKS)

The trend in adopting applications based on microservices requires the use of state-of-the-art solutions capable of managing a large number of containers and the ways in which these interact in application with each other, as Azure Kubernetes Service (AKS). As part of the design of Azure Kubernetes Service architectures (AKS) there are several elements that need to be evaluated to obtain an appropriate network topology that can ensure maximum efficiency and security. This article outlines the main points to consider, accompanied by some proposals, to make informed choices when designing network architectures for AKS.

What is Azure Kubernetes Service (AKS)?

Azure Kubernetes Service (AKS) is the fully managed Azure service that allows the activation of a Kubernetes cluster, ideal for simplifying the deployment and management of microservices-based architectures. Thanks to the features offered by AKS it is possible to scale automatically according to the use, use controls to ensure the integrity of the services, implement load balancing policies and manage secrets. In microservices-based architectures, it is also common to adopt the Azure Container Registry that allows you to create, store and manage container images and artifacts in a private registry. The use of this managed service is integrated with the container development and deployment pipelines.

Figure 1 - Azure Kubernetes Service architecture example (AKS)

The network topology

In the network architecture of type Hub and Spoke, theHub is a virtual network on Azure that serves as the point of connectivity to the on-premises network. This connectivity can be done through VPN Site to site or through ExpressRoute. TheSpoke are virtual networks running the peering with the Hub and can be used to isolate workloads.

Figure 2 - Hub and Spoke network topology

This network topology is also recommended for AKS architectures as it can offer several advantages, including:

  • Environmental segregation to more easily enforce governance policies and gain greater control. This topology also supports the concept of "landing zones" by contemplating the separation of duties.
  • Minimizing the direct exposure of Azure resources to the public network (Internet).
  • Possibility of contemplating workloads attested on different Azure subscriptions, becoming a natural choice in these scenarios.
  • Ability to easily extend the architecture to accommodate new features or new workloads, simply by adding additional spoke virtual networks.
  • Ability to centralize Azure services shared by multiple workloads in a single location (attested on different VNet), such as DNS servers and any virtual network appliances. It also reduces the VPN Gateways to provide connectivity to the on-premises environment, resulting in savings on Azure costs and simplification of the architecture.

Figure 3 - Hub and Spoke network topology for AKS

Hub Virtual Network

In the Hub network it is possible to evaluate the adoption of the following services:

  • VPN or ExpressRoute Gateway: necessary to provide connectivity to the on-premises environment.
  • Firewall Solutions, necessary in case you want to control the traffic from your AKS environment, as pods or cluster nodes, outgoing to external services. In this context, the choice can fall between:
    • Azure Firewall, the firewall-as-a-service solution (FWaaS) which allows to secure the resources present in the Virtual Networks and to govern the related network flows.
    • Network Virtual Appliances (NVAs) provided by third party vendors. Such solutions are numerous and can offer advanced functionality, but typically the configuration of these solutions is more complex and the cost tends to be higher than the solution provided by the Azure platform. A comparison between the new Azure Firewall and third-party virtual appliances can be found in this article.
  • Azure Bastion, the PaaS service that offers secure and reliable RDP and SSH access to virtual machines, directly through the Azure portal.

Spoke Virtual Network

The AKS cluster is placed in the Spoke network together with other resources closely related to its operation. Spoke VNet is split into different subnets to accommodate the following components:

  • The two groups of nodes (node pools) in AKS:
    • AKS System Node pool: the pool of system nodes that host the pods needed to run the core services of the cluster.
    • AKS User Node pool: the pool of user nodes that run the application workloads and the ingress controller.

For multi-tenant application environments or for workloads with advanced needs, it may be necessary to implement isolation mechanisms of node pools that require the presence of different subnets.

  • AKS Internal Load Balancer: the balancer to route and distribute inbound traffic for Kubernetes resources. In this case the component is used Azure Load Balancer, which enables Layer-4 load balancing for all TCP and UDP protocols, ensuring high performance and very low latencies.
  • Azure Application Gateway: it is a service managed by the azure platform, with inherent features of high availability and scalability. The Application Gateway is a application load balancer (OSI layer 7) for web traffic, that allows you to govern HTTP and HTTPS applications traffic (URL path, host based, round robin, session affinity, redirection). The Application Gateway is able to centrally manage certificates for application publishing, using SSL and SSL offload policy when necessary. The Application Gateway may have assigned a private IP address or a public IP address, if the application must be republished in Internet. In particular in the latter case, it is recommended to turn onWeb Application Firewall (WAF), that provides application protection, based on rulesOWASP core rule sets. The WAF protects the application from vulnerabilities and against common attacks, such as X-Site Scripting and SQL Injection attacks.

Thanks to the adoption of Azure Private Link you can bring Azure services to a virtual network and map them with a private endpoint. In this way, all traffic is routed through the private endpoint, keeping it on the Microsoft global network. The data does not pass ever on the Internet, this reduces exposure to threats and helps to meet the compliance standards.

Figure 4 - Overview of Azure Private Link

In AKS environments theAzure Private Link they are usually created in the Spoke virtual network subnets for Azure Container Registry and Azure KeyVault.

Below is a diagram with the incoming and outgoing network flows for an AKS environment, which also includes the presence of Azure Firewall to control outgoing traffic.

Figure 5 - Example of network flows in a typical AKS architecture

Management traffic

In order to allow the management of the environment, such as creating new resources or carrying out activities to scale the cluster environment, it is advisable to provide access to the Kubernetes API. Good practice is apply network filters to authorize this access in a timely manner.

Private AKS cluster

In case you want to implement a totally private AKS environment, where no Internet service is exposed, it is possible to adopt a AKS cluster in "private" mode.

Conclusions

The increasing demand for microservices-based application architectures that useAzure Kubernetes Service (AKS) requires you to locate and build network architectures designed to be secure, flexible and with a high level of integration. All this must take place through a modern approach able to fully exploit the potential offered in the field of networking by Azure.