Category Archives: Datacenter Management

Azure IaaS and Azure Stack: announcements and updates (April 2021 – Weeks: 15 and 16)

This series of blog posts includes the most important announcements and major updates regarding Azure infrastructure as a service (IaaS) and Azure Stack, officialized by Microsoft in the last two weeks.

Azure

Compute

New M-series Msv2/Mdsv2 Medium Memory VMs for memory-optimized workloads

Azure Msv2/Mdsv2 Medium Memory Series offering up to 192vCPU and 4TB memory configurations and running on Cascade Lake processor are now generally available. Msv2/Mdsv2 medium memory VM sizes providing a 20% increase in CPU performance, increased flexibility with local disks, and a new intermediate scale up-option. These virtual machines provide unparalleled computational performance to support large in-memory databases and workloads such as SAP HANA and SQL Hekaton.

Azure Virtual Machines DCsv2-series in Azure Government (public preview)

Azure Government customers can build secure, enclave-based applications to protect code and data while it’s in use, in a dedicated cloud that meets stringent government security and compliance requirements. Confidential computing DCsv2-series virtual machines are now in preview for Azure Government customers (federal, state, local governments, and their partners) in US Government Virginia and Arizona regions. These VMs are backed by Intel XEON E-2288G processors with Intel Software Guard Extensions (SGX) technology.

Microsoft announces plans to establish first datacenter region in Malaysia

The new datacenter region is part of the “Bersama Malaysia” initiative to support inclusive economic growth in Malaysia.

Storage

Azure Blob storage supports objects up to 200 TB in size

Workloads that utilize larger file sizes such as backups, media, and seismic analysis can now utilize Azure Blob storage and ADLS Gen2 without breaking these large files into separate blobs. Each blob is made up of up to 50,000 blocks. Each block can now be 4GB in size for a total of 200 TB per blob or ADLS Gen2 file.

Lustre HSM tools to import from or export to Azure Storage

Lustre HSM (Hierarchical Storage Management) provides the capability to associate a Lustre file system with an external storage system and migrate file data between them.

Now available are the File System Hydrator and Copy Tool, which enables integrating a Lustre file system with an Azure storage account:

  • The File System Hydrator is used to import a file system namespace from an Azure storage account into a Lustre file system with the imported files left in the ‘released’/’exist’ state.
  • The Copy Tool is used to hydrate the content of the files in the storage account into the Lustre file system on-demand. The copy tool can also be used to archive content of files back into the storage account, including changed or added files.

Networking

Application Gateway URL Rewrite

Azure Application Gateway now supports the ability to rewrite host name, path and query string of the request URL. In addition to header rewrites, you can now also rewrite URL of all or some of the client requests based on matching one or more conditions as required. You can choose to route the request based on the original URL or the rewritten URL. This feature enables several important scenarios such as allowing path based routing for query string values and support for hosting friendly URLs.

Azure IaaS and Azure Stack: announcements and updates (April 2021 – Weeks: 13 and 14)

This series of blog posts includes the most important announcements and major updates regarding Azure infrastructure as a service (IaaS) and Azure Stack, officialized by Microsoft in the last two weeks.

Azure

Compute

Virtual machine (VM) level disk bursting available on all Dsv3 and Esv3 families

Virtual machine level disk bursting allows your virtual machine to burst its disk IO and MiB/s throughput performance for a short time daily. This feature is now enabled on all our Dsv3-series and Esv3-series virtual machines, with more virtual machine types and families support soon to come. There is no additional cost associated with this new capability or adjustments on the VM pricing and it comes enabled by default.

Cloud Services (extended support) is generally available

Cloud Services (extended support), which is a new Azure Resource Manager (ARM)-based deployment model for Azure Cloud Services, is generally available. Cloud Services (extended support) has the primary benefit of providing regional resiliency along with feature parity with Azure Cloud Services deployed using Azure Service Manager (ASM). It also offers some ARM capabilities such as role-based access and control (RBAC), tags, policy, private link support, and use of deployment templates. The ASM-based deployment model for Cloud Services has been renamed Cloud Services (classic). Customers retain the ability to build and rapidly deploy web and cloud applications and services. Customers will be able to scale cloud services infrastructure based on current demand and ensure that the performance of applications can keep up while simultaneously reducing costs. The platform-supported tool for migrating existing cloud services to Cloud Services (extended support) also goes into preview. Migrating to ARM will allow customers to set up a robust infrastructure platform for their applications. 

Storage

Azure File Sync agent v12 

Improvements and issues that are fixed in the v12 release:

  • New portal experience to configure network access policy and private endpoint connections
    • You can now use the portal to disable access to the Storage Sync Service public endpoint and to approve, reject and remove private endpoint connections. To configure the network access policy and private endpoint connections, open the Storage Sync Service portal, go to the Settings section and click Network.
  • Cloud Tiering support for volume cluster sizes larger than 64KiB
  • Measure bandwidth and latency to Azure File Sync service and storage account
    • The Test-StorageSyncNetworkConnectivity cmdlet can now be used to measure latency and bandwidth to the Azure File Sync service and storage account. Latency to the Azure File Sync service and storage account is measured by default when running the cmdlet. Upload and download bandwidth to the storage account is measured when using the “-MeasureBandwidth” parameter. To learn more, see the release notes.
  • Improved error messages in the portal when server endpoint creation fails
    • We heard your feedback and have improved the error messages and guidance when server endpoint creation fails.
  • Miscellaneous performance and reliability improvements
    • Improved change detection performance to detect files that have changed in the Azure file share.
    • Performance improvements for reconciliation sync sessions.
    • Sync improvements to reduce ECS_E_SYNC_METADATA_KNOWLEDGE_SOFT_LIMIT_REACHED and ECS_E_SYNC_METADATA_KNOWLEDGE_LIMIT_REACHED errors.
    • Files may fail to tier on Server 2019 if Data Deduplication is enabled on the volume.
    • AFSDiag fails to compress files if a file is larger than 2GiB.

To obtain and install this update, configure your Azure File Sync agent to automatically update when a new version becomes available or manually download the update from the Microsoft Update Catalog.

More information about this release:

  • This release is available for Windows Server 2012 R2, Windows Server 2016 and Windows Server 2019 installations.
  • A restart is required for servers that have an existing Azure File Sync agent installation.
  • The agent version for this release is 12.0.0.0.
  • Installation instructions are documented in KB4568585.

Encryption scopes in Azure Storage

Encryption scopes introduce the option to provision multiple encryption keys in a storage account for blobs. Previously, customers using a single storage account for multi-tenancy scenarios were limited to using a single account-scoped encryption key for all the data in the account. With encryption scopes, you now can provision multiple encryption keys and choose to apply the encryption scope either at the container level (as the default scope for blobs in that container) or at the blob level. 

Azure Data Explorer external tables

An external table is a schema entity that references data stored outside the Azure Data Explorer database. Azure Data Explorer Web UI can create external tables by taking sample files from a storage container and creating schema based on these samples. You can then analyze and query data in external tables without ingestion into Azure Data Explorer.

Azure Governance: how to control system configurations in hybrid and multicloud environments

There are several companies that are investing in hybrid and multicloud technologies to achieve high flexibility, that enables you to innovate and meet changing business needs. In these scenarios, customers face the challenge of using IT resources efficiently, in order to best achieve your business goals, implementing a structured IT governance process. This can be achieved more easily if you have solutions that, in a centralized way, allow you to inventory, organize and enforce control policies on your IT resources wherever you are. Azure Arc solution involves different technologies with the aim of supporting hybrid and multicloud scenarios, where Azure services and management principles are extended to any infrastructure. In this article we will explore how, thanks to the adoption of the Azure Guest Configuration Policy it is possible to control the configurations of systems running in Azure, in on-premises datacenters or other cloud providers.

The principle behind Azure Arc

The principle behind Azure Arc is to extend Azure management and governance practices to different environments and to adopt typically cloud solutions, as DevOps techniques (infrastructure as code), also for on-premises and multicloud environments.

Figure 1 – Azure Arc overview

Enabling systems to Azure Arc

Enabling Azure Arc servers allows you to manage physical servers and virtual machines residing outside Azure, on the on-premises corporate network or at another cloud provider. This applies to both Windows and Linux systems. This management experience is designed to provide consistency with Azure native virtual machine management methodologies. In fact, connecting a machine to Azure through Arc is considered in all respects as an Azure resource. Each connected machine has a specific ID, is included in a resource group and benefits from standard Azure constructs such as Azure Policies and tagging.

To offer this experience, the installation of the specific Azure Arc agent is required on each machine that is planned to connect to Azure ("Azure Connected Machine"). The following operating systems are currently supported:

  • Windows Server 2008 R2, Windows Server 2012 R2 or higher (this includes core servers)
  • Ubuntu 16.04 and 18.04 LTS (x64)
  • CentOS Linux 7 (x64)
  • SUSE Linux Enterprise Server (SLES) 15 (x64)
  • Red Hat Enterprise Linux (RHEL) 7 (x64)
  • Amazon Linux 2 (x64)
  • Oracle Linux 7

The Azure Arc Connected Machine agent consists of the following logical components:

  • TheHybrid Instance Metadata service (HIMDS) that manages the connection to Azure and the Azure identity of the connected machine.
  • The Guest Configuration agent that provides in-guest policy and guest configuration features.
  • TheExtension Manager agent that manages installation processes, uninstalling and updating machine extensions.

Figure 2 – Azure Arc Agent Components

The Connected Machine agent requires secure outbound communication to Azure Arc on TCP port 443.

This agent provides no other features and does not replace the Azure Log Analytics agent, which remains necessary when you want to proactively monitor the operating system and workloads running on the machine.

For more information about installing Azure Arc, see this official Microsoft document.

Azure Arc-enabled servers can benefit from several Azure Resource Manager-related features such as Tags, Policies and RBAC, as well as some features related to Azure Management.

Figure 3 – Azure Management for all IT resources

Guest Configuration Policy di Azure

Guest Configuration Policies allow you to control settings within a machine, both for virtual machines running in Azure environment and for "Arc Connected" machines. Validation is performed by the client and by the Guest Configuration extension as regards:

  • Operating system configuration
  • Configuration or presence of applications
  • Environment settings

At the moment, most of the Azure Guest Configuration Policies only allow you to make checks on the settings inside the machine, but they don't apply configurations. The exception is a built-in time zone configuration policy operating system for Windows machines.

Requirements

Before you can check the settings inside a machine, through guest configuration policies, you must:

  • Enable a’extension on the Azure VM, required to download assigned policy assignments and corresponding configurations. This extension is not required for "Arc Connected" machines as it is included in the Arc agent.
  • Make sure that the machine has a system-managed identity, used for the authentication process when reading and writing to the guest configuration service.

Operation

Azure provides built-in specification platform Initiatives and a large number of Guest Configuration Policy, but you can also create custom one both in Windows environment, both in Linux environment.

Guest Configuration policy assignment works the same way as standard Azure Policies, so you can group them into initiative. Specific parameters can also be configured for Guest Configuration Policies and there is at least one parameter that allows you to include Azure Arc-enabled servers. When you have the desired policy definition, it is possible to assign it to a subscription and possibly in a more limited way to a specific Resource Group. You also have the option of excluding certain resources from the application of the policy.

Following the assignment, it is possible to assess the compliance status in detail directly from the Azure portal.

Inside the machine, the Guest Configuration agent uses local tools to audit the configurations:

The Guest Configuration agent checks for new or modified guest policy assignments each 5 minutes and once the assignment is received the settings are checked at intervals of 15 minutes.

The Cost of the Solution

The cost of Azure Guest Configuration Policies is based on the number of servers registered to the service and which have one or more guest configurations assigned. Any other type of Azure Policy that is not based on guest configuration is offered at no additional cost, including virtual machine extensions to enable services such as Azure Monitor and Azure Security Center or auto tagging policies. The billing is distributed on an hourly basis and also includes the change tracking features present through Azure Automation. For more details on costs please visit the Microsoft's official page.

Conclusions

IT environments are constantly evolving and often have to deliver business-critical applications based on different technologies, active on heterogeneous infrastructures and which in some cases use solutions provided in different public clouds. The adoption of a structured IT governance process is easier also thanks to the Guest Configuration Policies and the potential of Azure Arc, that allow you to more easily control and support hybrid and multicloud environments.