Category Archives: Datacenter Management

Azure Management services: What's New in April 2021

Microsoft is constantly announcing news regarding Azure management services. This summary, released on a monthly basis, allows you to have an overall overview of the main news of the current month, in order to stay up to date on these news and have the necessary references to conduct further study.

The following diagram shows the different areas related to management, which are covered in this series of articles, in order to stay up to date on these topics and to better deploy and maintain applications and resources.

Figure 1 – Management services in Azure overview

Monitor

Azure Monitor

New agent version for Windows Systems

A new version of the Log Analytics agent has been released this month for Window systemss. The new version includes a new tool for troubleshooting and handles changes to certificates in Azure services differently.

The uniqueness of the name of the Log Analytics workspaces is now per resource group

In the past, the uniqueness of the Azure Monitor Log Analytics workspace was globally for all subscriptions. This meant that when a workspace name was used by a customer, it could not be reused by others. Microsoft has changed the way in which the uniqueness of the workspace name is requested and is now managed in the context of the resource group.

New definitions built-in of the Azure Policy for data encryption in Azure Monitor

Azure Monitor provides built-in policies for data encryption governance and control over the key used for encryption at rest. Here are the new built-in policies available for data encryption:

  • Azure Monitor logs clusters should be encrypted with customer-managed key – Audit if log analytics cluster is defined with customer-managed key.
  • Azure Monitor logs clusters should be created with infrastructure-encryption enabled (double encryption) – Audit log analytics cluster is created with Infrastructure enabled.
  • Azure Monitor logs for application insights should be linked to a log analytics workspace – Audit if application insights is linked to store data in log analytics workspace. Workspace can then be linked to a log analytics cluster for customer-managed key settings.
  • Saved-queries in Azure Monitor should be saved in customer storage account for logs encryption – Audit if workspace has linked storage account, which allows the encryption using customer-managed key.
  • Log alert queries in Azure Monitor will be saved in customer storage account, if workspace has linked storage account, which allows the encryption using customer-managed key.

Improvements for Log Alerts

Log Alerts are available in Azure Monitor that allow users to use a Log Analytics query to evaluate the resources logs at a set frequency and activate an alert based on the results obtained. Rules can trigger one or more actions using Action Groups. In this context, two new highly requested features have been released (in preview):

  • Stateful Log Alert: with this feature enabled, activated alerts are automatically resolved once the condition is no longer satisfied. In this way, the same behavior is adopted as in the alerts related to metrics.
  • Frequency of 1 minute: with this feature enabled, the alert query is evaluated every minute to verify the specified condition, thus reducing the overall time for activating a Log Alert.

Availability in new regions

Azure Monitor Log Analytics is also available in the region South India.

To check the availability of the service in all the Azure regions you can consult this document.

Container insights: support for the monitor of Kubernetes Azure Arc enabled environment (preview)

Containers insights in Azure Monitor has extended its monitor capabilities to Azure Arc Kubernetes clusters as well, providing the same monitoring capabilities present for the Azure Kubernetes service (AKS), which:

  • Visibility on the performance of the environment, through the memory and processor metrics for the controllers, nodes and containers.
  • View information collected through workbooks and in the Azure portal.
  • Alert and possibility of querying historical data for problem solving.
  • Ability to verify Prometheus metrics.

Configure

Azure Automation

Availability in new regions

Azure Automation is also available in the region South India.

Support for System Assigned Managed Identities for cloud and Hybrid job (public preview)

Azure Automation has introduced support for System Assigned Managed Identities for cloud and Hybrid jobs. Among the advantages of using Managed Identities we find:

  • The ability to authenticate to any Azure service that supports Azure AD authentication.
  • Elimination of the management overhead associated with managing Run As accounts in runbook code. This makes it possible to access resources via the Managed Identity of an Automation account from a runbook, without having to worry about creating RunAsCertificate, RunAsConnection, etc.
  • It is not necessary to renew the certificate used by the Automation Run As account.

Govern

Azure Cost Management

Updates related toAzure Cost Management and Billing

Microsoft is constantly looking for new methodologies to improve Azure Cost Management and Billing, the solution to provide greater visibility into where costs are accumulating in the cloud, identify and prevent incorrect spending patterns and optimize costs . Inthis article some of the latest improvements and updates regarding this solution are reported, including:

Secure

Azure Security Center

New features, bug fixes and deprecated features of Azure Security Center

Azure Security Center development is constantly evolving and improvements are being made on an ongoing basis. To stay up to date on the latest developments, Microsoft updates this page, this provides information about new features, bug fixes and deprecated features. In particular, this month the main news concern:

Protect

Azure Backup

Azure Dedicated Host protection support

Azure Backup has introduced support for the backup and recovery of virtual machines residing on Azure Dedicated Host, physical servers dedicated to your organization whose capacity is not shared with other customers. This feature is available in all Azure regions where Azure Dedicated Host can be activated.

Azure VM Scale sets protection with orchestration templates (preview)

Azure Backup now allows you to backup and restore Azure VM Scale sets with orchestration models, which provide a logical grouping of virtual machines managed by the platform.

Improvements in encryption using customer managed keys (preview)

Azure Backup now allows you to use your own keys to encrypt backup data residing in the Recovery Services vaults. This new feature allows you to increase the control of the encryption of your data. Furthermore, you can use the Azure Policy to control and apply encryption using keys managed directly by the customer.

Azure Site Recovery

Support for Azure Policy (preview)

The ability to use Azure Policy is now provided to enable large-scale use of Azure Site Recovery for virtual machines. After creating a disaster recovery policy for a resource group, all new virtual machines that will be added to this resource group will have Site Recovery enabled automatically. Furthermore, through a Remediation process, Site Recovery can also be enabled for all virtual machines already present in the Resource Group.

Support for cross-continental disaster recovery (for 3 region pairs)

Azure Site Recovery introduced support for cross-continental disaster recovery. Thanks to this feature, a virtual machine can be replicated from an Azure region in one continent to a region in another continent. In the event of a planned or unplanned outage, you will be able to fail over the virtual machine on all continents and, once the interruption has been mitigated, it can be brought back to the continent of origin (fail-back) and protected. This feature is currently available for the following 3 pairs of intercontinental regions:

  • Southeast Asia and Australia East
  • Southeast Asia and Australia Southeast
  • West Europe and South Central US

Support of “proximity placement groups” in hybrid and cloud disaster recovery scenarios

Azure Site Recovery introduced support for “proximity placement groups (PPG)” in hybrid and cloud disaster recovery scenarios. With this support it will be possible to replicate an on-premises physical or virtual machine or an Azure virtual machine within a PPG, in the chosen Azure target area. Upon activation of the failover plan, Site Recovery will activate the failover VM within the target PPG selected by the user. This functionality is available both through the Azure portal and through PowerShell and REST API, across all Azure regions.

Migrate

Azure Migrate

New Azure Migrate releases and features

Azure Migrate is the service in Azure that includes a large portfolio of tools that you can use, through a guided experience, to address effectively the most common migration scenarios. To stay up-to-date on the latest developments in the solution, please consult this page, that provides information about new releases and features. In particular, this new release was released this month:

  • The tools Azure Migrate: Discovery and Assessment and Azure Migrate: Server Migration can be used by connecting privately and securely to the Azure Migrate service via ExpressRoute or via a site-to-site VPN, using Azure private links. This connectivity method is recommended to use when there is an organizational requirement to access the Azure Migrate service and other Azure resources without crossing public networks or if you want to get better results in terms of bandwidth or latency.

Evaluation of Azure

To test for free and evaluate the services provided by Azure you can access this page.

Azure IaaS and Azure Stack: announcements and updates (April 2021 – Weeks: 15 and 16)

This series of blog posts includes the most important announcements and major updates regarding Azure infrastructure as a service (IaaS) and Azure Stack, officialized by Microsoft in the last two weeks.

Azure

Compute

New M-series Msv2/Mdsv2 Medium Memory VMs for memory-optimized workloads

Azure Msv2/Mdsv2 Medium Memory Series offering up to 192vCPU and 4TB memory configurations and running on Cascade Lake processor are now generally available. Msv2/Mdsv2 medium memory VM sizes providing a 20% increase in CPU performance, increased flexibility with local disks, and a new intermediate scale up-option. These virtual machines provide unparalleled computational performance to support large in-memory databases and workloads such as SAP HANA and SQL Hekaton.

Azure Virtual Machines DCsv2-series in Azure Government (public preview)

Azure Government customers can build secure, enclave-based applications to protect code and data while it’s in use, in a dedicated cloud that meets stringent government security and compliance requirements. Confidential computing DCsv2-series virtual machines are now in preview for Azure Government customers (federal, state, local governments, and their partners) in US Government Virginia and Arizona regions. These VMs are backed by Intel XEON E-2288G processors with Intel Software Guard Extensions (SGX) technology.

Microsoft announces plans to establish first datacenter region in Malaysia

The new datacenter region is part of the “Bersama Malaysia” initiative to support inclusive economic growth in Malaysia.

Storage

Azure Blob storage supports objects up to 200 TB in size

Workloads that utilize larger file sizes such as backups, media, and seismic analysis can now utilize Azure Blob storage and ADLS Gen2 without breaking these large files into separate blobs. Each blob is made up of up to 50,000 blocks. Each block can now be 4GB in size for a total of 200 TB per blob or ADLS Gen2 file.

Lustre HSM tools to import from or export to Azure Storage

Lustre HSM (Hierarchical Storage Management) provides the capability to associate a Lustre file system with an external storage system and migrate file data between them.

Now available are the File System Hydrator and Copy Tool, which enables integrating a Lustre file system with an Azure storage account:

  • The File System Hydrator is used to import a file system namespace from an Azure storage account into a Lustre file system with the imported files left in the ‘released’/’exist’ state.
  • The Copy Tool is used to hydrate the content of the files in the storage account into the Lustre file system on-demand. The copy tool can also be used to archive content of files back into the storage account, including changed or added files.

Networking

Application Gateway URL Rewrite

Azure Application Gateway now supports the ability to rewrite host name, path and query string of the request URL. In addition to header rewrites, you can now also rewrite URL of all or some of the client requests based on matching one or more conditions as required. You can choose to route the request based on the original URL or the rewritten URL. This feature enables several important scenarios such as allowing path based routing for query string values and support for hosting friendly URLs.

Azure IaaS and Azure Stack: announcements and updates (April 2021 – Weeks: 13 and 14)

This series of blog posts includes the most important announcements and major updates regarding Azure infrastructure as a service (IaaS) and Azure Stack, officialized by Microsoft in the last two weeks.

Azure

Compute

Virtual machine (VM) level disk bursting available on all Dsv3 and Esv3 families

Virtual machine level disk bursting allows your virtual machine to burst its disk IO and MiB/s throughput performance for a short time daily. This feature is now enabled on all our Dsv3-series and Esv3-series virtual machines, with more virtual machine types and families support soon to come. There is no additional cost associated with this new capability or adjustments on the VM pricing and it comes enabled by default.

Cloud Services (extended support) is generally available

Cloud Services (extended support), which is a new Azure Resource Manager (ARM)-based deployment model for Azure Cloud Services, is generally available. Cloud Services (extended support) has the primary benefit of providing regional resiliency along with feature parity with Azure Cloud Services deployed using Azure Service Manager (ASM). It also offers some ARM capabilities such as role-based access and control (RBAC), tags, policy, private link support, and use of deployment templates. The ASM-based deployment model for Cloud Services has been renamed Cloud Services (classic). Customers retain the ability to build and rapidly deploy web and cloud applications and services. Customers will be able to scale cloud services infrastructure based on current demand and ensure that the performance of applications can keep up while simultaneously reducing costs. The platform-supported tool for migrating existing cloud services to Cloud Services (extended support) also goes into preview. Migrating to ARM will allow customers to set up a robust infrastructure platform for their applications. 

Storage

Azure File Sync agent v12 

Improvements and issues that are fixed in the v12 release:

  • New portal experience to configure network access policy and private endpoint connections
    • You can now use the portal to disable access to the Storage Sync Service public endpoint and to approve, reject and remove private endpoint connections. To configure the network access policy and private endpoint connections, open the Storage Sync Service portal, go to the Settings section and click Network.
  • Cloud Tiering support for volume cluster sizes larger than 64KiB
  • Measure bandwidth and latency to Azure File Sync service and storage account
    • The Test-StorageSyncNetworkConnectivity cmdlet can now be used to measure latency and bandwidth to the Azure File Sync service and storage account. Latency to the Azure File Sync service and storage account is measured by default when running the cmdlet. Upload and download bandwidth to the storage account is measured when using the “-MeasureBandwidth” parameter. To learn more, see the release notes.
  • Improved error messages in the portal when server endpoint creation fails
    • We heard your feedback and have improved the error messages and guidance when server endpoint creation fails.
  • Miscellaneous performance and reliability improvements
    • Improved change detection performance to detect files that have changed in the Azure file share.
    • Performance improvements for reconciliation sync sessions.
    • Sync improvements to reduce ECS_E_SYNC_METADATA_KNOWLEDGE_SOFT_LIMIT_REACHED and ECS_E_SYNC_METADATA_KNOWLEDGE_LIMIT_REACHED errors.
    • Files may fail to tier on Server 2019 if Data Deduplication is enabled on the volume.
    • AFSDiag fails to compress files if a file is larger than 2GiB.

To obtain and install this update, configure your Azure File Sync agent to automatically update when a new version becomes available or manually download the update from the Microsoft Update Catalog.

More information about this release:

  • This release is available for Windows Server 2012 R2, Windows Server 2016 and Windows Server 2019 installations.
  • A restart is required for servers that have an existing Azure File Sync agent installation.
  • The agent version for this release is 12.0.0.0.
  • Installation instructions are documented in KB4568585.

Encryption scopes in Azure Storage

Encryption scopes introduce the option to provision multiple encryption keys in a storage account for blobs. Previously, customers using a single storage account for multi-tenancy scenarios were limited to using a single account-scoped encryption key for all the data in the account. With encryption scopes, you now can provision multiple encryption keys and choose to apply the encryption scope either at the container level (as the default scope for blobs in that container) or at the blob level. 

Azure Data Explorer external tables

An external table is a schema entity that references data stored outside the Azure Data Explorer database. Azure Data Explorer Web UI can create external tables by taking sample files from a storage container and creating schema based on these samples. You can then analyze and query data in external tables without ingestion into Azure Data Explorer.

Azure Governance: how to control system configurations in hybrid and multicloud environments

There are several companies that are investing in hybrid and multicloud technologies to achieve high flexibility, that enables you to innovate and meet changing business needs. In these scenarios, customers face the challenge of using IT resources efficiently, in order to best achieve your business goals, implementing a structured IT governance process. This can be achieved more easily if you have solutions that, in a centralized way, allow you to inventory, organize and enforce control policies on your IT resources wherever you are. Azure Arc solution involves different technologies with the aim of supporting hybrid and multicloud scenarios, where Azure services and management principles are extended to any infrastructure. In this article we will explore how, thanks to the adoption of the Azure Guest Configuration Policy it is possible to control the configurations of systems running in Azure, in on-premises datacenters or other cloud providers.

The principle behind Azure Arc

The principle behind Azure Arc is to extend Azure management and governance practices to different environments and to adopt typically cloud solutions, as DevOps techniques (infrastructure as code), also for on-premises and multicloud environments.

Figure 1 – Azure Arc overview

Enabling systems to Azure Arc

Enabling Azure Arc servers allows you to manage physical servers and virtual machines residing outside Azure, on the on-premises corporate network or at another cloud provider. This applies to both Windows and Linux systems. This management experience is designed to provide consistency with Azure native virtual machine management methodologies. In fact, connecting a machine to Azure through Arc is considered in all respects as an Azure resource. Each connected machine has a specific ID, is included in a resource group and benefits from standard Azure constructs such as Azure Policies and tagging.

To offer this experience, the installation of the specific Azure Arc agent is required on each machine that is planned to connect to Azure ("Azure Connected Machine"). The following operating systems are currently supported:

  • Windows Server 2008 R2, Windows Server 2012 R2 or higher (this includes core servers)
  • Ubuntu 16.04 and 18.04 LTS (x64)
  • CentOS Linux 7 (x64)
  • SUSE Linux Enterprise Server (SLES) 15 (x64)
  • Red Hat Enterprise Linux (RHEL) 7 (x64)
  • Amazon Linux 2 (x64)
  • Oracle Linux 7

The Azure Arc Connected Machine agent consists of the following logical components:

  • TheHybrid Instance Metadata service (HIMDS) that manages the connection to Azure and the Azure identity of the connected machine.
  • The Guest Configuration agent that provides in-guest policy and guest configuration features.
  • TheExtension Manager agent that manages installation processes, uninstalling and updating machine extensions.

Figure 2 – Azure Arc Agent Components

The Connected Machine agent requires secure outbound communication to Azure Arc on TCP port 443.

This agent provides no other features and does not replace the Azure Log Analytics agent, which remains necessary when you want to proactively monitor the operating system and workloads running on the machine.

For more information about installing Azure Arc, see this official Microsoft document.

Azure Arc-enabled servers can benefit from several Azure Resource Manager-related features such as Tags, Policies and RBAC, as well as some features related to Azure Management.

Figure 3 – Azure Management for all IT resources

Guest Configuration Policy di Azure

Guest Configuration Policies allow you to control settings within a machine, both for virtual machines running in Azure environment and for "Arc Connected" machines. Validation is performed by the client and by the Guest Configuration extension as regards:

  • Operating system configuration
  • Configuration or presence of applications
  • Environment settings

At the moment, most of the Azure Guest Configuration Policies only allow you to make checks on the settings inside the machine, but they don't apply configurations. The exception is a built-in time zone configuration policy operating system for Windows machines.

Requirements

Before you can check the settings inside a machine, through guest configuration policies, you must:

  • Enable a’extension on the Azure VM, required to download assigned policy assignments and corresponding configurations. This extension is not required for "Arc Connected" machines as it is included in the Arc agent.
  • Make sure that the machine has a system-managed identity, used for the authentication process when reading and writing to the guest configuration service.

Operation

Azure provides built-in specification platform Initiatives and a large number of Guest Configuration Policy, but you can also create custom one both in Windows environment, both in Linux environment.

Guest Configuration policy assignment works the same way as standard Azure Policies, so you can group them into initiative. Specific parameters can also be configured for Guest Configuration Policies and there is at least one parameter that allows you to include Azure Arc-enabled servers. When you have the desired policy definition, it is possible to assign it to a subscription and possibly in a more limited way to a specific Resource Group. You also have the option of excluding certain resources from the application of the policy.

Following the assignment, it is possible to assess the compliance status in detail directly from the Azure portal.

Inside the machine, the Guest Configuration agent uses local tools to audit the configurations:

The Guest Configuration agent checks for new or modified guest policy assignments each 5 minutes and once the assignment is received the settings are checked at intervals of 15 minutes.

The Cost of the Solution

The cost of Azure Guest Configuration Policies is based on the number of servers registered to the service and which have one or more guest configurations assigned. Any other type of Azure Policy that is not based on guest configuration is offered at no additional cost, including virtual machine extensions to enable services such as Azure Monitor and Azure Security Center or auto tagging policies. The billing is distributed on an hourly basis and also includes the change tracking features present through Azure Automation. For more details on costs please visit the Microsoft's official page.

Conclusions

IT environments are constantly evolving and often have to deliver business-critical applications based on different technologies, active on heterogeneous infrastructures and which in some cases use solutions provided in different public clouds. The adoption of a structured IT governance process is easier also thanks to the Guest Configuration Policies and the potential of Azure Arc, that allow you to more easily control and support hybrid and multicloud environments.