Category Archives: Azure Storage

Azure IaaS and Azure Stack: announcements and updates (April 2021 – Weeks: 13 and 14)

This series of blog posts includes the most important announcements and major updates regarding Azure infrastructure as a service (IaaS) and Azure Stack, officialized by Microsoft in the last two weeks.

Azure

Compute

Virtual machine (VM) level disk bursting available on all Dsv3 and Esv3 families

Virtual machine level disk bursting allows your virtual machine to burst its disk IO and MiB/s throughput performance for a short time daily. This feature is now enabled on all our Dsv3-series and Esv3-series virtual machines, with more virtual machine types and families support soon to come. There is no additional cost associated with this new capability or adjustments on the VM pricing and it comes enabled by default.

Cloud Services (extended support) is generally available

Cloud Services (extended support), which is a new Azure Resource Manager (ARM)-based deployment model for Azure Cloud Services, is generally available. Cloud Services (extended support) has the primary benefit of providing regional resiliency along with feature parity with Azure Cloud Services deployed using Azure Service Manager (ASM). It also offers some ARM capabilities such as role-based access and control (RBAC), tags, policy, private link support, and use of deployment templates. The ASM-based deployment model for Cloud Services has been renamed Cloud Services (classic). Customers retain the ability to build and rapidly deploy web and cloud applications and services. Customers will be able to scale cloud services infrastructure based on current demand and ensure that the performance of applications can keep up while simultaneously reducing costs. The platform-supported tool for migrating existing cloud services to Cloud Services (extended support) also goes into preview. Migrating to ARM will allow customers to set up a robust infrastructure platform for their applications. 

Storage

Azure File Sync agent v12 

Improvements and issues that are fixed in the v12 release:

  • New portal experience to configure network access policy and private endpoint connections
    • You can now use the portal to disable access to the Storage Sync Service public endpoint and to approve, reject and remove private endpoint connections. To configure the network access policy and private endpoint connections, open the Storage Sync Service portal, go to the Settings section and click Network.
  • Cloud Tiering support for volume cluster sizes larger than 64KiB
  • Measure bandwidth and latency to Azure File Sync service and storage account
    • The Test-StorageSyncNetworkConnectivity cmdlet can now be used to measure latency and bandwidth to the Azure File Sync service and storage account. Latency to the Azure File Sync service and storage account is measured by default when running the cmdlet. Upload and download bandwidth to the storage account is measured when using the “-MeasureBandwidth” parameter. To learn more, see the release notes.
  • Improved error messages in the portal when server endpoint creation fails
    • We heard your feedback and have improved the error messages and guidance when server endpoint creation fails.
  • Miscellaneous performance and reliability improvements
    • Improved change detection performance to detect files that have changed in the Azure file share.
    • Performance improvements for reconciliation sync sessions.
    • Sync improvements to reduce ECS_E_SYNC_METADATA_KNOWLEDGE_SOFT_LIMIT_REACHED and ECS_E_SYNC_METADATA_KNOWLEDGE_LIMIT_REACHED errors.
    • Files may fail to tier on Server 2019 if Data Deduplication is enabled on the volume.
    • AFSDiag fails to compress files if a file is larger than 2GiB.

To obtain and install this update, configure your Azure File Sync agent to automatically update when a new version becomes available or manually download the update from the Microsoft Update Catalog.

More information about this release:

  • This release is available for Windows Server 2012 R2, Windows Server 2016 and Windows Server 2019 installations.
  • A restart is required for servers that have an existing Azure File Sync agent installation.
  • The agent version for this release is 12.0.0.0.
  • Installation instructions are documented in KB4568585.

Encryption scopes in Azure Storage

Encryption scopes introduce the option to provision multiple encryption keys in a storage account for blobs. Previously, customers using a single storage account for multi-tenancy scenarios were limited to using a single account-scoped encryption key for all the data in the account. With encryption scopes, you now can provision multiple encryption keys and choose to apply the encryption scope either at the container level (as the default scope for blobs in that container) or at the blob level. 

Azure Data Explorer external tables

An external table is a schema entity that references data stored outside the Azure Data Explorer database. Azure Data Explorer Web UI can create external tables by taking sample files from a storage container and creating schema based on these samples. You can then analyze and query data in external tables without ingestion into Azure Data Explorer.

Azure Security Center: Azure Storage protection

Azure Security Center, the cloud solution that allows you to prevent, detect and respond to security threats affecting hybrid architectures, it also provides enhanced protection for storage resources in Azure. The solution detects unusual and potentially harmful attempts to access or use Azure Storage. This article describes how to effectively protect storage in Azure with this solution, looking at the news recently announced in this area.

Azure Security Center (ASC) is possible to activate it in two different tiers:

  • Free tier. In this tier ASC is totally free and performs a continuous assessment, providing recommendations relating to the security of the Azure environment.
  • Standard tier. Compared to tier free adds enhanced threat detection, using behavioral analysis and machine learning to identify zero-day attacks and exploits. Through machine learning techniques and through the creation of whitelist is possible to control the execution of applications to reduce exposure to network attacks and malware. Furthermore, the standard level adds the ability to perform in an integrated manner a Vulnerability Assessment for virtual machines in Azure. Azure Security Center Standard supports several resources including: VMs, Virtual machine scale sets, App Service, SQL servers, and Storage accounts.

Advanced Threat Protection (ATP) for Azure Storage, it is one of several features in Azure Security Center Standard.

Figure 1 – Comparison of the features of the different tiers of ASC

Enabling the Security Center Standard tier is strongly recommended to improve security postures in your Azure environment.

The Advanced Threat Protection feature (ATP) for Azure Storage was announced last year, allowing you to detect common threats such as malware, access from suspicious sources (including TOR nodes), data exfiltration activities and more, but all limited to blob containers. Support for Azure Files and Azure Data Lake Storage Gen2 has also been included recently. This also helps customers protect data stored in file shares and data stores designed for the analysis of corporate big data.

Enabling this feature from the Azure portal is very simple and can be done at the Security Center-protected subscription level or selectively on individual storage accounts.

To enable this protection on all storage accounts in your subscription, you must go to the "Pricing & Settings” of Security Center and activate the protection of Storage Accounts.

Figure 2 – ATP activation for Azure Storage at the subscription level

If you prefer to enable it only on certain storage accounts, you need to activate it in the respective settings of Advanced security.

Figure 3 – ATP activation on the single storage account

When anomaly occurs on a storage account, security alerts are sent by email to Azure subscription administrators, with details of detected suspicious activity and related recommendations on how to investigate and resolve threats.

Details included in the event notification include::

  • The nature of the anomaly
  • The name of the storage account
  • The time of the event
  • The type of storage
  • Potential causes
  • The recommended steps to investigate what has been found
  • The actions to be taken to remedy what happened

Figure 4 – Example of a security alert sent in the face of a detection of a threat

In this example, the EICAR test file was used to validate that the solution was working correctly.. This is a file developed by the’European Institute for Computer Anti-Virus Research (EICAR) which is used to securely validate security solutions.

Security alerts can be viewed and managed directly from Azure Security Center, where details and actions to investigate current threats and address future threats are displayed..

Figure 5 – Example of a security alert in the ASC Security alerts tile

To get the full list of possible alerts generated by unusual and potentially malicious attempts to log in or use storage accounts, you can access the Threat protection for data services in Azure Security Center.

This protection is very useful even if you have architecture that uses the service Azure File Sync (AFS), which allows you to centralize the network folders of your infrastructure in Azure Files.

Conclusions

Business companies are increasingly moving their data to the cloud, looking for distributed architecture, high performance and cost optimization. All features offered by the public cloud require you to strengthen cybersecurity, particularly given the increasing complexity and sophistication of cyberattacks. By adopting Advanced Threat Protection (ATP) for Azure Storage, you can increase the level of storage security used in your Azure environment easily and effectively.

Protect Azure File Sync through Azure Backup

Azure File Sync service allows you to centralize your infrastructure's network folders in Azure Files, allowing you to maintain the typical characteristics of a file server on-premises, in terms of performance, compatibility and flexibility and at the same time to benefit from the potential offered by cloud. Azure File Sync integrates with Azure Backup making it possible to centrally manage protection policies in the cloud. This article describes how these two solutions are integrated and what you need to consider to enable effective protection.

The main features of Azure File Sync are the following:

  • Cloud tiering: are maintained locally only recently accessed data.
  • Multi-site sync: you have the option to sync between different sites, allowing write access to the same data between different Windows Servers and Azure Files.
  • Integration with Azure backup: ability to enable content protection using Azure Backup.
  • Disaster recovery: you have the option to immediately restore metadata files and retrieve only the data you need, for faster service reactivation in Disaster Recovery scenarios.
  • Direct access to the cloud: you can directly access content on the File share from other Azure resources (IaaS and PaaS).

Azure File Sync can turn Windows Server into a "cache" to quickly access content on a given Azure File share. Local access to data can occur with any protocol available in Windows Server. You have the possibility to have multiple "cache" servers in different geographic locations.

The ability to enable the Cloud Tiering makes Azure File Sync an increasingly popular solution, but this aspect in particular requires you to make the necessary considerations in the strategy to be adopted for data protection. As well as antivirus solutions, backup solutions may cause files stored in the cloud to be recalled through the Cloud Tiering feature. Microsoft recommends a cloud backup solution to back up Azure File share instead of an on-premises backup solution. If you are using a local backup solution, backups must be performed on a server belonging to a sync group where cloud tiering is disabled.

How the backup job works

Azure File share security is done under the following architecture:

Figure 1 – Architecture for securing Azure File share

The Azure File Share security process involves the following steps::

  1. The presence of a Recovery Service Vault is required in order to configure backups. Therefore, you should proceed with the creation of it if it is not available.
  2. Azure Backup will perform a discovery required to complete the process of registering the storage account that hosts the Azure File shares to be protected.
  3. Completed the registration process, Azure Backup will store the list of File shares present on the storage account in its catalog.
  4. You can select the Azure File share to protect and associate them with its backup policies, specific scheduling and data retention policies.
  5. Based on the policies configured Azure Backup performs backups. A key aspect to consider is that the backup is currently being backed up by generating a snapshot of the Azure File share. Data in Azure File share are never transferred to the Recovery Service Vault, but Azure Backup simply creates and manages snapshots that are part of the storage account.
  1. In the event of a restore, snapshots will be used, the relative URL of the backups, is taken from the metadata store in the Recovery Service Vault.
  2. The backup and restore job monitor is sent to the Azure Backup Monitoring service. This allows you to get an overall view of all backups, including Azure File Share. Furthermore, you can also configure alerts or e-mail notifications if you have problems performing backups.

 

Benefits of adopting this security strategy

  • Zero infrastructure: no infrastructure is required to enable environmental protection.
  • Customizing retention policies: backups can be configured with data retention policies daily, weekly, monthly and yearly, based on your needs. Annual backups can now be kept up to 10 years.
  • Built-in management capabilities: you can schedule your backups and specify the retention period you want in a way that is fully integrated into the platform.
  • Instant Restore: Azure File Share backup uses snapshots, this allows you to select only the files you want to restore instantly.
  • Alerts and reports: you can configure alerts for backup and restore operations that present errors. You can also use the reporting solution provided by Azure Backup to get detailed information about backup jobs.

Protect against accidental deletion of Azure File shares

To provide greater protection against cyberattacks and accidental deletion, Azure Backup recently added an extra layer of security to the Azure File shares snapshot management solution. If you delete the File shares, content and its recovery points (Snapshots) are retained for a configurable period of time, enabling full recovery without data loss. When you configure protection for a File share, Azure Backup enables soft-delete functionality at the account storage level with a retention period of 14 days, which is configurable according to your needs. This setting determines the time window in which File Share content and snapshots can be restored after any accidental deletion operations. Once the File share is restored, backups resume working without the need for additional configurations.

Conclusions

This solution allows in very simple, reliable and secure way to configure protection for Azure File shares and easily recover data when needed. The integration between Azure File Sync and Azure Backup will surely see the release of several new features in the coming months, including, very much heard, the ability to configure data transfer to the Recovery Service Vault instead of keeping snapshots in the same storage account where the data resides. To understand all the support scopes and limits in using the Azure Backup service to protect Azure File shares, you can see this Microsoft article.