Microsoft recently announced the availability of the new managed service Azure Data Share, specially designed for sharing data between different organizations. Azure Data Share allows you to share and combine data in an easy way, in order to ensure effective business collaboration between different realities, in respect of safety and governance. In this article are covered the principles of operation and shows the procedure to be followed for the corresponding configuration.

To date, the most commonly used solutions for the exchange and sharing of corporate data are based on the File Transfer Protocol (FTP) or on custom Web API, which in addition to having to manage a specific infrastructure, are unable to adhere to corporate standards in terms of security and governance. Furthermore, these solutions are not suitable for the exchange of large volumes of data. Azure Data Share is a fully managed service with the aim of simplifying, secure and controlled the process of sharing data between different business realities. Is not required to configure any specific infrastructure and service can rapidly scale to meet big data sharing needs. Meanwhile, your safety is critical for a service of this kind and Azure Data Share exploits the main intrinsic safety measures in Azure for data protection.

Activation of the service

The activation of the service Azure Data Share must be carried out primarily by those who must share data, as follows.

Figure 1 - Starting the process of creation

Figure 2 - Parameters required by the creation process

The deployment is very fast and after activating the service you can begin the process of sharing data.

Using the solution

Azure Data Share has an intuitive interface, can be used directly from the Azure portal, and with a few simple steps you can choose what information to share and with whom to share them.

Figure 3 – Start the process of sharing

Figure 4 – Definition of the name and details of the share

It is possible to govern the use of the data by binding specific terms of use for each share that is created. To allow receiving data, recipients must accept the terms of use specified.

Figure 5 – Added the Dataset and the selection of the type

For data sharing today you can use as Dataset Azure Blob Storage and Azure Data Lake Storage, but soon will be introduced also new Azure data sources.

Figure 6 – Selection of the container that contains the data to be shared

Figure 7 - Added email address of the person you share files

The service also provides the ability to schedule the sharing of new content or changes, while maintaining complete control of any share.

Figure 8 - Scheduling of snapshots

Figure 9 - Review and creation of the share

The recipient will receive at the end of the sharing process a notification by email.

Figure 10 - Notification received via email from the person to whom you want to send data

Figure 11 - Invitation of Data Share from the Azure portal

One who receives the shared data must have a service Azure Data Share to see the shared data.

Figure 12 – Accept invitation to Date Share

In accepts the invitation you must specify the target, in this case the storage account, on which to bring the data received. Such content you can choose to keep it synchronized according to the scheduling specified by the sender.

Figure 13 - Configuration of the storage account target

Figure 14 - Detail of a receipt sharing

In the received shares it is also given the option to manually trigger a snapshot (full or incremental).

Conclusions

In a world where the amount, the type and variety of data is growing, you must have services that enable organizations to work together by sharing information of any kind and size. Thanks to the Azure Data Share you can share data, between different companies, in a simple way, safe and respecting the governance policies. Adding new datasets for data sharing, expected in the short term, will make this service even more complete and effective.

The Azure governance is possible thanks to a series of specially designed services to enable a management and a constant control of the various Azure resources on a large scale. Among these services are Resource Graph, a powerful tool that allows you to quickly obtain via command-line details regarding the different Azure artifacts. Using Resource Graph you can retrieve information that previously required necessarily complex and iterative scripting. This article lists the characteristics of the solution and how you can use it to find out the details of Azure resources on large scale.

Characteristics of the service

In the presence of complex Azure environments who see the presence of many subscriptions, maintain overall visibility of all Azure resources can be complex without the use of tools specifically developed. These requirements which typically need to be addressed:

• Ability to view resources and their properties in a transversal way among different subscriptions.
• Be able to efficiently perform queries on resources by setting filters, groupings and by imposing a specific sort order on their properties.
• Explore iteratively the different resources.
• Assessing the impact achieved by applying policies on a large number of cloud resources.

The service Azure Resource Graph allows, thanks to the use of an efficient and powerful language to perform the following actions:

• Query on resources by applying filters, complex groupings and sorts.
• Explore iteratively resources based on the governance requirements.
• Assess the impact given by the application of the policy in a vast cloud.
• Detailing the changes that are made to the Azure resource properties. Recently was introduced the ability to view the last 14 days of history regarding the changes made to resources, to identify which properties have been changed and when. This feature is particularly useful in the process of troubleshooting, to detect any change events in a specific time slot. Furthermore, it is functional to understand the properties that were changed when a resource has changed the status of compliance, to consider adopting Azure Policy to properly manage such properties. For further details please visit the Microsoft's official documentation.

All these actions provide important aspects in order to govern the most of their Azure environment.

When an Azure resource is updated, Resource Graph Resource Manager is notified by the relevant changes and updates its database accordingly. Resource Graph also regularly performs a complete scan of resources to ensure that your information is up to date in case of missing notifications or updates that take place outside of Resource Manager.

How to use Resource Graph

The query of Azure Resource Graph are based on the Kusto language, also used by Azure Data Explorer, Application Insights and Azure Log Analytics. For more details on using the query language of Azure Resource Graph you can see the Microsoft's official documentation, that shows how it is structured and what are the operators and supported features.

Resource Graph supports theAzure CLI, Azure PowerShell and Azure SDK for .NET. Querying Resource Graph requires the addition of the relative extension in the Azure CLI environment , while in Azure PowerShell the installation of the Resource Graph module is required. The queries are always structured in an identical manner, regardless of where they are performed.

The use of Resource Graph requires that the user with which it performs the query has at least read permissions, through Role-based access control (RBAC), about resources that you intend to query. If you don't have at least read permission on specific resources, queries will not return results related to them.

The service Azure Resource Graph is also used when performing research in the search bar of the Azure portal, in the new list of resources (‘All resources’) and in the change history of Azure Policy.

Figure 1 – Experience of ‘All resources’ using Azure Resource Graph

Sample Query

Below are some examples of query of Resource Graph and its result.

Figure 2 - Query to count by resources type (Resource Type)

Figure 3 - Query to count the resources by geographic location

The query of Azure Resource Graph have the advantage that besides being able to achieve the desired result in a simple way, they are also very performant:

Figure 4 - Running time of the query to count the resources based on location

If anyone wanted to achieve this result using the classic PowerShell method in complex Azure environments, should join to the single Azure subscription, search the necessary information and move to the next subscription. This approach was the only possible until the arrival of Resource Graph, but it was more labor intensive and much less powerful.

Figure 5 - List of VMs with the OS disk not Managed

Conclusions

Azure Resource Graph allows you to quickly and efficiently explore and analyze Azure resources, allowing to maintain a total visibility even on particularly complex Azure environments, consisting of several subscriptions, each of which with a large number of elements. Particularly useful is the functionality that allows you to see the history of changes to Azure resources. Azure Resource Graph is a tool that allows you to make a significant contribution for the governance of the Azure environment.

In the presence of environments with a high number of Azure subscriptions it is necessary to have a different level of abstraction in order to effectively manage the accesses, policies and compliance. To this end, Azure Management Groups have been introduced, that allow to organize different subscriptions into logical containers, on which define, implement and verify government policies needed. This article examines in detail the concepts and shows directions to better organize the Azure resources in order to facilitate the process of governance.

To effectively organize Azure resources is fundamental to define a hierarchy of management groups and subscriptions to which you can apply Azure Policy, the service that allows you to create, assign and manage audit policy. The use of the Management Group is also helpful to effectively manage the assignment of permissions via role-based access control (RBAC), for administrative delegation.

Figure 1 - Example of a hierarchy of Management Groups

Each resource Azure is contained within a specific Azure Subscription, which it is associated to a single Azure Active Directory tenant, and inherits the permissions set at that level.

At the moment, a constraint to consider, is that a Management Group can contain multiple subscriptions as long as the same form part of the same Azure Active Directory tenant . In other words,, the Management Groups reside within a tenant and cannot contain subscriptions of different tenants. The security principal that can be used on management group can come only by the tenant for the management group.

Figure 2 – Relationship between Azure AD and organizational structure

Azure resources belonging to a subscription are contained in Resource Groups. The resource groups are containers of resources that, for administrative purposes, allow to obtain the following benefits:

• They facilitate administrative delegation because resources contained inherit permissions to the resource group level.
• On the resource group you can set tags, although these are not automatically inherited by the resources, but it is necessary to foresee specific mechanisms if they are deemed necessary.

Figure 3 – Relationship between the levels of the organisational structure

The Azure Policy can be assigned to a subscription or Management Group level and can be defined exceptions for Resource Group. In this regard it is recommended whenever possible, to organize policies in "initiative" and assign them to Management Group level.

The root Management Group is the top level and contains all configured Management Groups and various Azure subscriptions. Root Management Group cannot be removed or moved. The structure can be created with up to six levels deep, without considering the Root level and the level of subscription. Each Management Group can have more children, but supports only one parent for each Management Group and for each subscription.

In the absence of specific requirements, Microsoft recommends that you split production environments than those "DevTest", creating two tiers of management groups. The management group root by default will have fundamental policies, such as those relating to security. On the remaining Management Groups are associated specific policies. The hierarchy of Management Groups provide a model for which the policies that are defined at higher levels in the hierarchy cannot be overwritten by lower levels.

Figure 4 – Management Group & Subscription Modeling Strategy

This approach enables you to manage complex Azure environments, who see the presence of more subscriptions. in a more simple and flexible way, for the following reasons:

• The concept of inheritance allows with a single association to apply the desired controls and the assignment of roles on different subscriptions.
• It has a centralized management.
• You may include additional subscription in the hierarchy, with the knowledge that will adhere to established policies and who will have the assignment of desired roles.

Conclusions

The goverance processes by which an organization can ensure an effective and efficient use of IT resources, in order to achieve their goals, cannot refrain from adopting a model that allows to organize effectively the Azure resources. The use of Management Groups, in environments with a significant number of subscriptions, is essential to meet the common need of standardize, and in some cases impose, how you configure the different resources in the cloud.

One of the main features of the cloud is the ability to create new resources with ease and speed. At the same time an important and fundamental challenge is to be able to keep under control the expenses to be incurred for the resources created in the cloud. The tool Azure Cost Management makes it easy to find from which services are generated costs, prevent unnecessary costs and optimize resource costs. This article lists the characteristics of the solution and provides guidance in order to best use it for maximize and optimize investments in cloud resources.

Features of the solution

Azure Cost Management is enabled by default and accessible from Azure portal for all Microsoft Enterprise Agreement and Pay-As-You-Go subscriptions. The availability of the solution for CSP subscriptions (Cloud Solution Providers) is scheduled for the second half of the year. Azure Cost Management contemplates the Azure services, including reservations, and cost data by the use of third-party solutions coming from the Azure Marketplace. All costs shown are based on negotiated prices and the data is updated every four hours.

Azure Cost Management allows you to do the following regarding the cost of cloud resources.

Monitor the cloud costs

With this solution you can provide, to the various departments involved in the use of cloud resources, cost visibility of resources for which they are responsible. Furthermore, you have the option to go into detail and view the trend of costs with a very intuitive and interactive experience. From section Cost analysis you can view the costs, with a chance to put filters on time period and eventually group them according to different parameters.

Figure 1 – Cost analysis – costs accumulated in the last month

Setting for the selected period the daily granularity you can have a graph that shows precisely the day-by-day costs.

Figure 2 – Cost analysis – costs incurred daily

Azure Cost Management also offers the possibility to export the data to Excel or CSV, after setting the required view in section Cost analysis.

Figure 3 -Export the data view created in Cost analysis

The downloaded file contains details about the context used for file generation:

Figure 4 – Summary of Excel sheet created

This feature can be useful for detailed analysis that require a consolidation of this information with other data.

In case there is a need for more functionality in terms of integration and customization you can use Power BI connectors for the creation of specific dashboards and Azure Cost Management APIs to process information with other solutions.

Assignment of responsibility to the various project teams

To raise awareness of the various project teams by making adequate use of resources in terms of spending, you can define your budget. So you can get a significant cost optimization, with no impact in terms of agility in creating resources .

Figure 5 – Creating a budget

When creating a budget it is defined a spending for a certain period of time and you can set the alert to warn those directly responsible when it reaches a certain percentage of the preset threshold.

Optimization of costs

Although you may opt to manage the various Azure resource costs to team by giving them a budget, you can not always be assumed to know how to optimize resource efficiency and reduce costs. Azure Cost Management is able to provide specific recommendations to achieve cost savings.

Figure 6 – Advisor recommendations

In the specific case, to optimize Azure resource costs , it is recommended the purchase ofVirtual Machine Reserved Instances (VM RIs), estimating the annual savings that could be obtained by adopting VM RIs.

Azure Cost Management will soon be enriched with a new feature (currently in public preview) involving the management of costs incurred in the AWS, with the same characteristics as shown for Azure. This integration simplifies management of costs in multi-cloud scenarios.

The Cost of the Solution

You can use Azure Cost Management for free, in all its features, for the Azure environment. As for the management of AWS costs is expected, in the final release, a charge equal to 1% of total spend managed for AWS. For more details on the cost of the solution you can consult the pricing of Cost Management.

Conclusions

Azure Cost Management is a great tool that allows you to maintain complete visibility of costs and to drive you to have a better manage of the expenses in the cloud resources. Thanks to the flexibility of this tool, constantly evolving, you can get the most from the investment in the cloud easily and intuitively.

IT governance is the process through which an organization can ensure an effective and efficient use of IT resources, in order to achieve their goals. Azure Policy is a service available in the Microsoft public cloud that can be used to create, assign and manage policies to control the resources in Azure. Azure Policy, natively integrated into the platform, are a key element for the governance of the cloud environment. In this article, the principles of operation and features of the solution are reported.

In Azure environments you can find different subscriptions on which develop and operate several groups of operators. The common requirement is to standardize, and in some cases impose, how to configure the resources in the cloud. All this is done to obtain specific environments that meet compliance regulations, monitor security, resource costs and standardize the design of different architectures. These goals can be achieved with a traditional approach, that includes a block of operators (Dev/Ops) in the direct access to cloud resources (through the portal, API or cli). This traditional approach is, however, inflexible, because it involves a loss of agility in controlling the deployment of resources. Instead, using the mechanism that comes natively from Azure platform is possible to drive the governance to achieve the desired control, without impacting speed, the basic element in the operations of the modern IT.

Figure 1 – Traditional approach vs cloud-native governance

In Azure Policy you can do the following reported:

• Enable built-in policy or configure them to meet your needs.
• Perform real-time evaluation of the criteria set out in the policy and force execution.
• Assess the compliance of the policy periodically or on request.
• Enable audit policy on the virtual machine guest environment (Vm In-Guest Policy).
• Apply policies on Management Groups in order to gain control over the entire organization.
• Apply multiple criteria and aggregate the various states of the policies.
• Configure scope over which the exclusions are applied.
• Enable real-time remediation steps, also for existing resources.

All this translates into the ability to apply and enforce policy compliance on a large scale and its remediation actions.

The working mechanism of the Azure Policy is simple and integrated into the platform. When a request is made for an Azure resource configuration using ARM, this is intercepted by the layer containing the engine that performs the evaluation of policy. This engine makes an assessment based on active Azure policies and establishes the legitimacy of the request.

Figure 2 – Working principle of Azure Policy in creating resources

The same mechanism is then repeated periodically or upon request, to assess the compliance of existing resources.

Figure 3 – Working principle of Azure Policy in resource control

Azure already contains many built-in policies ready to be applied. Furthermore, in this GitHub repository you can find different definitions of Azure Policies, that can be used directly or modified to suit your needs. The definition of the Azure Policy is made in JSON and follows a well defined structure, described in this Microsoft's document. You also have the possibility of creating Initiatives, they are a collection of multiple policies.

Figure 4 – Example of defining a Azure Policy

When you have the desired policy definition, it is possible to assign it to a subscription and possibly in a more limited way to a specific Resource Group. You also have the option of excluding certain resources from the application of the policy.

Figure 5 – Process of assigning an Azure Policy

Following the assignment, you can evaluate the State of compliance in detail and if it is necessary apply remediation actions.

Figure 6 – State of compliance

Figure 7 -Example of remediation action

Conclusions

Through the use of Azure Policy you can totally control your own Azure environment, in a simple and efficient way. Statistics provided by Microsoft cite that considering the 100 top Azure Customers, 92 these use Azure Policy to control their environment. This is because, when you increase the complexity and amount of services on Azure is essential to adopt instruments, as Azure Policy, to have effective governance policies.