Category Archives: Log Analytics

Azure Application Gateway: monitoring with Log Analytics

Azure Application Gateway is an application load balancer (OSI layer 7) for web traffic, available in Azure environment, that manages HTTP and HTTPS traffic of the applications. This article is discussed how to monitor of Azure Application Gateway using Log Analytics provides.

Figure 1 - Azure Application Gateway basic schema

Using the Azure Application Gateway you can take advantage of the following features:

  • URL-based routing
  • Redirection
  • Multiple-site hosting
  • Session affinity
  • Secure Sockets Layer (SSL) termination
  • Web application firewall (WAF)
  • Native support for WebSocket and HTTP/2 protocols

For more details on Azure Application Gateway can be found in the Microsoft's official documentation.

Configuring Diagnostics logs for the Application Gateway

The Azure Application Gateway can send diagnostic logs to a workspace of Log Analytics . This feature is very useful for checking the performance, to detect any errors and is essential for troubleshooting steps, in particular in the presence of the WAF module. To enable the diagnostic from the Azure portal you can select the Application Gateway resource and go to the "Diagnostics logs":

Figure 2 – Starting configuration of Diagnostics logs

Figure 3 – Configuring Diagnostics logs

After choosing your Log Analytics workspace where to send diagnostics data, in the Log section, you can select which type of log collecting among the following:

  • Access log (ApplicationGatewayAccessLog)
  • Performance log (ApplicationGatewayPerformanceLog)
  • Firewall log (ApplicationGatewayFirewallLog): these logs are generated only if the Web Application Firewall is configured on the Application Gateway.

In addition to these logs are also collected by default Activity Log generated by Azure. These logs are maintained for 90 days in the store of the Azure event logs. For more details you can refer this specific document.

Azure Application Gateway analytics solution of Log Analytics

Microsoft offers the solution Azure Application Gateway analytics that can be added to the workspace of Log Analytics by following these simple steps:

Figure 4 - Launching the procedure of adding the solution to the OMS workspace

Figure 5 – Selection of the Azure Application Gateway analytics solution

Figure 6 - Addition of the solution in the selected workspace

After enabling the sending of diagnostics logs into the workspace of Log Analytics and adding the solution to the same, by selecting the tile Azure Application Gateway analytics in the Overview page, you can see an overview of the collected log data from the Application Gateway:

Figure 7 – Screen overview of the Azure Application Gateway analytics solution

You can also view the details for the following categories.

  • Application Gateway Access logs:
    • Client and server errors for Application Gateway access logs
    • Requests per hour for each Application Gateway
    • Failed requests per hour for each Application Gateway
    • Errors by user agent for Application Gateways

Figure 8 - Screenshot of the Application Gateway Access logs

  • Application Gateway performance:
    • Host health for Application Gateway
    • Maximum and 95th percentile for Application Gateway failed requests

Figure 9 – Screenshot of the Application Gateway performance

Customized dashboard of Log Analytics for the Application Gateway monitor

In addition to this solution can also be convenient to use a special dashboard of Log Analytics, specifically for the monitoring of the Application Gateway, available at this link. The deployment of the dashboard is via ARM template and requires also in this case the Diagnostics logs of the Application Gateway enabled, as described above. The various queries of Log Analytics, used by the dashboard, are documented in this blog. Thanks to these queries the dashboard shows several additional information exposed by the diagnostic of the Application Gateway.

Figure 10 – Custom dashboard of Log Analytics for Application Gateway monitoring

Query of Log Analytics to monitor the Firewall Log

Using the solution Azure Application Gateway analytics of Log Analytics or the custom dashboard (stated in the previous paragraph) are not contemplated at the time the Firewall log, generated when is active the Web Application Firewall (WAF) on the Application Gateway. The WAF is based on rules of OWASP Core Rule Set 3.0 or 2.2.9 to intercept attacks, for the web applications, that exploit the known vulnerabilities. To name a few, we find for example the SQL injection and attacks cross site scripting.

In this case, if you decide to check the Firewall log, you must directly query the Log Analytics, for example:

Figure 11 – The Query to retrieve blocked requests by the WAF module, over the past 7 days, for a specific URI, divided by RuleID

To see the list of rules of the WAF, by associating the RuleId to its description, you can consult this document.

The descriptive message of the rule is also listed within the results returned by the query:

Figure 12 – The Query to retrieve blocked requests by the WAF module, over the past 7 days, for a specific URI and for a specific RuleId

Conclusions

In my experience, in Azure architectures that require secure publishing of web services to Internet, is often used Azure Application Gateway service with the WAF module active. With the ability to send diagnostic logs of this component to Log Analytics you have the option of having a qualified monitor, that is fundamental to analyse any error conditions and to assess the state of the component in all its facets.

Microsoft Azure: network monitoring solutions overview

Microsoft Azure provides several solutions that allow you to monitor network resources, not only for cloud environments, but even in the presence of hybrid architectures. That are cloud-based features, to check the health of your network and connectivity to your applications. In addition, they give detailed information about network performance. This article will be made an overview of the various solutions such as the main features, needed to orient the use of the network monitor tools most appropriate for your needs.

Network Performance Monitor (NPM) is a suite that includes the following solutions:

  • Performance Monitor
  • ExpressRoute Monitor
  • Service Endpoint Monitor

In addition to the tools included in the Network Performance Monitor (NPM) you can use Traffic Analytics and DNS Analytics.

Performance Monitor

The most commonly used approach is to have hybrid environments with heterogeneous networking, that allows you to connect your own on-premises infrastructure with the environment implemented in the public cloud. In some cases you may also have different cloud providers, that make the network infrastructure even more complicated . These scenarios require the use of flexible monitor tools that can work across on-premises, in cloud (IaaS), and in hybrid environments. Performance Monitor has all of these characteristics and thanks to the use of synthetic transactions, provides the ability to monitor, almost in real time, the network parameters to get performance information, like packet loss and latency. In addition, this solution allows to easily locate the source of a problem in a specific network segment or identifying a particular device. The solution requires the presence of the OMS agent and keeping track of the retransmission packets and the roundtrip time, it is able to return a graph of easy and immediate interpretation.

Figure 1 - Hop-by-hop chart provided by Performance Monitor

Where to install the agents

The installation of the agent of Operations Management Suite (OMS) is necessary on at least one node connected to each subnet from which it intends to monitor the connectivity to other subnets. If you plan to monitor a specific network link you must install agents on both endpoints of the link. In cases where you do not know the exact network topology, one possible approach is to install agents on all servers that hold critical workloads and for which you need to monitor your network performance.

The Cost of the Solution

The cost of the feature Performance Monitor in NPM is calculated on the basis of the combination of these two elements:

  • Monitored Subnet link. To obtain the costs for monitoring of a single subnet link for one month, you can see Ping Mesh.
  • Data volume.

For more details please visit the Microsoft's official page.

ExpressRoute Monitor

Using ExpressRoute Monitor it is possible to monitor the end-to-end connectivity and verify the performance between on-premises environment and Azure, in the presence of ExpressRoute connectivity with Azure Private peering and Microsoft peering connections. The key features of this solution are:

  • Auto-detection of the circuit ExpressRoute associated with your subscription Azure.
  • Detection of network topology.
  • Capacity planning and bandwidth usage analysis.
  • Monitoring and alerting both the primary and the secondary path of the circuit ExpressRoute.
  • Monitoring connectivity towards the Azure services such as Office 365, Dynamics 365 using ExpressRoute as connectivity.
  • Detection of possible deterioration of connectivity with the various virtual network.

Figure 2 – Topology view of a VM on Azure (left) connected to a VM on-prem (right), via ExpressRoute

Figure 3 - Trend on the use of the bandwidth and latency on the ExpressRoute circuit

Where to install the agents

In order to use ExpressRoute Monitor you need to install an Operations Management Suite agent on a system that resides on Azure virtual network and at least one agent on a machine attested on the subnet on-premises, connected via private peering of ExpressRoute.

The Cost of the Solution

The cost of ExpressRoute Monitor solution is calculated based on the volume of data generated during the monitoring operations. For more details please visit the specific section in the cost page of NPM .

Service Endpoint Monitor

Using this solution, you have the ability to monitor and test the reachability of your services and your applications, almost in real time, simulating user access. You also have the ability to detect network side performance problems and identify the problematic network segment.

Here are reported the main features of the solution:

  • It does the monitor end-to-end of the network connections to your applications. The monitor can be done by any endpoint "TCP-capable" (HTTP, HTTPS, TCP, and ICMP), as websites, SaaS applications, PaaS applications, and SQL databases.
  • It correlates application availability with network performance, to precisely locate the degradation point on the network, starting from the user's request until the application.
  • It tests applications reachability from different geographical location .
  • It determines the network latencies and lost packets to reach the applications.
  • It detects hot spots on the network that can cause performance problems.
  • It does the monitor of the availability of applications Office 365, through specific built-in test for Microsoft Office 365, Dynamics 365, Skype for Business and other Microsoft services.

Figure 4 - Creating of a Service Connectivity Monitor test

Figure 5 – Diagram showing the topology of the network, generated by different nodes, for a Service Endpoint

Where to install the agents

To use Service Endpoint Monitor you must install the Operations Management Suite agent on each node where you want to monitor network connectivity to a specific service endpoint.

The Cost of the Solution

The cost for using Service Endpoint Monitor is based on these two items:

  • Number of connections, where the connection is understood as reachability test of a single endpoint, from a single agent, for the entire month. In this regard you can see Connection Monitoring in the cost page.
  • Volume of data generated by the monitor. The cost is obtained from cost page of Log Analytics, in the section Data Ingestion.

Traffic Analytics

Traffic Analytics is a totally cloud-based solution, allowing you to have an overall visibility on network activities that are undertaken in the cloud environment. In Azure to allow or deny network communication to the resources connected with Azure Virtual Networks (vNet) it uses the Network Security Group (NSG), containing a list of access rules. The NSGs are applied to network interfaces connected to the virtual machines, or directly to the subnet. The platform uses NSG flow logs to maintain the visibility of inbound and outbound network traffic from the Network Security Group. Traffic Analytics is based on the analysis of NSG flow logs and after an appropriate aggregation of data, inserting the necessary intelligence concerning security, topology and geographic map, can provide detailed information about the network traffic of your Azure cloud environment.

Using Traffic Analytics you can do the following:

  • View network activities cross Azure subscriptions and identify hotspots.
  • Intercept potential network security threats, in order to take the right remedial actions. This is made possible thanks to the information provided by the solution: which ports are open, what applications attempt to access to Internet and which virtual machines connect to unauthorized networks.
  • Understand network flows between different Azure regions and Internet, in order to optimize their deployment for network performance and capacity.
  • Identify incorrect network configurations that lead to having incorrect communication attempts.
  • Analysis of the VPN gateway capabilities or other services, to detect problems caused by over-provisioning and underutilization.

Figure 6 – Traffic Analytics overview

Figure 7 - Map of Active Azure Regions on the subscription

DNS Analytics

DNS Analytics solution is able to collect, analyze and correlate logs of DNS and provides administrators the following features:

  • Identifies clients that try to resolve domains considered malevolent.
  • Finds records that belong to obsolete resources.
  • It highlights domain names frequently questioned.
  • View the load of requests received by the DNS server.
  • It does the monitor of dynamic DNS registrations failed.

Figure 8 – Overview of DNS Analytics solution

Where to install the agents

The solution requires the presence of the OMS agent or the Operations Manager agent installed on each DNS server to be monitored.

Conclusions

With increasing complexity of network architectures in hybrid environments, consequently increases the need to be able to use tools able to contemplate different network topologies. Azure provides several cloud based tools and integrated into the fabric, such as those described in this article, that allow you to fully and effectively monitor the networking of these environments. Remember to test and evaluate free Operations Management Suite (OMS) you can access this page and select the mode that is most appropriate for your needs.

OMS and System Center: What's New in June 2018

In June have been announced, by Microsoft, a considerable number of news about Operations Management Suite (OMS) and System Center. Our community, through these articles released monthly, aims to provide a general overview of the main new features of the month, in order to stay up to date on these arguments and have the necessary references for further information.

Operations Management Suite (OMS)

Log Analytics

Recently it was officially announced that the OMS portal will be deprecated, in favour of the Azure Portal. In this article are examined the aspects related to this change and what you should know to avoid being caught unprepared.

Figure 1 - Notifications in the OMS portal

Azure Backup

Azure Backup is enriched with an important new feature that allows you to natively protect SQL workload, running in IaaS virtual machines that reside in Azure. In this article are showed the benefits and the characteristics of this new feature.

Figure 2 – Protection of SQL Server on Azure VMs with Azure Backup

Released an updated version of the’Azure Backup agent (MARS), which can be obtained by accessing this link.

Using Azure Backup there is the possibility of generating the reports needed to be able to easily check the status of resource protection, details on the different backup jobs configured, the actual storage utilization and status of its alert. All this is made possible by using Power BI, allowing you to have a high degree of flexibility in the generation and customization of reports. In this video, recently published, there is show how to configure a Power BI workspace for sharing reports of Azure Backup within your organization. To analyze the steps required to configure the reporting of Azure Backup you can refer this article.

Figure 3 – Sharing PowerBI reports of Azure Backup

Azure Backup introduces the ability to protect workloads running on Azure Stack environment. The tenant who use the Azure Stack solution can then have a short term protection directly on the Azure Stack environment and can make use of Azure Recovery Service vault for long term retention and to perform offsite. For more details on this you can consult therelease announcement.

Figure 4 – Azure Stack Tenant backup with Microsoft Azure Backup Server

Azure Site Recovery

In Azure Site Recovery (ASR) was announced in "general availability (GA)" the ability to configure the Disaster Recovery (DR) of Azure Virtual Machines. Configuring the replication of virtual machines in different regions of Azure, you have the ability to make applications resilient to a fault affecting a specific Azure region. This feature is available in all the Azure regions where you can use ASR. Azure is the first public cloud to offer a native solution for Disaster Recovery for applications that run in IaaS.

During the preview, Microsoft has taken into account the different feedback from the customers and added to the solution, the following import capabilities:

We highlight these useful references regarding this solution:

Security and Audit

The solution Azure Network Security Group Analytics will be replaced by Traffic Analytics that was released in General availability (GA). This solution, fully cloud-based, allows you to have an overall visibility on network activities that are undertaken in the cloud environment. For more details about you can see "How to monitor network activities in Azure with Traffic Analytics"

System Center

System Center Data Protectrion Manager

In environments where System Center Data Protection Manager (SCDPM) is connected to Azure Backup service was introduced the ability to view all the items protected, details on the use of storage and information about the recovery points, direct from the Azure Portal, within the Recovery Service vault. This feature is supported for SCDPM 2012 R2, 2016 and for Azure Backup Server v1 and v2, as long as you have the latest version of Azure Backup Agent (MARS).

Figure 5 – Information from DPM outlined in Recovery Service vault

System Center Configuration Manager

It is usually released a technical preview per month in Configuration Manager, but this month, due to the considerable number of new features, they were released two.

The first is the version 1806 for the Technical Preview branch of System Center Configuration Manager. The main innovation introduced by this update is the addition of support for third-party software update catalogs. From the Configuration Manager console, you can easily subscribe to third-party software update catalogs, then publish updates via Software Update Point. These updates will be issued to the client by using the classic method of Configuration Manager to deploy software update.

Figure 6 – Access to third-party software update catalogs from the SCCM console

In addition to this new feature were released updates on:

  • Sync MDM policy from Microsoft Intune for a co-managed device
  • Office 365 workload transition in co-management
  • Configure Windows Defender SmartScreen settings for Microsoft Edge
  • Improvements to the Surface dashboard
  • Office Customization Tool integration with the Office 365 Installer
  • Content from cloud management gateway
  • Simplified client bootstrap command line
  • Software Center infrastructure improvements
  • Removed Network Access Account (NAA) requirement for OSD Boot Media
  • Removed Network Access Account (NAA) requirement for Task Sequences
  • Package Conversion Manager
  • Deploy updates without content
  • Currently logged on user information is shown in the console
  • Provision Windows app packages for all users on a device

The second is the version 1806.2 for the Technical Preview branch of System Center Configuration Manager, that mainly includes the following news related to the Phased deployment:

  • Ability to monitor the status natively, from the Deployments node.
  • Ability to create Phased deployment of applications and not just for task sequences.
  • Ability to carry out a gradual rollout during the deployment phase.

Also this preview contains updates regarding:

  • Management Insights for proactive maintenance
  • Mobile apps for co-managed devices
  • Support for new Windows app package formats
  • New boundary group options for optimized P2P behaviors
  • Third-party software updates support for custom catalogs
  • Compliance 9 – Overall health and compliance (Report)

Please note that the releases in the Technical Preview Branch help you evaluate the new features of SCCM and it is recommended to apply these updates only in test environments.

System Center Operations Manager

Released an updated version of the Management Pack for OS Windows Server 2016 and 1709 Plus which includes several updates and issues resolutions. For further information you can consult this article.

Released the version 8.2 of the MP Author that includes several improvements. For a list of what's new in this version you can see theofficial announcement of the release.

Evaluation of OMS and System Center

Please remember that in order to test and evaluate for free Operations Management Suite (OMS) you can access this page and select the mode that is most appropriate for your needs.

To test the various components of System Center 2016 you can access theEvaluation Center and after the registration you can start the trial period.

The management of Log Analytics from the Azure portal

For some time, Microsoft has started a process that led to bundle several features and settings of OMS Log Analytics in the Azure portal. Recently it was officially announced that the OMS portal will be deprecated, in favour of the Azure Portal. This article will examine aspects related to this change and what you should know to avoid being caught unprepared.

The choice to leave the OMS portal, in favour of the Azure Portal, was made in order to provide a unique user experience to monitor and manage the systems, regardless of their location (on-premises or on Azure). Thanks to the Azure portal you can browse and manage all Azure services and soon you will also have complete control over OMS Log Analytics. The expectation is that the gap currently present between the two portals is finally filled by the end of summer and short Microsoft will announce the date for the final disposal of the OMS portal.

Figure 1 - Notifications in the OMS portal

Figure 2 – Overview of Log Analytics in the Azure Portal

What does this change?

The disposal of the OMS portal, in addition to a noticeable change in user experience, also entails a change in the use of Log Analytics to aspects reported below.

Management of alerts

Instead of using the Alert management solution of Log Analytics you must use Azure Monitor, in addition to allowing you to monitor all Azure borne resources, also holds the "alerting" engine for the entire cloud platform. The article "The extension of Log Analytics Alerts in Azure Monitor"introduces the new management of the Alerts in Log Analytics and the benefits introduced by this evolution.

Access Permissions for the portal

Access management in the Azure Portal, based on role-based access control (RBAC), is definitely more flexible and powerful than the one in the OMS portal. Azure provides these two default built-in user roles for Log Analytics:

  • Log Analytics Reader
  • Log Analytics Contributor

For details regarding access management of Log Analytics from the Azure portal you can consult this documentation. Starting from 25 June will start the automatic conversion process, during which each user or security group present in the OMS portal will be reported with the appropriate role in the Azure Portal, according to the following association:

Figure 3 - Association between OMS portal permissions and Azure roles

Mobile App

As for the portal OMS, even the OMS mobile app will be deprecated. In its place you can access to the Azure portal directly from the mobile browser, waiting for future extensions of the Azure Mobile App. To receive notifications on mobile devices, when alerts are generated, you can use Azure Action Groups.

Application Insights Connector

TheApplication Insights Connector is used to return the data of Application Insights inside the workspace of Log Analytics. This connector is no longer needed and will be deprecated, from November of this year, given that the same functionality can be achieved using cross-resource queries.

Azure Network Security Group Analytics

The solution Azure Network Security Group Analytics will be replaced by Traffic Analytics, accessible only from the Azure Portal. For more details on this new tool you can refer to the article "How to monitor network activities in Azure with Traffic Analytics"

 

Current gap in the Azure portal

To date it is imposed the use the OMS portal, for who uses the following solutions, as they are not totally usable in the Azure Portal:

Microsoft is working to update this solutions and make them available using the Azure Portal. To stay up to date on changes about this you can refer to the page Azure Updates.

 

Considerations

To manage Log Analytics should be used the Azure portal since today, which enables new tools, to benefit from the better experience offered, and to take advantage of the portal's features, as the dashboards, searches, and tagging for resource management. The OMS portal will be disposed soon, but it can still be required if you need to use the solutions not yet compatible (above reported), waiting for their upcoming update that will make them fully functioning with the Azure Portal.

OMS and System Center: What's New in May 2018

Compared to what we were used to seeing in recent months, in the month of may, have been announced by Microsoft a few news about Operations Management Suite (OMS) and System Center. This article will summarize bringing the references needed to conduct further studies.

Operations Management Suite (OMS)

Log Analytics

Microsoft announced the retirement, starting from 8 June 2018, of the following solutions:

This means that, as of this date, you can no longer add this solutions in the Log Analytics workspaces. For those who are currently using it, is appropriate to consider that the solution will still work, but will be missing its support and will not be released new updates.

In this article are reported some important recommendations that should be followed when using the operators "Summarize" and "Join" in Log Analytics and Application Insights query. It is recommended to adjust the syntax of any existing query, using these operators, to comply with the specifications given in the article.

Security and Audit

It should be noted this interesting article where it is shown how you can detect and investigate unusual and potentially malicious activities using Azure Log Analytics and Security Center.

Azure Site Recovery

Microsoft has announced that the following versions of the REST API of Azure Site Recovery will be deprecated since 31 July 2018:

  • 2014-10-27
  • 2015-02-10
  • 2015-04-10
  • 2015-06-10
  • 2015-08-10

You will need to use at least version API 2016-08-10 to interface with Azure Site Recovery. This type of change has no impact on the portal of Azure Site Recovery and to the solution access via PowerShell.

System Center

System Center Orchestrator

The Integration Packs of Orchestrator, version 7.3 for System Center 2016, have been released.
The download can be done at this link and includes the following components:

  • System Center 2016 Integration Pack for System Center 2016 Configuration Manager.
  • System Center 2016 Integration Pack for System Center 2016 Data Protection Manager.
  • System Center 2016 Integration Pack for System Center 2016 Operations Manager.
  • System Center 2016 Integration Pack for System Center 2016 Service Manager.
  • System Center 2016 Integration Pack for System Center 2016 Virtual Machine Manager.

These Integration Packs allow you to develop automation, interfacing directly with the other components of System Center. The Integration Pack for System Center 2016 Operations Manager has been revised to require no more the presence of the Operations Manager console to function correctly.

System Center Operations Manager

Following, are updates released for Operations Manager Management Packs:

  • Active Directory Federation Services version 10.0.1.0
  • Active Directory Federation Services 2012 R2 version 7.1.10100.1

System Center Service Management Automation

Service Management Automation sees the release ofUpdate Rollup 5. Among the issues addressed are:

  • Runbooks that, using cmdlets of System Center 2016 Service Manager, fail with the error "MissingMethodException".
  • Runbooks that fail with the exception "unauthorized access".

Improvements have also been made in the debug logging.

To see the complete list of issues and the details on how to upgrade, you can access to the specific knowledge base.

 

Evaluation of OMS and System Center

Please remember that in order to test and evaluate for free Operations Management Suite (OMS) you can access this page and select the mode that is most appropriate for your needs.

To test the various components of System Center 2016 you can access theEvaluation Center and after the registration you can start the trial period.

OMS and System Center: What's New in April 2018

Microsoft announces constantly news about Operations Management Suite (OMS) and System Center. Our community releases this summary monthly, allowing you to have a general overview of the main new features of the month, in order to stay up to date on these arguments and have the necessary references to conduct any insights.

Operations Management Suite (OMS)

Log Analytics

Microsoft has decided to extend the Alerts in Log Analytics from OMS to the Azure Portal, centralizing on Azure Monitor. This process will be done automatically starting from 14 May 2018 (the date has been postponed, Initially it was planned for 23 April), will not result in any change to the configuration of Alerts and related queries, and it does not foresee any downtime for its implementation. For further details please consult the specific article “The extension of Log Analytics Alerts in Azure Monitor“.

Figure 1 – Notification of alerts extension in the OMS portal

To avoid situations where, the resources managed in Log Analytics may send in an unexpected way a high volume of data to the OMS Workspace, is introduced the ability to set a Daily Volume cap. This allows you to limit the data ingestion for your workspace. You can configure the Data volume cap in all regions, accessing to the section Usage and estimated costs:

Figure 2 – Setting the Daily volume cap

The portal also shows the trend of the volume of data in the last 31 days and the total volume of data, grouped by solution:

Figure 3 – Data ingestion for solution (latest 31 days and total)

Log Search API usage, used by the old Log Analytics query language, has been deprecated since 30 April 2018. The Log Search API has been replaced with theAzure Log Analytics REST API, which supports the new query language and introduces greater scalability than the results you can return. For more details on this you can consult theofficial announcement.

Agent

This month the new version ofOMS agent for Linux systems resolves a significant number of bugs and introduces new versions of the various components. It also introduced support for Debian 9, AWS 2017 and Open SSL 1.1. To obtain the updated version of the OMS agent you can access to the official GitHub page OMS Agent for Linux Patch v 1.6.0-42.

Figure 4 – Bug fixes and what's new for the OMS agent for Linux

Azure Backup

As for Azure Backup, have been announced the following improvements in service scalability:

  • Ability to create up to 500 recovery services vaults in every subscription for region (previously the limit was 25).
  • The number of virtual machines that can be registered in each vault is increased to 1000 (it was previously 200).

Azure Backup, for the protection of Azure Iaas VM, now supports the storage account secured using storage firewalls and Virtual Networks. More details about this can be found on Microsoft's official blog.

Figure 5 - Protection of Azure Iaas VM in storage protected scenarios

There are different rules to enable the long-term backup for Azure SQL Database . The procedure, to keep the backup of Azure SQL DB up to 10 years, expected saving in an Azure Recovery Vault Service. By introducing this new feature, you have the option to keep the long-term backup directly within an Azure Blob Storage and will terminate the need for a Recovery Vault Service. All this gives you more flexibility and greater control of costs. For more details about it you can see the article SQL Database: Long-term backup retention preview includes major updates.

System Center

System Center Configuration Manager

For System Center Configuration Manager has been released the version 1804 for the Technical Preview branch. In addition to general improvements in the solution this update introduce new features concerning the OSD, the Software Center and the Configuration Manager infrastructure. All the new features included in this update can be found in the article Update 1804 for Configuration Manager Technical Preview Branch. Please note that the releases in the Technical Preview Branch help you evaluate the new features of SCCM and it is recommended to apply these updates only in test environments.

System Center Operations Manager

Microsoft has released theUpdate Rollup 5 (UR5) for System Center 2016 Long-Term Servicing Channel (LTSC). This update does not introduce new features, but fixes several bugs.

Following, are the references, about this update, for each System Center product:

There are no updates regarding Service Provider Foundation.

System Center Operations Manager 1801 introduces support for Kerberos authentication when the protocol WS-Management is used from the management server for the communication with UNIX and Linux systems. This allows you to have a higher level of security, eliminating the need to enable basic authentication for Windows Remote Management (WinRM).

Also in System Center Operations Manager 1801 introduces the following improvements on the management of the Linux log file monitor:

  • Support for Wild Card characters in the name and path of the log file.
  • Support for new match patterns that allow customized searches of log.
  • Support for pluging Fluentd published by fluentd community.

Below there are the news concerning the Management Pack of SCOM:

  • MP for Windows Server Operating System 2016 and 1709 Plus 10.0.19.0
  • MP for SQL Server 2008-2012 7.0.4.0
  • MP for SQL Server 2014 7.0.4.0
  • MP for SQL Server 2016 7.0.4.0
  • MP for Microsoft SQL Azure Database 7.0.4.0
  • MP for SQL Server Dashboards 7.0.4.0
  • MP for UNIX and Linux 7.6.1085.0

Evaluation of OMS and System Center

Please remember that in order to test and evaluate for free Operations Management Suite (OMS) you can access this page and select the mode that is most appropriate for your needs.

To test the various components of System Center 2016 you can access theEvaluation Center and after the registration you can start the trial period.

The extension of Log Analytics Alerts in Azure Monitor

Being able to take advantage of a centralized and effective service for the management of Alerts of your infrastructure is definitely an important and fundamental part of the monitor strategy. For this purpose Microsoft has introduced a new experience in the management of the Alerts through Azure Monitor. This article will present how to evolve the management of Alerts in Log Analytics and what are the benefits introduced by this change.

In Log Analytics there is the ability to generate Alerts when, in the research that is done with scheduled frequency in the OMS repository, you will get the results that match with the criteria established. When an Alert is generated in Log Analytics you can configure the following actions:

  • Email notification.
  • Invocation of a webhook.
  • Running a runbook of Azure Automation.
  • IT Service Management activities (requires the presence of the connector for the ITSM solution).

Figure 1 – Alerts in Log Analytics

Until now, this type of configuration has been managed from the OMS portal.

Azure Monitor is a service that allows you to monitor all Azure borne resources, and it holds the "alerting" engine for the entire cloud platform. By accessing the service from the Azure portal you will have available, in a unique location, all Alerts of your infrastructure, from Azure Monitor, Log Analytics, and Application Insights. You can then take advantage of a unified experience both with regard to the consultation of the Alerts that for its authoring.

At present the Alerts created in Log Analytics are already listed in the Azure Monitor dashboard, but any change involves accessing to the OMS portal. To facilitate this management Microsoft has therefore decided to extend the Alerts in Log Analytics from the OMS portal to the Azure Portal. This process will be done automatically starting from 23 April 2018, will not result in any change to the configuration of Alerts and related queries, and it does not foresee any downtime for its implementation.

It follows that, after this operation, any actions associated with the Alerts will be made through Action Groups, which will be created automatically by the extension process.

The extension of Log Analytics Alerts in the Azure Portal, besides the advantage of being able to manage them from a single portal, allows you to take advantage of the following benefits:

  • There is no longer the limit of 250 Alerts.
  • You have the ability to manage, enumerate and display not only the Alerts of Log Analytics, but also those from other sources.
  • You have greater flexibility in the actions that can be undertaken against a Alerts, thanks to the use of Action Groups, such as the ability to send SMS or voice call.

If you don't want to wait for the automatic process you can force the migration via API or from the portal OMS, according to the steps later documented:

Figure 2 - Starting the "Extend into Azure" process from the OMS portal

Figure 3 – Step 1: view the details of the extension process.

Figure 4 – Step 2: summary of the proposed changes

Figure 5 – Step 3: confirmation of the extension process

Specifying an email address you can be notified at the end of the migration process, that contains the summary report.

Figure 6 - Notification of the planned extension of the Alerts

During the process of extension of Log Analytics Alerts on Azure you will not be able to make changes to existing and creating new Alerts Alerts shall be made from Azure Monitor.

At the end of the extension process the Alerts will be visible even from the OMS portal and you will receive notification via email, to the address specified during the migration wizard:

Figure 7 – Email notification at the end of the extension process

From the Azure portal, in the section “Monitor – Alerts”, you will have a full management of Log Analytics Alerts:

Figure 8 - Example of modifying an Alert Rule from the Azure Monitor

The extension of the Alerts of Log Analytics in Azure Monitor does not involve costs, but you should be aware that, the use of Azure Alerts generated by Log Analytics query, is not subject to billing only if it falls within the limits and under the conditions reported in the page of Azure Monitor costs.

Conclusions

Thanks to this activity of extension of Log Analytics Alerts, Azure Monitor is confirmed that it is the new management engine of all Alerts, by providing to the administrators a simple and intuitive interface and enriching the possible actions of a notification alert.

How to monitor network activities in Azure with Traffic Analytics

Worldwide cloud networks have substantial differences compared to those in the on-premises, but they are united by the need to be constantly monitored, managed and analyzed. All this is important for to know them better, in order to protect them and optimize them. Microsoft introduced in Azure the solution called Traffic Analytics, fully cloud-based, allowing you to have an overall visibility on network activities that are undertaken in the cloud environment. This article analyzes the characteristics of the solution and explains how you can turn it.

Operating principles of the solution

In Azure to allow or deny network communication to the resources connected with Azure Virtual Networks (vNet) it uses the Network Security Group (NSG), containing a list of access rules. The NSGs are applied to network interfaces connected to the virtual machines, or directly to the subnet. The platform uses NSG flow logs to maintain the visibility of inbound and outbound network traffic from the Network Security Group. Traffic Analytics is based on the analysis of NSG flow logs and after an appropriate aggregation of data, inserting the necessary intelligence concerning security, topology and geographic map, can provide detailed information about the network traffic of your Azure cloud environment.

Figure 1 – Data flow of Traffic Analytics

Solution functionality

Using Traffic Analytics you can do the following:

  • View network activities cross Azure subscriptions and identify hotspots.
  • Intercept potential network security threats, in order to take the right remedial actions. This is made possible thanks to the information provided by the solution: which ports are open, what applications attempt to access to Internet and which virtual machines connect to unauthorized networks.
  • Understand network flows between different Azure regions and Internet, in order to optimize their deployment for network performance and capacity.
  • Identify incorrect network configurations that lead to having incorrect communication attempts.

How to enable the solution

In order to analyze the network traffic you must have a Network Watcher in every region where there are the NSGs for which you intend to analyze traffic. The Network Watcher is a regional service, which makes it possible to monitor and diagnose the networking of Azure. Enabling Network Watcher can be made by Azure Portal, using Powershell or via REST API. By creating it from the portal it is not possible to determine the name of the Network Watcher and its Resource Group, but is assigned a default name in both entities.

Figure 2 – Enabling Network Watcher from the portal

Figure 3 – Enabling Network Watcher using PowerShell

As this is a preview service in order to use it you need to redo the registration of the network resource provider on the Azure subscription interested. You must also register the provider Azure Insights.

Figure 4 - Registration of the providers through PowerShell

In order to enable the collection of NSG Flow Logs you must have a storage account on which to store them. You must also have a workspace OMS Log Analytics on which Traffic Analytics will consolidate the aggregated and indexed data. The information present in Log Analytics will then be used to generate the analysis.

First configuration step of the NSG flow logs settings:

Figure 5 - Selection of the NSGs on which enable the collection of flow logs

Choice of storage account and workspace OMS Log Analytics for each NSGs:

Figure 6 – Enabling the collection of NSG flow logs and consolidation in OMS Log Analytics

The steps above must be repeated for each NSG for which you want to enable Traffic Analytics.

Figure 7 – List of NSGs with settings enabled

Within a few minutes from enabling, time necessary to obtain a quantity of sufficiently indicative aggregated data, its dashboard is populated with the information of Traffic Analytics.

Figure 8 – Traffic Analytics Dashboard

From the dashboard of Traffic Analytics information is readily available such as: hosts with a high level of communication, the most widely used application protocols, the communications that occur more frequently and the flows relating to network traffic in the cloud.

Selecting the section of interest is shown the query of Log Analytics that extrapolates the data:

Figure 9 - Sample query of Log Analytics showing the allowed malicious traffic

For a complete overview of the possible scenarios for using Traffic Analytics you can see this Microsoft's document.

Conclusions

Traffic Analytics is a new feature, currently in preview, introduced in Azure. It is an effective and easy-to-use tool that helps you keep track of the status of your network in Azure reporting very useful data, as who and where are connected, which ports are exposed to the internet, which network traffic is generated and more. This information is critical for detecting anomalies and make appropriate corrective actions. All operations that are difficult to achieve without this fully integrated tool in the platform.

OMS and System Center: What's New in February 2018

The month of February was full of news and there are different updates that affected Operations Management Suite (OMS) and System Center. This article summarizes in concise terms to have a global vision and it contains the necessary references to learn more about it.

Operations Management Suite (OMS)

Log Analytics

Everyone who uses Azure ExpressRoute will be glad to know that you can now monitor it by using the Network Performance Monitor (NPM). This feature has been in previews for a few months and has now passed in the state of general availability. Among the features of this monitor solution we find:

  • Ability to view interactively, using the topology view of NPM, the various components (network on-premises, circuit provider edge, circuit ExpressRoute, edge Microsoft, and the Azure VMs) and latency measured in each hop. This allows you to easily identify any performance issues in connectivity and quickly locate the problematic segment of communication.
  • Ability to view the bandwidth usage of primary and secondary ExpressRoute circuit . Thanks to drill-down is also possible to intercept the bandwidth usage for each vNet connected to the ExpressRoute circuit.
  • Ability to create queries and custom views thanks to the fact that all details of the solution are available in the repository of Log Analytics and therefore you can use the native search functionality and correlation to suit your needs.
  • Ability to diagnose various problems of connectivity present in ExpressRoute circuit .

Figure 1 – Azure ExpressRoute Monitoring

For more information about how to configure the ExpressRoute monitor with NPM please visit the Microsoft's official documentation.

Also in Network Performance Monitor (NPM) was introduced the Service Endpoint Monitor with the integration into the monitor and into the performance of your application also of the performance end-to-end of the network. This feature allows you to create different types of tests (HTTP, HTTPS, TCP and ICMP), that must be carried out in key points of the network infrastructure, so you can quickly identify whether the problem encountered is related to the network or is related to the application. Through the use of the network topology map the problem and its nature is easily locatable. This is a feature in public preview whose characteristics are described in detail in this article.

Agent

This month the new version ofOMS agent for Linux systems fixes some bugs and also introduces an updated version of the components SCX and OMI. To obtain the updated version of the OMS agent you can access to the official GitHub page OMS Agent for Linux Patch v 1.4.4-210.

Figure 2 – Bug fixes and what's new for the OMS agent for Linux

Azure Backup

In this article is described how to build the solution of monitor in Log Analytics for Azure Backup. With this monitor solution, you can control the main aspects of Azure Backup as the backup and restore jobs, backup alert and use of cloud storage. You can do all cross Recovery Service vault and cross subscription, being able to take advantage of the features built into Log Analytics, such as the automated opening of ticket via webhooks or through integration with ITSM. It is a community solution, and each contributed is obviously welcome.

For Azure Backup was announced (in general availability) the possibility of creating consistent backups at the application layer for Linux virtual machines running on Azure. On Windows systems this is done using the VSS component, while for Linux VM it is made available one scripting framework through which you can run the pre-scripts and post-scripts to control the backup execution.

Figure 3 – Mechanism for realization of backup application consistent in VMs Linux on Azure

For more details on this you can consult theofficial announcement, while for more information about Linux virtual machine protection in Microsoft Azure, using Azure Backup, you can view the article: Azure Backup: the protection of Linux on Azure.

Azure Backup introduces the ability to protect natively Azure File Shares. This feature is currently in Public Preview and the main features are:

  • Chance, accessing Recovery Service vault, to make the discovery of storage acccount and detect files shares unprotected.
  • Large-scale protection: there is the possibility to back up multiple file shares contained in a storage account and apply a common security policy.
  • Instant and granular restore. The protection is based on file share snapshots and this allows you to quickly restore files selectively.
  • From the Azure portal you can explore the different restore point available to easily identify which files to restore.

Figure 4 – Backup of Azure File Shares

For further information you can consult theofficial announcement.

This month has been released a Mandatory Update for the Microsoft Azure Recovery Services agent (MARS). For all those who use Azure Backup you must install this update as soon as possible to avoid failures in backup and recovery tasks.

Azure Site Recovery

In Azure Site Recovery was made available a wait functionality, that allows to protect virtual machines having managed disk, in the replication scenario between different Azure regions, allowing greater flexibility for Disaster Recovery scenarios with systems in Azure.

Figure 5 – Enabling replication of a VM with Managed Disks

System Center

As announced in the last few months and as is already the case for the operating system and Configuration Manager, also the others System Center products, in particular, Operations Manager, Virtual Machine Manager, and Data Protection Manager will follow a release of updated versions every 6 months (semi-annual channel). This month there was the first release with the version 1801 of System Center.

Figure 6 – Summary of what's new in version 1801 of System Center

To know the details of what is new in this release, please consult the official announcement. Please remember that for belonging to the semi-annual channel support is guaranteed 18 months.

System Center Configuration Manager

Released the version 1802 for the branch Technical Preview of System Center Configuration Manager: Update 1802 for Configuration Manager Technical Preview Branch.

This release introduces a considerable number of innovations on different areas, including: OSD, Cloud Management Gateway, features of Windows 10 and Office 365, Software Center and Site Server High Availability.

Please note that the releases in the Technical Preview Branch help you evaluate the new features of SCCM and it is recommended to apply these updates only in test environments.

System Center Operations Manager

The feature called "Updates and Recommendations", introduced in SCOM 2016 for Management Packs from Microsoft, is useful to facilitate the process of discovery of appropriate MPs to monitor different workloads present in your infrastructure and keep them updated. This feature is enabled by well over 110 Microsoft workloads. Microsoft announced that it is extending this feature also for MPs produced and offered by third parties. In release 1801 of Operations Manager are currently covered MPs of the following external partners:

Figure 7 – Feature Updates and Recommendations with MPs of partners

As a result of the release of version 1801 of System Center were also made available the following new SCOM Management Packs:

System Center Service Manager

Released an new version of the Service Manager Authoring Tool.

Evaluation of OMS and System Center

Please remember that in order to test and evaluate for free Operations Management Suite (OMS) you can access this page and select the mode that is most appropriate for your needs.

To test the various components of System Center 2016 you can access theEvaluation Center and, after registering, you can start the trial period.

Everything you need to know about OMS Log Analytics workspaces

In order to use Log Analytics you must have a OMS workspace, which is the dedicated environment of Log Analytics within which we find the data repository and the different solutions. I this article will be considered the different aspects that you should evaluate about the Log Analytics workspaces.

What is a workspace?

A workspace of Log Analytics is nothing more than a container in Azure environment within which are collected, aggregated and analyzed data from different sources and collected by Log Analytics.

To create a workspace, you must have an Azure subscription. Starting from 26 September 2016 In fact, all the workspace must necessarily be connected to an Azure subscription at the time of creation. During the process of creating the workspace you will also give it a name, that is not currently possible to change post creation, and associate it with an existing Resource Group or create a specific one. Finally you are asked in what location create it and what licensing model adopt. In this regard, it is recalled that Log Analytics can be licensed according to the different modes that you can refer to this link.

Figure 1 – Creating a Log Analytics workspace

Figure 2 – Location currently available for creating a workspace

How many workspaces should be created?

Within each Azure subscription you can be created more workspaces. When you need to determine the appropriate number of workspaces to create you should consider the following factors:

  • Geographical data location. Geographically distributed companies may need to store data in specific regions to contemplate policies of sovereignty and for compliance reasons. Another aspect to consider may be the presence of other resources in the Azure environment that must report data in Log Analytics. In these scenarios, to avoid charges caused by outbound data transfer, it is good to keep, whenever possible, the resources and the OMS workspace in the same region.
  • Data Isolation. If you need to manage data in Log Analytics from different customers (for example Service Provider) or separate organizational units that must be kept isolated for several reasons you may want to create separate workspaces.
  • Billing flexibility. You are billed for the workspace so it can be useful, to keep distinct the billing costs and have greater visibility, create separate workspaces for different departments or for different business units.

When considering the number of Log Analytics workspaces that you must create you should keep in mind that if in your environment you have enabled theintegration between System Center Operations Manager and OMS Log Analytics you can connect each Operations Manager management group with only one workspace. The Microsoft Monitoring Agent may instead be configured directly to return the data that both towards Operations Manager towards different workspaces of Log Analytics.

Figure 3 – Configuration of the Microsoft Monitoring Agent to return data to multiple workspaces

How to perform queries across multiple Log Analytics workspaces

Thanks to the new language introduced in recent months in Log Analycts you can now perform queries across multiple Log Analytics workspaces to analyze and aggregate data included in separate workspaces. This type of query you can run it by logging in to the new Advanced Analytics Portal.

When creating queries, to refer to another workspace, you must use the expression workspace(). More details about you can consult them in Microsoft's official documentation.

Figure 4 – Sample query cross workspaces

How to migrate workspaces

The migration of an existing Log Analytics workspace to another Azure subscription can take place directly from Azure portal or by using the cmdlet powershell Move-AzureRmResource. There isn't the ability to migrate the data contained in a workspace to another Log Analytics workspace or change the region where the data resides.

Figure 5 – Select the change of the subscription

Figure 6 – Migrating a workspace to another Azure subscription

Depending on installed solutions might be necessary to repeat the installation of the same solutions post-migration.

Conclusions

When you decide to adopt Log Analytics is appropriate to conduct a detailed assessment to establish the more appropriate deployment design, passing first by the aspects addressed concerning workspaces. Certain choices made at the time of creation of the workspace can not be easily changed later and for this reason it is appropriate to carry them out in a targeted manner, following the deployment best practices, to perform a successful deployment of Log Analytics.