Log-Analytics

The data available through the Office 365 Security & Compliance Center contains data for all entities stored in that particular Office 365 / Azure Active Directory tenant.

A while ago I wrote an article on how to estimate data consumption for Log Analytics. Since then there were changes to the way that Log Analytics tracks the volume and the cost associated with data flowing into the workspace.

A request that I get many times is to provide guidance and best-practices on how to implement monitoring and operations management with and/or for Azure. Most organizations have subtle differences which makes it hard to put out a generic concept of how to enable this. There are high-level ideas but in my discussions with customers and partners I’m hearing that this is not detailed enough.

This is why my job is so much fun. I have been working with the CISO of a large retailer that operates multiple independent web portals (some with shopping functions enabled).

[UPDATE 02/12/2019] There has been changes to the way Log Analytics tracks usage data. Check out the post on the updated mechanisms.

This problem is probably as old as there are monitoring tools on the market. “How can I exclude legitimately deallocated VMs from alerting” is a question that I’ve heard many times.

This problem is probably as old as there are monitoring tools on the market. “How can I exclude legitimately deallocated VMs from alerting” is a question that I’ve heard many times.

This problem is probably as old as there are monitoring tools on the market. “How can I exclude legitimately deallocated VMs from alerting” is a question that I’ve heard many times.

I’ve had a question today coming from a colleague that has no prior experience in System Center Operations Manager (SCOM). He wanted to know how data flows from connected agents (regardless of Windows or Linux) to Log Analytics and subsequently to Azure Security Center.

Every once in a while you might need to create an alert which runs a Log Analytics or Application Insights query. When designing the alert you need to define some attributes: the query, the time period, the frequency and the threshold.

The Microsoft Monitoring Agent is able to send data to more than one workspace at the same time. Unfortunately only a single workspace at a time can be configured through the Azure Portal.

In a time before cross-resource queries were possible the Application Insights Connector would copy data from Application Insights to a Log Analytics workspace. With the emergence of cross-resource queries the duplication of data is not required anymore as queries can be sent to both (or even more) entities at the same time in real time.

Photo by Ilya Pavlov on Unsplash Change Tracking is a versatile feature that allows to monitor changes on a system (both Windows and Linux). Change tracking covers software installation, changes to services, daemons, Registry and the file system. This is available for both cloud based (Azure, AWS, GCP), on-premises and service provider hosted systems (given network connectivity to Azure).

Photo by Matt Artz on Unsplash In many scenarios there is the requirement to enrich or lookup data with meta information from the infrastructure. In this scenario a file with machine, location and other meta information was placed during deployment on the VM for both Azure and AWS.

Photo by Samuel Zeller on Unsplash Since Friday May 25th 2018 the General Data Protection Regulation (GDPR) took effect across Europe. It governs how data should be processed and provides extensive rights to the person which data is used.

Photo by frank mckenna on Unsplash Monitoring the container infrastructure which is running your applications is important. With the emergence of managed Kubernetes such as Aure Container Service (AKS) this becomes more tricky as part of the infrastructure is managed by somebody else.

Almost all Azure management services run in/for any cloud. Among them is Update Management which automates OS patching for both Linux and Windows machines whether they are running on-premises, in Azure or in other clouds.

Azure Policy is a great tool to define governance controls in Azure. With addition of the compliance pieces this feature which was part of Azure for quite some time finally had it’s appearance on main stage (deep dive on implementing governance at scale in this video from Ignite 2017 by Joseph Chan and Liz Kim)

Nearly every customer I talk to about Azure management asks me this: “How can I do process monitoring?”. As there is currently no way to directly instrument either the Windows or the Linux agent to do explicit process monitoring another way need to be found.

Around Ignite 2017 Azure Security Center was migrated to use Log Analytics as its foundation both for collecting data through the same agent and storing most of its data.