Getting ALL the Microsoft Sentinel rule templates

Introduction

I recently had someone tell me that code I had written a while ago to allow the user to automatically update rules from rule templates didn’t get all the rule templates. After mumbling some comments under my breath that I should not mention here, I decided to take a look.

First of all, I promise that the code worked when it was written. “When it was written” is the key phrase here. As you know, Microsoft Sentinel is a SaaS product and, as such, it can change quite frequently. Well, it changed. In this case, it changed the way it gets the rule templates.

It used to be that a single call to the rule template REST API would give you all the rule templates. That is not the case anymore. Now there are three different calls that need to be made.

You need one call to get the “classic” rule templates. One to get the Anomalies, and one to get those rule templates created by a Content Hub solution.

If you are not already aware, a Content Hub solution will allow for the installation of data connectors, rule templates, workbook, and more. It is all packaged up into one solution, making it far easier to install. This will help prevent clutter around your data connectors and analytic rule templates as you install only what you need for your environment.

The original REST API

If you have read some of my old blog posts, this should be familiar to you. It is a simple REST API call and the only thing that has changed is the “api-version” as I like to use the latest and the greatest.

https://management.azure.com/subscriptions/<subscriptionID>/resourceGroups/<resourceGroupName>/providers/Microsoft.OperationalInsights/workspaces/<workspaceName>/providers/Microsoft.SecurityInsights/alertruletemplates?api-version=2022-11-01-preview

Again, this used to return ALL the rule templates, but it does not do that anymore. Will it be changed in the future to do so again? No idea. It would be nice.

Anomalies

Again, these used to be part of the REST API discussed above, but they have been moved into their own REST API call. It makes sense since they are not listed in the rule templates anymore, but rather in their own tab. I still include them here since you can create rules from them.

This is also just a simple REST API call

https://management.azure.com/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.OperationalInsights/workspaces/<workspaceName>/providers/Microsoft.SecurityInsights/securityMLAnalyticsSettings?api-version=2022-11-01-preview

The data is returned in almost the same format as the “classic” REST API, making it easy to traverse

Solutions

Now we come to getting the rule templates that solutions create. While it also uses a REST API call, or a PowerShell module (more on that later), it is a bit different. However, rather than using a Microsoft Sentinel REST API call, you use the Microsoft Resource Graph API call. There are a few differences between the two types of calls.

First of all, there is a body that needs to be sent so you need to use a POST rather than a GET when making the call. BTW, don’t try to use a PUT or you will get some weird errors. Not that I would ever do that and spend a large amount of time banging my head against the wall wondering why my code doesn’t work. 🙂

Second, the data being returned is in a different format. More on that later.

The body is comprised of two parts. The first is the “subscriptions” which is an array of those subscriptions you want to look at. Since you are logged into a single account, these subscriptions must be readable by that account you are currently logged into. I have not tried this with Lighthouse but my gut feeling is that is won’t work as this REST API is not really part of Microsoft Sentinel.

Second, you need to pass in a KQL query using “query”. Since the Resource Graph contains information on all the resources, without the query, you would get a lot of information that you don’t want. BTW, you need at least READ access to any resource for it to be returned in this REST API, so don’t worry about people seeing information they shouldn’t.

Here is some sample code to make the call. I’ll go through each line afterwards.

$context = Get-AzContext
$azureProfile = [Microsoft.Azure.Commands.Common.Authentication.Abstractions.AzureRmProfileProvider]::Instance.Profile
$profileClient = New-Object -TypeName Microsoft.Azure.Commands.ResourceManager.Common.RMProfileClient -ArgumentList ($azureProfile)
$token = $profileClient.AcquireAccessToken($context.Subscription.TenantId)
$authHeader = @{
    'Content-Type'  = 'application/json'
    'Authorization' = 'Bearer ' + $token.AccessToken
}
$SubscriptionId = $context.Subscription.Id

$query = "Resources 
| where type =~ 'Microsoft.Resources/templateSpecs/versions' 
| where tags['hidden-sentinelContentType'] =~ 'AnalyticsRule' 
and tags['hidden-sentinelWorkspaceId'] =~ '/subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.OperationalInsights/workspaces/<workspaceName>' 
| extend version = name 
| extend parsed_version = parse_version(version) 
| extend resources = parse_json(parse_json(parse_json(properties).template).resources) 
| extend metadata = parse_json(resources[array_length(resources)-1].properties)
| extend contentId=tostring(metadata.contentId) 
| summarize arg_max(parsed_version, version, properties) by contentId 
| project contentId, version, properties"

$body = @{
    "subscriptions" = @($SubscriptionId)
    "query"         = $query
}

$uri = "https://management.azure.com/providers/Microsoft.ResourceGraph/resources?api-version=2021-03-01"

$verdict = Invoke-RestMethod -Uri $uri -Method POST -Headers $authHeader -Body ($body | ConvertTo-Json -EnumsAsStrings -Depth 5)

The first 9 lines should be familiar to anyone who has worked with REST API called before. The query starts on line 11.

The main thing here are the first filters. The filter on line 12 will make sure we are looking for only those templates that are versions. I am not an expert on how the Resource Graph works, so I am not sure exactly why this filter is being used, but it is the code I got directly from the website.

The filter on lines 13 and 14 make sure that we are only looking at Analytic rules and they are those rules that are for the Microsoft Sentinel instance in question. Make sure to replace the needed values for subscription ID, resource group name, and workspace name. NOTE: I broke the lines up to make them readable. You will want to make sure it is a single line of text for it to work correctly.

The rest of the query creates some columns that are needed in the “summarize” command. The “summarize” command will make sure that only the latest and greatest version of the rule template is being returned.

The data being returned is a bit different than what are used to. Here is what my “$verdict” looks like:

Even then, it is still not what we want, as shown below. Although it is nice to be able to get to the version number easily.

OK, well, let’s look at the resources then. Success! It actually is returning what we expect to see. Although I have no idea why there are two different property fields. Very odd.

In any case, going into the properties field gives use the data that matches the properties field from the “classic” REST API call. We just needed to go down a few levels to get there.

Will the real rule template please step forward?

You will notice that there is a quite a bit of overlap between what is returned from the “classic” REST API and the one we just looked at. From what I can tell, the Resource Graph will win over the “classic” REST API which makes sense since the solutions are the new way of creating these rule templates

Summary

Working with a SaaS program is both a blessing and a curse. It is a blessing in that the code can be updated quickly and behind the scenes. It is a curse for that same reason! In this case, Microsoft has changed how the rule templates are stored, leading to some old code not working correctly. Hopefully the code listed here will help to fix that!