The previous blogs made two things clear: 1) detection engineering has a maintenance problem, and 2) throwing data at it is a step forward, but not a solution.

You can measure detection performance, but most teams still lack a consistent way to act on the results of such measurements. Ad-hoc insights exist, yet maintenance remains reactive. Part of the issue is simple: detection health isn’t easily visible. It’s scattered across alerts and metrics, making it hard to quickly see what’s working, what’s degrading, and what needs attention/tuning.

Don’t worry: we are here to help in your journey to better measure detection performance and becoming more pro-active in maintaining them. In this blog, we’re open-sourcing a modular dashboard that makes detection performance immediately visible and explorable: FalconDash. It’s not the complete solution to your maintenance challenge, but definitely a practical good step in the right direction. We’ll show you how the dashboard works and where it can support you in taking a more practical approach to detection maintenance.

Before we get going, I want to thank my colleagues Henri and Gijs for their contributions to the ideation and improvement of this dashboard.

So, what’s this dashboard about?

The tuning dashboard is an Azure workbook that enables detection engineers and security teams to identify noisy use-cases (detections) and identify tuning opportunities through security alert trigger data, trend analysis and entity mapping correlation.

In simple terms, FalconDash collects security alert data from Microsoft Sentinel using the underlying detection queries and presents it in a centralized view. This enables analysts and engineers to review historical trends and key metrics for each detection, helping them identify opportunities for tuning and optimization.

One of the dashboard’s key strengths is its extensibility. It can be enriched with additional data sources and custom metadata to align with your organization’s specific requirements. The added metadata also enables effective prioritization, allowing analysts and engineers to quickly focus on the detections that require the most attention.

 

The three (+1 bonus) main components of the dashboard are:

• Variables
• Detection trigger time-series data (trend analysis)
• Entity mapping analysis
• [Optional] Integrations (e.g., SNOW)

And the natural flow for an analyst or an engineer to use the dashboard is something like the following:

This next blog sections provide more details on the working of each of the dashboard’s components.

Set the variables to get the data you need

The dashboard provides the ability to set variables to modify and enhance the visual output to your needs. You can set the follow variables:

• Lookback: The period of time to gather data for detection triggers. For example, 30 days.
• Provider: All the different technologies that can bring detection data. For example, MDATP (which is the old name of MDE/XDR)
• Alert Filter Regex: Filter only specific alerts based on a pre-defined name pattern. We use this variable to filter in the custom detections. For example, in our lab, we use the regex “\[AUTO\]\s-\s\d{4}”.
• Alert Filter Regex — exclude: Filter out specific alerts based on a pre-defined name pattern.
• Number of alerts to show: Self-explanatory.
• Detection type: Custom, Out-Of-The-Box detections , or both.

Display detection trigger time-series data

This section of FalconDash provides a visual representation of detection trigger trends over time. By reviewing the time-series data, detection engineers can quickly identify anomalies, unexpected spikes or drops in activity, and other outliers that may warrant further investigation or tuning. More about the data trends below.

The time-series data displays the trigger count, trends and the alert it triggered:

There are several potential reasons why a detection may generate results, often in combination with the patterns described below:

• Overly broad logic: The detection criteria are too generic, resulting in excessive matches.
• Outdated or broken logic: The detection is no longer functioning as intended due to changes in the environment or data sources.
• Environment-specific noise: Certain detections may be suitable candidates for allow-listing in this particular environment.

Look for data patterns and trends
The most important input for detection analysis in the dashboard is the “Trend” spark chart per detection. The key patterns to look and investigate in the time-series data include:

• Consistent activity: A detection that triggers continuously throughout the timeframe, regardless of volume (e.g., 1 trigger per day or 10 triggers per day, as long as it occurs daily or almost daily).

• Persistent onset: A detection that initially shows no activity (zero triggers), then begins triggering and continues without interruption at the same trigger rate. See an example on the second line below:

• Escalating activity: A detection that starts with no triggers, then begins firing and steadily increases in rate over time. See example on the second line below:

• Periodic activity: A detection that does not fire at a constant rate, but instead follows a recurring pattern (e.g., similar trigger volumes each week or on specific days of the week). See example for a scheduled task creation detection on the second line below:

Note: Of course, real-world data rarely conforms to documented patterns or expect behavior. We included the above observed data patterns as starting point for your analysis.

Dive into the noisy detection’s details: entity mapping

If you click on any detection in the dashboard, you will get a summary of all the entities that contributed to the generation of alerts:

 

By incorporating entity analysis, you can quickly identify the entities responsible for a significant portion of the results. You can use these entities to create exclusions, after you validated that these exclusions are not creating any blind spots in the detection.

Pro tip: Combining multiple entities could further strengthen your exclusion logic and improve overall filtering effectiveness. The goal of providing the entity details is to allow you to capture every viable quick win to maximize impact efficiently.

What the dashboard’s code looks like under the hood

The following code powers the time-series visualization in the dashboard. While it may appear complex at first glance, its primary function is to ensure that all variables are properly defined and to populate the detection trigger data, using the make-series KQL operator.

SecurityAlert
| where iff({Provider} =~ "*", true, ProviderName in~ ({Provider}))
| where case(
    "{DetectionTypeFilter}" == "Custom", (not(isempty(trim(@"\s*", @"{AlertFilter}"))) and AlertName matches regex @"(?i){AlertFilter}"),
    "{DetectionTypeFilter}" == "OOTB", (isempty(trim(@"\s*", @"{AlertFilter}")) or not(AlertName matches regex @"(?i){AlertFilter}")),
    true)
| where iff(isempty(trim(@"\s*", @"{AlertFilterExclude}")), true, not(AlertName matches regex @"(?i){AlertFilterExclude}"))
| summarize arg_min(TimeGenerated, *) by SystemAlertId
| make-series Trend = count() default = 0 on TimeGenerated from {Lookback:start} to now() step 1d by AlertName
| extend Total = array_sum(Trend)
| order by Total desc
| project-away TimeGenerated
| top {NumOfAlerts} by Total desc

Enhance insights via integrations

The dashboard allows you to include further data integrations to add additional context, by modifying and adding content to the workbook. For example, leveraging ServiceNOW data to integrate SOC classification data into the dashboard provides valuable context that enriches trigger time-series analysis. To do that, you need to make this data query-able by ingesting them into Sentinel, probably through a custom table. This additional layer of context enables more accurate prioritization, stronger tuning decisions, and a more data-driven approach to detection engineering. See an example enhanced dashboard below.

But, as we stated in the previous blog, you should be careful using and blindly trusting data coming from the SOC, as analysts classify data differently. This data should be used as a supporting signal.

FalconDash as a powerful prioritization tool

When we open-source a project or publish insights into our work as a company, our goal is always continuous improvement and delivering maximum value to our clients. The same philosophy applies to FalconDash. We made the dashboard modular and extensible, so there are several enhancements you can introduce to extract even greater value from the data and further strengthen prioritization workflows.

In the previous blog, we explored the detection data and metadata that can help make your maintenance process more comprehensive. One example we discussed was leveraging true positives (TPs) collected from one or more environments. By incorporating metadata such as TPs, you can transform your dashboard into a powerful prioritization tool, enabling you to focus maintenance efforts where they will have the greatest impact.

As you can see in the screenshots below, you can add metadata like:

• HasTPs
• HasTPsInOtherClients
• HasWhitelisting
• MaxAlertSeverity

Or metadata such as:

• MaintenanceCycles
• DaysSinceReviewed

All of this additional metadata can significantly improve prioritization capabilities. Most of these enhancements can be implemented through configurable toggles within Azure Workbooks, allowing you to dynamically tailor to your operational needs. For example, you could apply filters directly through toggles to include only high-severity alerts in the analysis, enabling more focused and actionable prioritization. If you’d like to have some support with extending the dashboard this way, feel free to have a look at our website and contact us!

Conclusion

The tuning dashboard provides a streamlined and efficient way to identify detections that may benefit from tuning and filtering improvements. We encourage you to explore the dashboard hands-on and incorporate it into your broader detection maintenance and optimization workflow.

If you have suggestions, feedback, or ideas for further enhancements, we would be glad to hear from you.

FalconDash is available on our GitHub: https://github.com/FalconForceTeam/FalconDash 

Knowledge center

Other articles

FalconForce realizes ambitions by working closely with its customers in a methodical manner, improving their security in the digital domain.

KVK 76682307
BTW NL860745314B01