Deploying Detections at Scale — Part 0x01 use-case format and automated validation
Deploying Detections at Scale — Part 0x01 use-case format and automated validation
March 13, 2023

Gijs Hollestelle

Building a repository of use-cases

  • It is tailored to Sentinel and does not cover Microsoft 365 Defender.
  • Use-case IDs are random GUIDs.
  • It does not allow for full customization and dynamic allow-listing. Note this will be the topic of a future blog post.
  • There is no link between ATT&CK techniques and tactics.
  • It is missing a change-log that records when the use-case was modified and why.
  • Limited meta-data is available, for example reference URLs, expected false positive rate, tags and response actions are not part of the template.
Automated validation of schema and auto completion based on json-schema in VS Code.

Implementing automated validity testing

  1. Validate that the usecase.yml file is correct according to the json-schema specification. This schema describes the formats of the various elements in the YAML file, as well as their type and whether they are mandatory or optional.
  2. Perform additional validations in Python that are hard to express in json-schema. An example of this is that the change log entries are required to be sorted with the newest one on top.
  3. Perform optional validations against query best-practices. These validations are optional in the sense that they will generate a warning rather than an error. Also, these validations can be silenced. An example of such a validation is that the FileProfile function in Microsoft 365 Defender is always used with a second argument specifying the maximum number of hashes.
  4. Validation of the actual query for syntax errors and validation against the Sentinel and Microsoft 365 Defender schemas. We have developed a custom KQL query analyzer REST server can analyze the query and provide meta-data, such as which tables are referenced and whether there are syntax errors present.
  5. Validation of entity mapping and custom details against the query output. This validation uses the same KQL query analyzer REST server mentioned above to verify that the entity mappings and custom details that are specified are actually present in the output of the query.

Example usage

  • We entered an invalid value for the severity: L.
  • We referenced ATT&CK technique T1210, which is a valid technique, but cannot be combined with tactic TA0009.
  • The change log is not in the correct order with the latest change at the top.
  • In the entity_mapping section there is a a reference to userAccount column (with lowercase u) which is not correct, since it should be UserAccount.

Automatically running validations using a CI Pipeline

Example output from a CI pipeline that verifies the syntax when a pull request is created.

Query syntax validation

Example usage of the KQL analyzer REST service

“We hope that our tools are useful in helping organizations to better manage and deploy detection use-cases at scale!”

Knowledge center

Other articles

Together. Secure. Today.

Stay in the loop and sign up to our newsletter

FalconForce realizes ambitions by working closely with its customers in a methodical manner, improving their security in the digital domain.

Energieweg 3
3542 DZ Utrecht
The Netherlands

FalconForce B.V.
[email protected]
(+31) 85 044 93 34

KVK 76682307
BTW NL860745314B01