Differences between normal and direct system calls

A way to distinguish between a normal system call via the regular Windows APIs and a direct system call is to look at the call stack being generated. While our EDR of choice (Microsoft Defender for Endpoint) does not provide call stack traces, the excellent and free Sysmon tool does provide these for Event ID 10 (Process Access).

To test if these call stack traces can be used to distinguish between regular system calls and direct system calls we developed a very small BOF file that only opens a process handle to another process, sleeps for a while and then exits. Opening a process handle to another process is an essential part in many offensive techniques such as credential dumping and process injection.

The first version of the BOF uses the regular method of opening a process using the regular OpenProcess call.

#include <windows.h>
#include <stdio.h>
#include "beacon.h"

WINBASEAPI HANDLE kernel32$OpenProcess(
  DWORD dwDesiredAccess,
  BOOL  bInheritHandle,
  DWORD dwProcessId
);

WINBASEAPI void kernel32$Sleep(
  DWORD dwMilliseconds
);

void go(char* args, int length) {
    int target_pid = 6244; // hardcoded pid of an explorer.exe running under same user account for ease of development
    HANDLE h;
    BeaconPrintf(CALLBACK_OUTPUT, "Opening process: %d\n", target_pid);
    h = kernel32$OpenProcess(MAXIMUM_ALLOWED, FALSE, target_pid);
    BeaconPrintf(CALLBACK_OUTPUT, "Handle: %x\n", h);
    BeaconPrintf(CALLBACK_OUTPUT, "Sleeping\n", h);
    kernel32$Sleep(5000);
    BeaconPrintf(CALLBACK_OUTPUT, "Done\n", h);
    return;
}

The second version of the BOF uses the direct system call method by directly calling NtOpenProcess via the ‘syscall’ instruction.

#include <windows.h>
#include <stdio.h>
#include <winternl.h>
#include "beacon.h"
#include "syscalls.h"

WINBASEAPI void kernel32$Sleep(
  DWORD dwMilliseconds
);

void go(char* args, int length) {
    int target_pid = 6244; // hardcoded pid of an explorer.exe running under same user account for ease of development
    HANDLE h;

NTSTATUS status;
    OBJECT_ATTRIBUTES ObjectAttributes;

InitializeObjectAttributes(&ObjectAttributes, NULL, 0, NULL, NULL);
    CLIENT_ID uPid = { 0 };

uPid.UniqueProcess = (HANDLE)(DWORD_PTR)target_pid;
    uPid.UniqueThread = (HANDLE)0;

BeaconPrintf(CALLBACK_OUTPUT, "Opening process: %d\n", target_pid);
    status = NtOpenProcess(&h, MAXIMUM_ALLOWED, &ObjectAttributes, &uPid);
    BeaconPrintf(CALLBACK_OUTPUT, "Handle: %x\n", h);
    BeaconPrintf(CALLBACK_OUTPUT, "Sleeping\n", h);
    kernel32$Sleep(5000);
    BeaconPrintf(CALLBACK_OUTPUT, "Done\n", h);
    return;
}

Using Sysmon we captured the following traces:

Trace 1 — BOF using Regular OpenProcess API:

TargetProcessId: 6244
TargetImage: C:\Windows\Explorer.EXE
GrantedAccess: 0x1FFFFF
CallTrace: 
 C:\Windows\SYSTEM32\ntdll.dll+9d2e4
|C:\Windows\System32\KERNELBASE.dll+2c03e
|UNKNOWN(0000022ECF78004A)

Trace 2 — BOF using Direct NtOpenProcess System Call:

TargetProcessId: 6244
TargetImage: C:\Windows\Explorer.EXE
GrantedAccess: 0x1FFFFF
CallTrace: 
 UNKNOWN(0000022ECF78048F)

For reference we also included a call trace for a regular windows executable that calls Openprocess.

Trace 3 — Regular Windows executable calling OpenProcess API:

TargetProcessId: 6244
TargetImage: C:\Windows\Explorer.EXE
GrantedAccess: 0x1FFFFF
CallTrace: 
 C:\Windows\SYSTEM32\ntdll.dll+9d2e4
|C:\Windows\System32\KERNELBASE.dll+2c03e
|c:\temp\Test.exe+1e0d
|c:\temp\Test.exe+2480
|C:\Windows\System32\KERNEL32.DLL+17034
|C:\Windows\SYSTEM32\ntdll.dll+52651

In these call traces the ordering is top-down based on the last function called, meaning the most recently called function, which actually invoked the NtOpenProcess system call is at the top.

Detecting direct system calls

These three traces provide some insight into how direct system call behaviour by BOFs can be detected:

• Cobalt Strike BOFs run from a memory page which is not mapped to an executable or DLL, showing as UNKNOWN in the Sysmon call traces above. This is even true in case the Cobalt Strike option module_x64 / module_x86 is used which makes the Beacon payload itself appear to originate from a memory mapped DLL.
• Using the regular OpenProcess API shows a call stack that goes from executable to KERNELBASE to NTDLL. While using direct system calls these intermediate layers are bypassed and the function is called directly.

Some larger scale investigation showed that regular calling of OpenProcess typically follows one of the following patterns:

• Executable -> KERNELBASE/KERNEL32 -> NTDLL/WIN32U
• Executable -> KERNELBASE/KERNEL32-> NTDLL/WIN32U/WOW64WIN -> WOW64 -> NTDLL/WIN32U/WOW64WIN (32-bit executables running on 64-bit Windows)

Based on this behaviour we can build two detection rules. These rules assume Sysmon data being ingested into Azure Sentinel.

Rule 1 — Identify direct system calls by looking if the first entry in the call stack is not NTDLL, WIN32U or WOW64WIN

let ValidDlls=dynamic([
  "ntdll.dll",
  "win32u.dll",
  "wow64win.dll"
]);
Sysmon
| where EventID == 10
| extend Callers=split(CallTrace, "|")
| extend FirstCaller=tostring(split(Callers[0], "+")[0])
| where not(FirstCaller has_any (ValidDlls))

A simple Sysmon configuration for Process Access can be used to capture the relevant events:

<ProcessAccess onmatch="include">
  <Rule groupRelation="and">
    <CallTrace condition="not begin with">C:\Windows\SYSTEM32\ntdll.dll</CallTrace>  
    <CallTrace condition="not begin with">C:\Windows\SYSTEM32\win32u.dll</CallTrace>
    <CallTrace condition="not begin with">C:\Windows\SYSTEM32\wow64win.dll</CallTrace>
  </Rule>
</ProcessAccess>

Rule 2 — Identify usage of NtOpenProcess from NTDLL directly, this is similar to a direct system call but slightly different with the program bypassing KERNEL32 and calling directly into NTDLL

This can yield some false positives as there are some native Windows DLLs that call directly into NTDLL so these need to be whitelisted such as lsasrv.dll.

let ValidCallers=dynamic([
  "kernelbase.dll",
  "wow64.dll",
  "kernel32.dll",
  "lsasrv.dll",
  "themeservice.dll",
  "wow64win.dll"
]);
Sysmon
| where EventID == 10
| extend Callers=split(CallTrace, "|")
| extend FirstCaller=tostring(split(Callers[0], "+")[0])
| extend SecondCaller=tostring(split(Callers[1], "+")[0])
| extend ThirdCaller=tostring(split(Callers[2], "+")[0])
| where not(SecondCaller has_any (ValidCallers))
| where not(ThirdCaller has_any (ValidCallers))

Unfortunately, writing a Sysmon filter to capture all relevant events for this second detection rule is challenging since Sysmon only allows for simple string based filtering. This makes it hard to write a Sysmon filter that only captures the Process Access events related to directly calling the undocumented APIs in NTDLL. The rule can still be useful if you already have an existing Sysmon config that logs suspicious Process Access calls based on other heuristics, in which case the fact that an undocumented API call was used could be an additional indicator to identify malicious events.

Hopefully we have shown that direct system calls can be detected, especially when calling the NtOpenProcess system call which directly generates a Sysmon event which includes a call trace.