Lets Make Malware – Bypassing Behavioral Detections (System Calls)
Introduction
Last time we covered using trampolines and unhooking to bypass behavioral detections through hooked API functions. This time we will discuss another method, adding another technique to our arsenal. We will be using system calls (syscalls).
syscall itself is a literal assembly instruction telling our CPU to enter kernel-mode to perform some high-privileged tasks such as opening a file on-disk.
However, in this context, syscall refers to many of the low-level, kernel-mode NT API functions like NtOpenProcess (what OpenProcess ends up calling) or NtProtectVirtualMemory (what VirtualProtect ends up calling).
What’s interesting here is that many of these functions have user-mode equivalents in NTDLL.dll named the same.
Syscalls
Essentially, OpenProcess calls NtOpenProcess in NTDLL.dll which would end up calling NtOpenProcess in the kernel (ntoskrnl.exe). It is that final NtOpenProcess performing all the actual work.
EDRs can hook the NtOpenProcess function that exists in NTDLL.dll, but not the one that exists in ntoskrnl.exe. Because everything that happens in the kernel can crash the entire system (Blue Screen of Death), Microsoft introduced Kernel Patch Protection (KPP or PatchGuard) in Windows 2003.
For our purposes, it means EDRs cannot hook functions in kernel-mode. They may only do so in user-mode, where we also have control.
Furthermore, syscalls are special. The main difference between them is each has a unique number associated with them called a System Service Number (SSN) or syscall ID. They are identical (in x64 assembly) otherwise. Here is an example. Note that this format is called a syscall stub.
mov r10, rcx
mov eax, <SSN>
syscall
ret
So, if we know the SSN to the function we want to use, we can implement it in Assembly, call it directly, and bypass the EDR’s hooks that way.
We can use inline assembly with MinGW or Visual Studio (x86 only) or we can put our stub (including the correct SSN for the syscall we want to make) in a separate assembly file, assemble it into an object file and link it to our C codebase during compilation to accomplish this.
But, of course, things aren’t so simple. SSNs can change depending on architecture (x64, x86) as well as Windows versions (even Service Packs/Updates can change them).
We have a few options here.
We can include separate implementations of every system call we want to use for several (or all) Windows versions (ex: one implementation of NtOpenProcess for Windows 11, one for Windows 10 21H2, another for Windows 10 20H1 and so on)
We can target one or two specific versions of Windows if we know the specifics of our target system.
We can dynamically retrieve the SSNs from NTDLL.dll once our malware runs.
Note: there are several methods of dynamically retrieving SSNs. Some of these methods work on a hooked NTDLL.dll and some require an unhooked version. Some notable examples are Hell’s Gate, Halo’s Gate, Tartarus Gate, RecycledGate, the FreshyCalls/SysWhispers2 method, and by using the Exception Directory
Mark of the Syscall
Any of these options can work, however each presents the same detection opportunity. We call this detection “Mark of the Syscall”.
This is one issue with this technique, the Direct Syscall. We have included, necessarily, the “syscall” instruction. The bytes 0f 05 which correspond to the syscall instruction now show up in our binary.
This is not normal. Normal user-mode applications do not make system calls directly. They go through NTDLL.dll, directly in some cases or more commonly, indirectly, through kernel32.dll.
We could use int 2Eh instead of our syscall instruction which will do the same thing and is a bit stealthier, however, we have the same problem. Something that should not be present in our binary is present.
We have another issue as well. Because we are making a system call directly in our binary, after this function completes, it returns to our binary. This is another anomaly.
Normally, a system call should return to NTDLL.dll (more specifically, within the DLL’s memory address space). From there, it would return either to our binary (in some cases) or again, more commonly, to kernel32.dll and then to us.
This is our call stack (also referred to as the function call stack), which an EDR has some visibility into, and it does not look normal.
Indirect Syscalls
Luckily, again, we have an alternative: the indirect syscall. The indirect syscall works much in the same way the direct syscall does. The difference being with the indirect syscall technique we will find and save an address where a syscall instruction occurs in NTDLL.dll.
Finding the address of a syscall instruction could be done while dynamically retrieving the SSN for example.
Also, note that the address of the syscall instruction does not necessarily need to correspond to the correct function. This means we could use the address of the syscall instruction that is within NtOpenProcess, for example, even if the system call we are making is NtCreateProcessEx.
It is probably better if we match them, however, we do not have to.
So, instead of including the syscall instruction directly, we can use the jmp instruction along with the address we found. This fixes both issues direct syscalls have.
Whether or not we implement direct or indirect syscalls within our malware, one issue remains. Our own codebase may be the only thing utilizing them. If we include shellcode, or we are packing a PE file, depending on how it was created, it may try to use the Win32 API functions which are likely hooked. This, of course, could end up burning us.
So, if we want to include third-party code, we may still want to unhook NTDLL.dll (at a minimum. We may also want to unhook other DLLs such as kernel32.dll and kernelbase.dll). This time, we can use syscalls to unhook them without having to rely on hooked functions or trampolines.
Conclusion
And with that, our discussion on syscalls is over. In the future, we may dive deeper into how different methods of retrieving SSNs work and how to implement their logic but including them here would have made this post incredibly long, so we will leave things here.
Resources
Undocumented NTInternals - Undocumented NTAPI Functions
ired.team - Calling Syscalls Directly from Visual Studio to Bypass AVs/EDRs
Klez Virus - Syswhispers is Dead, Long Live Syswhispers
GitHub - Windows Syscall Stubs
MDSec - Resolving System Service Numbers Using the Exception Directory