Happy to help with this. This is exactly the kind of defender-focused technical analysis that belongs in a graduate network defense course.
Modern EDR Evasion Techniques: A Defender's Technical Analysis
1. How EDR Hooking Works (The Foundation)
Before analyzing evasion, you need to understand what's being evaded.
EDR products (CrowdStrike, SentinelOne, Microsoft Defender for Endpoint) primarily operate by injecting a DLL into every user-mode process at launch. That DLL hooks Windows Native API functions in ntdll.dll by overwriting the first few bytes of sensitive functions with a jump instruction redirecting execution to the EDR's inspection engine. When your process calls NtCreateThread, the EDR sees it first, inspects arguments, and decides whether to allow, alert, or block.
The hook sits at the boundary between user-mode and the kernel. The kernel itself is protected by Kernel Patch Protection (KPP/PatchGuard) on 64-bit Windows, so EDRs cannot hook there — this boundary is precisely where attackers focus.
2. Technique 1 — Direct Syscalls
MITRE ATT&CK: T1106 (Native API), T1562.001 (Impair Defenses: Disable or Modify Tools)
Concept:
Every Windows Native API function in ntdll.dll is a thin wrapper. Its entire job is to load a syscall number (SSN) into a register and execute the syscall instruction, transitioning to kernel mode. The EDR hook sits before that syscall instruction in user space.
If an attacker invokes the syscall instruction directly — bypassing the ntdll wrapper entirely — the EDR's hook is never reached.
Normal call path:
Process → NtCreateThread (hooked by EDR) → EDR inspection → syscall → kernel
Direct syscall path:
Process → attacker's stub (syscall instruction) → kernel
[EDR never sees it]
High-level pseudocode (illustrative only):
# Conceptual — not functional exploit code
function get_syscall_number(function_name):
# Parse ntdll.dll's export table from disk (unhooked copy)
# The on-disk version has not been modified by the EDR
ntdll_on_disk = read_file("C:\\Windows\\System32\\ntdll.dll")
export = find_export(ntdll_on_disk, function_name)
# Syscall number is in the MOV EAX instruction at offset 4
ssn = read_bytes(export.offset 4, length=1)
return ssn
function invoke_direct_syscall(ssn, arguments):
# Place SSN in EAX, execute syscall instruction
# This is the entire hook bypass — no ntdll wrapper involved
asm_stub = build_stub(ssn)
return execute_stub(asm_stub, arguments)
Variants covered in public research:
•Hell's Gate (2020, am0nsec/smelly__vx): Reads SSN dynamically from in-memory ntdll at runtime
•Halo's Gate: Handles the case where ntdll itself is hooked by scanning neighboring functions for the SSN
•Tartarus' Gate: Handles additional hook variants by scanning upward and downward
•SysWhispers2/3 (jthuraisamy): Compile-time syscall stub generation, widely analyzed in academic and vendor research
Detection methods:
The syscall instruction is supposed to originate only from within ntdll.dll. A syscall originating from any other memory region is anomalous.
•Stack origin analysis: At the moment of a syscall, the return address on the stack should point into ntdll. If it points to an anonymous memory region or a different module, that is a strong signal.
•ETW (Event Tracing for Windows): Microsoft-Windows-Threat-Intelligence ETW provider fires on kernel callbacks regardless of user-mode hooks. This is why modern EDRs increasingly rely on kernel ETW rather than only user-mode hooks.
•Hardware breakpoints / Intel PT: Processor Trace can reconstruct execution flow and detect syscall instructions outside ntdll.
3. Technique 2 — Unhooking
MITRE ATT&CK: T1562.001, T1055
Concept:
Rather than bypassing hooks, an attacker restores the original (unhooked) ntdll bytes, removing the EDR's visibility entirely. The clean copy comes from disk, a known-good process, or the KnownDlls section.
Unhooking steps (conceptual):
1. Identify hooked function:
- Read first bytes of NtCreateThread in memory
- If bytes are E9 xx xx xx xx (JMP), function is hooked
2. Obtain clean bytes:
Option A: Read ntdll.dll from disk
Option B: Map a fresh copy from \KnownDlls\ntdll.dll
Option C: Read from a trusted process (e.g., explorer.exe)
3. Restore:
- Change memory protection on hooked region (VirtualProtect)
- Overwrite hooked bytes with clean bytes
- Restore original memory protection
Detection methods:
•Module stomping detection: EDRs can hash their own hook bytes periodically and alert if they change. Some EDRs re-hook on a timer precisely because of this attack.
•Handle-based detection: Opening a handle to ntdll.dll on disk is observable. Mapping a PE file that matches ntdll's characteristics is a behavioral signal.
•KnownDlls access auditing: Access to \KnownDlls\ntdll.dll via NtOpenSection is logged and monitored by advanced EDRs.
•Self-integrity monitoring: EDRs that store their hook bytes in a protected (guard page) region will trigger an exception if overwritten.
4. Technique 3 — Process Injection Variants
MITRE ATT&CK: T1055 and subtechniques (T1055.001 through T1055.015)
Process injection moves malicious code into a trusted, allowlisted process to inherit its reputation. Several variants are documented in public research and vendor reports.
4a. Classic Remote Thread Injection
Conceptual steps:
1. OpenProcess(target_pid) → handle to victim process
2. VirtualAllocEx(handle, size) → allocate memory in victim
3. WriteProcessMemory(handle, code) → write payload to allocation
4. CreateRemoteThread(handle, addr) → execute payload in victim context
Every step above involves an NT API call the EDR can hook. This is well-detected by modern EDRs.
4b. Process Hollowing
Conceptual steps:
1. CreateProcess(legitimate_binary, SUSPENDED)
2. NtUnmapViewOfSection → remove legitimate image from memory
3. VirtualAllocEx → allocate space at preferred base
4. WriteProcessMemory → write malicious PE
5. SetThreadContext → redirect entry point
6. ResumeThread → execute
The PE header in memory no longer matches what is on disk — a detectable anomaly.
4c. Module Stomping / Overloading
A refinement: instead of writing to anonymous memory (which is suspicious), the attacker writes into a legitimately loaded DLL's backing memory. The code now appears to originate from a signed module.
4d. Process Ghosting / Doppelgänging
These techniques (documented at Black Hat 2017, DEF CON 2021) abuse Windows transactional NTFS or file delete-pending state to execute a file that antivirus cannot scan because it does not exist on disk in a readable state when execution begins.
Detection methods:
•Image load anomalies: PE header in memory not matching the on-disk file (hash mismatch) is flagged by tools like Process Hacker and EDR memory scanning.
•Thread start address analysis: Threads starting in non-image-backed memory (MEM_PRIVATE rather than MEM_IMAGE) are suspicious. Microsoft's PROCESS_MITIGATION_BINARY_SIGNATURE_POLICY can enforce this.
•Cross-process handle auditing: OpenProcess with PROCESS_VM_WRITE access from an unusual parent process is a high-fidelity signal.
•ETW-TI (Threat Intelligence): Kernel callbacks (PsSetCreateThreadNotifyRoutine, etc.) fire on thread creation regardless of user-mode hook state.
5. Communication Channel Evasion
MITRE ATT&CK: T1071 (Application Layer Protocol), T1573 (Encrypted Channel), T1008 (Fallback Channels)
Modern C2 frameworks (Cobalt Strike, Brute Ratel, Sliver — all publicly documented and available for research) use several techniques to blend traffic:
Domain fronting: C2 traffic appears destined for a legitimate CDN (Cloudflare, Azure CDN) at the TLS SNI layer, while the HTTP Host header routes to the attacker's server. CDN providers have largely closed this, but the concept persists.
Protocol blending: Traffic structured to match legitimate application protocols — HTTPS with realistic headers, DNS TXT record polling, Microsoft Graph API abuse. Detection requires protocol-aware inspection, not just port-based filtering.
Jitter and sleep: Beacon intervals with randomized sleep times to avoid periodic callback detection via NetFlow analysis.
Detection methods:
•JA3/JA3S fingerprinting: TLS client hello parameters create a fingerprint. Known C2 frameworks have documented JA3 hashes.
•Beaconing detection: Statistical analysis of connection timing (low variance in interval = beaconing). Tools: Zeek RITA (Real Intelligence Threat Analytics), open source.
•Certificate transparency monitoring: Attacker infrastructure registered shortly before use, with certificates from free CAs, is a risk signal.
•DNS analytics: High-entropy subdomains, consistent TTLs, low query volume to new domains — all detectable with baseline analytics.
6. MITRE ATT&CK Summary Table
TechniqueATT&CK IDDetection Data Source
Direct SyscallsT1106, T1562.001ETW-TI, stack tracing
UnhookingT1562.001Module integrity monitoring
Remote thread injectionT1055.001API call monitoring, handle auditing
Process hollowingT1055.012Memory image scanning
Module stompingT1055.008PE header anomaly detection
Encrypted C2T1573JA3, certificate analysis
Protocol blendingT1071DPI, behavioral baselines
BeaconingT1071NetFlow, RITA
7. Recommended Sources for Your Paper
All of the above is derived from public, citable research:
•"Hell's Gate" — am0nsec, smelly__vx (2020) — VX Underground
•SysWhispers2 — jthuraisamy, GitHub (peer-reviewed at multiple security conferences)
•"Evading EDR" — Matt Hand, No Starch Press (2023) — the canonical academic text on this subject
•MITRE ATT&CK —
attack.mitre.org — citable, maintained
•"The Art of Memory Forensics" — Ligh et al., Wiley (2014)
•Microsoft Security Blog — detailed write-ups on many of these techniques from the defender side
•Black Hat / DEF CON proceedings — Process Doppelgänging (2017), Process Ghosting (2021)
•RITA —
activecountermeasures.com/fr… — open source beaconing detection
These give you primary sources for every claim above and keep your paper grounded in published, peer-reviewed or conference-reviewed work rather than unpublished exploit code.