Anti Reversing Techniques

Seth HobsonCommunityCreated by Dávid BalatoniDistributed by Seth Hobson

Skill
Tool integration
Security
MIT
No issues found

From the author

Understand anti-reversing, obfuscation, and protection techniques encountered during software analysis. Use this skill when analyzing malware evasion techniques, when implementing anti-debugging protections for CTF challenges, when reverse engineering packed binaries, or when building security research tools that need to detect virtualized environments.

— the description by Dávid Balatoni

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides a structured approach to identifying and neutralizing anti-reversing protections in binaries. It covers common evasion patterns including API-based anti-debugging, PEB-based checks, timing-based detection, and code virtualization. The skill is designed for security researchers, malware analysts, and CTF participants to facilitate legitimate analysis by providing identification methods, bypass strategies, and code artifacts for neutralizing protection layers.

What it does

Identifies anti-debugging and anti-reversing techniques within binary files.
Provides specific bypass strategies for common protection mechanisms.
Offers guidance on manual unpacking and OEP identification.
Includes scripts for scanning timing-based detection patterns.
Explains how to handle virtualization-based code protections.
Details platform-specific differences for Windows, Linux, and ARM environments.

When to use it

When analyzing malware that employs evasion or anti-analysis techniques.
During authorized penetration testing or security research engagements.
When reverse engineering packed or protected binaries.
When developing security research tools that need to detect or bypass virtualized environments.
When implementing or testing anti-debugging protections for CTF challenges.

Advantages

Centralizes knowledge on diverse protection mechanisms and their respective bypasses.
Provides actionable scripts and command sequences for immediate application.
Addresses both static identification and dynamic analysis workflows.
Offers clear guidance on handling platform-specific nuances like ARM vs x86.
Includes safety and legal warnings for responsible security research.

Usage examples

“Identify potential anti-debugging checks in this Windows binary.”: The agent scans the binary for common API calls like IsDebuggerPresent and PEB-based flags, providing a list of offsets to investigate in a disassembler.
“How do I bypass a timing-based RDTSC check in my target binary?”: The agent explains how to use hardware breakpoints to avoid INT3 overhead or suggests patching the conditional jump following the timing comparison.
“Help me unpack this binary protected with a custom packer.”: The agent guides the user through the ESP trick in x64dbg to find the Original Entry Point (OEP) and suggests using Scylla to fix the import table.

Install

Download the skill as a ZIP:
https://autogineer.com/agent-skills/wshobson/anti-reversing-techniques/download

Unzip it into the cross-client skills folder of your project:

mkdir -p .agents/skills && unzip anti-reversing-techniques.zip -d .agents/skills

Documentation by Dávid Balatoni

AUTHORIZED USE ONLY: This skill contains dual-use security techniques. Before proceeding with any bypass or analysis:
Verify authorization: Confirm you have explicit written permission from the software owner, or are operating within a legitimate security context (CTF, authorized pentest, malware analysis, security research)
Document scope: Ensure your activities fall within the defined scope of your authorization
Legal compliance: Understand that unauthorized bypassing of software protection may violate laws (CFAA, DMCA anti-circumvention, etc.)
Legitimate use cases: Malware analysis, authorized penetration testing, CTF competitions, academic security research, analyzing software you own/have rights to

Anti-Reversing Techniques

Understanding protection mechanisms encountered during authorized software analysis, security research, and malware analysis. This knowledge helps analysts bypass protections to complete legitimate analysis tasks.

For advanced techniques, see references/advanced-techniques.md

Input / Output

What you provide:

Binary path or sample: the executable, DLL, or firmware image under analysis
Platform: Windows x86/x64, Linux, macOS, ARM — affects which checks apply
Goal: bypass for dynamic analysis, identify protection type, build detection code, implement for CTF

What this skill produces:

Protection identification: named technique (e.g., RDTSC timing check, PEB BeingDebugged) with location in binary
Bypass strategy: specific patch addresses, hook points, or tool commands to neutralize each check
Analysis report: structured findings listing each protection layer, severity, and recommended bypass
Code artifacts: Python/IDAPython scripts, GDB command sequences, or C stubs for bypassing or implementing checks

Detailed patterns and worked examples

Detailed pattern documentation lives in references/details.md. Read that file when the navigation tier above is insufficient.

Troubleshooting

Detection technique works on x86 but not ARM

RDTSC and CPUID are x86-only. On ARM, use MRS x0, PMCCNTR_EL0 (requires kernel PMU access) or clock_gettime(CLOCK_MONOTONIC). PEB/TEB do not exist on ARM — replace with /proc/self/status (Linux) or task_info (macOS). Rebuild detection logic with platform-specific APIs.

False positive on legitimate debugger or analysis tool

Timing checks fire when Process Monitor or AV hooks inflate syscall latency. Calibrate the threshold at startup: measure the guarded path 3 times and use mean + 3*stddev. For ptrace checks, verify the TracerPid comm name via /proc/<pid>/comm before exiting — it may be an unrelated monitoring tool, not a debugger.

Bypass patch causes crash instead of continuing execution

Before NOPing a conditional jump, trace the "detected" branch fully. If it initializes or frees heap state needed later, patching the jump skips that setup and corrupts state. Instead, patch the comparison operand to the expected "clean" value, or use x64dbg's "Set condition to always false" on the breakpoint rather than modifying bytes.

binary-analysis-patterns — static and dynamic analysis workflows for ELF/PE/Mach-O
memory-forensics — process memory acquisition, artifact extraction, and live analysis
protocol-reverse-engineering — decoding custom binary protocols and encrypted network traffic

Files

ATTRIBUTION.mdAdded by AutoGineer903 B
LICENSELicence file added by AutoGineer1.0 KiB
SKILL.md3.8 KiB
references/advanced-techniques.md10.4 KiB
references/details.md11.1 KiB

references/details.md

anti-reversing-techniques — detailed patterns and worked examples

Anti-Debugging Techniques

Windows Anti-Debugging

API-Based Detection

// IsDebuggerPresent
if (IsDebuggerPresent()) {
    exit(1);
}

// CheckRemoteDebuggerPresent
BOOL debugged = FALSE;
CheckRemoteDebuggerPresent(GetCurrentProcess(), &debugged);
if (debugged) exit(1);

// NtQueryInformationProcess
typedef NTSTATUS (NTAPI *pNtQueryInformationProcess)(
    HANDLE, PROCESSINFOCLASS, PVOID, ULONG, PULONG);

DWORD debugPort = 0;
NtQueryInformationProcess(
    GetCurrentProcess(),
    ProcessDebugPort,        // 7
    &debugPort,
    sizeof(debugPort),
    NULL
);
if (debugPort != 0) exit(1);

// Debug flags
DWORD debugFlags = 0;
NtQueryInformationProcess(
    GetCurrentProcess(),
    ProcessDebugFlags,       // 0x1F
    &debugFlags,
    sizeof(debugFlags),
    NULL
);
if (debugFlags == 0) exit(1);  // 0 means being debugged

Bypass: Use ScyllaHide plugin in x64dbg (patches all common checks automatically). Manually: force IsDebuggerPresent return to 0, patch PEB.BeingDebugged to 0, hook NtQueryInformationProcess. In IDA: ida_bytes.patch_byte(check_addr, 0x90).

PEB-Based Detection

// Direct PEB access
#ifdef _WIN64
    PPEB peb = (PPEB)__readgsqword(0x60);
#else
    PPEB peb = (PPEB)__readfsdword(0x30);
#endif

// BeingDebugged flag
if (peb->BeingDebugged) exit(1);

// NtGlobalFlag
// Debugged: 0x70 (FLG_HEAP_ENABLE_TAIL_CHECK |
//                 FLG_HEAP_ENABLE_FREE_CHECK |
//                 FLG_HEAP_VALIDATE_PARAMETERS)
if (peb->NtGlobalFlag & 0x70) exit(1);

// Heap flags
PDWORD heapFlags = (PDWORD)((PBYTE)peb->ProcessHeap + 0x70);
if (*heapFlags & 0x50000062) exit(1);

Bypass: In x64dbg, follow gs:[60] (x64) or fs:[30] (x86) in dump. Set BeingDebugged (offset +2) to 0; clear NtGlobalFlag (offset +0xBC on x64).

Timing-Based Detection

// RDTSC timing
uint64_t start = __rdtsc();
// ... some code ...
uint64_t end = __rdtsc();
if ((end - start) > THRESHOLD) exit(1);

// QueryPerformanceCounter
LARGE_INTEGER start, end, freq;
QueryPerformanceFrequency(&freq);
QueryPerformanceCounter(&start);
// ... code ...
QueryPerformanceCounter(&end);
double elapsed = (double)(end.QuadPart - start.QuadPart) / freq.QuadPart;
if (elapsed > 0.1) exit(1);  // Too slow = debugger

// GetTickCount
DWORD start = GetTickCount();
// ... code ...
if (GetTickCount() - start > 1000) exit(1);

Python script — timing-based anti-debug detection scanner:

#!/usr/bin/env python3
"""Scan a binary for common timing-based anti-debug patterns."""
import re
import sys

PATTERNS = {
    "RDTSC":              rb"\x0f\x31",                    # RDTSC opcode
    "RDTSCP":             rb"\x0f\x01\xf9",                # RDTSCP opcode
    "GetTickCount":       rb"GetTickCount\x00",
    "QueryPerfCounter":   rb"QueryPerformanceCounter\x00",
    "NtQuerySysInfo":     rb"NtQuerySystemInformation\x00",
}

def scan(path: str) -> None:
    data = open(path, "rb").read()
    print(f"Scanning: {path} ({len(data)} bytes)\n")
    for name, pattern in PATTERNS.items():
        hits = [m.start() for m in re.finditer(re.escape(pattern), data)]
        if hits:
            offsets = ", ".join(hex(h) for h in hits[:5])
            print(f"  [{name}] found at: {offsets}")
    print("\nDone. Cross-reference offsets in IDA/Ghidra to find check logic.")

if __name__ == "__main__":
    scan(sys.argv[1])

Bypass: Use hardware breakpoints (no INT3 overhead), NOP the comparison + conditional jump, freeze RDTSC via hypervisor, or hook timing APIs to return consistent values.

Exception-Based Detection

// SEH: if debugger is attached it consumes the INT3 exception
// and execution falls through to exit(1) instead of the __except handler
__try { __asm { int 3 } }
__except(EXCEPTION_EXECUTE_HANDLER) { return; }  // Clean: exception handled here
exit(1);  // Dirty: debugger swallowed the exception

// VEH: register handler that self-handles INT3 (increments RIP past INT3)
// Debugger intercepts first, handler never runs → detected
LONG CALLBACK VectoredHandler(PEXCEPTION_POINTERS ep) {
    if (ep->ExceptionRecord->ExceptionCode == EXCEPTION_BREAKPOINT) {
        ep->ContextRecord->Rip++;
        return EXCEPTION_CONTINUE_EXECUTION;
    }
    return EXCEPTION_CONTINUE_SEARCH;
}

Bypass: In x64dbg, set "Pass exception to program" for EXCEPTION_BREAKPOINT (Options → Exceptions → add 0x80000003).

Linux Anti-Debugging

// ptrace self-trace
if (ptrace(PTRACE_TRACEME, 0, NULL, NULL) == -1) {
    // Already being traced
    exit(1);
}

// /proc/self/status
FILE *f = fopen("/proc/self/status", "r");
char line[256];
while (fgets(line, sizeof(line), f)) {
    if (strncmp(line, "TracerPid:", 10) == 0) {
        int tracer_pid = atoi(line + 10);
        if (tracer_pid != 0) exit(1);
    }
}

// Parent process check
if (getppid() != 1 && strcmp(get_process_name(getppid()), "bash") != 0) {
    // Unusual parent (might be debugger)
}

Bypass (LD_PRELOAD hook):

# hook.c: long ptrace(int request, ...) { return 0; }
# gcc -shared -fPIC -o hook.so hook.c
LD_PRELOAD=./hook.so ./target

GDB bypass command sequence:

# 1. Make ptrace(PTRACE_TRACEME) always return 0 (success)
catch syscall ptrace
commands
  silent
  set $rax = 0
  continue
end

# 2. Bypass check after ptrace call: find "cmp rax, 0xffffffff; je <exit>"
#    Clear ZF so the conditional jump is not taken:
#    set $eflags = $eflags & ~0x40

# 3. Bypass /proc/self/status TracerPid check at the open() level
catch syscall openat
commands
  silent
  # If arg contains "status", patch the fd result to /dev/null equivalent
  continue
end

# 4. Bypass parent process name check
set follow-fork-mode child
set detach-on-fork off

Anti-VM Detection

Hardware Fingerprinting

// CPUID-based detection
int cpuid_info[4];
__cpuid(cpuid_info, 1);
// Check hypervisor bit (bit 31 of ECX)
if (cpuid_info[2] & (1 << 31)) {
    // Running in hypervisor
}

// CPUID brand string
__cpuid(cpuid_info, 0x40000000);
char vendor[13] = {0};
memcpy(vendor, &cpuid_info[1], 12);
// "VMwareVMware", "Microsoft Hv", "KVMKVMKVM", "VBoxVBoxVBox"

// MAC address prefix
// VMware: 00:0C:29, 00:50:56
// VirtualBox: 08:00:27
// Hyper-V: 00:15:5D

Registry/File Detection

// Windows registry keys
// HKLM\SOFTWARE\VMware, Inc.\VMware Tools
// HKLM\SOFTWARE\Oracle\VirtualBox Guest Additions
// HKLM\HARDWARE\ACPI\DSDT\VBOX__

// Files
// C:\Windows\System32\drivers\vmmouse.sys
// C:\Windows\System32\drivers\vmhgfs.sys
// C:\Windows\System32\drivers\VBoxMouse.sys

// Processes
// vmtoolsd.exe, vmwaretray.exe
// VBoxService.exe, VBoxTray.exe

Timing-Based VM Detection

// VM exits cause timing anomalies
uint64_t start = __rdtsc();
__cpuid(cpuid_info, 0);  // Causes VM exit
uint64_t end = __rdtsc();
if ((end - start) > 500) {
    // Likely in VM (CPUID takes longer)
}

Bypass: Use bare-metal environment, harden VM (remove guest tools, randomize MAC, delete artifact files), patch detection branches in the binary, or use FLARE-VM/REMnux with hardened settings.

For advanced VM detection (RDTSC delta calibration, VMware backdoor port, hypervisor leaf enumeration, guest driver artifact checks), see references/advanced-techniques.md.

Code Obfuscation

Control Flow Obfuscation

Control Flow Flattening

// Original
if (cond) {
    func_a();
} else {
    func_b();
}
func_c();

// Flattened
int state = 0;
while (1) {
    switch (state) {
        case 0:
            state = cond ? 1 : 2;
            break;
        case 1:
            func_a();
            state = 3;
            break;
        case 2:
            func_b();
            state = 3;
            break;
        case 3:
            func_c();
            return;
    }
}

Analysis Approach:

Identify state variable
Map state transitions
Reconstruct original flow
Tools: D-810 (IDA), SATURN

Opaque Predicates

int x = rand();
if ((x * x) >= 0) { real_code(); }   // Always true  → junk_code() is dead
if ((x*(x+1)) % 2 == 1) { junk(); }  // Always false → consecutive product is even

Analysis Approach: Identify invariant expressions via symbolic execution (angr, Triton), or pattern-match known opaque forms and prune them.

Data Obfuscation

String Encryption

// XOR encryption
char decrypt_string(char *enc, int len, char key) {
    char *dec = malloc(len + 1);
    for (int i = 0; i < len; i++) {
        dec[i] = enc[i] ^ key;
    }
    dec[len] = 0;
    return dec;
}

// Stack strings
char url[20];
url[0] = 'h'; url[1] = 't'; url[2] = 't'; url[3] = 'p';
url[4] = ':'; url[5] = '/'; url[6] = '/';
// ...

Analysis Approach:

# FLOSS for automatic string deobfuscation
floss malware.exe

# IDAPython string decryption
def decrypt_xor(ea, length, key):
    result = ""
    for i in range(length):
        byte = ida_bytes.get_byte(ea + i)
        result += chr(byte ^ key)
    return result

API Obfuscation

// Dynamic API resolution
typedef HANDLE (WINAPI *pCreateFileW)(LPCWSTR, DWORD, DWORD,
    LPSECURITY_ATTRIBUTES, DWORD, DWORD, HANDLE);

HMODULE kernel32 = LoadLibraryA("kernel32.dll");
pCreateFileW myCreateFile = (pCreateFileW)GetProcAddress(
    kernel32, "CreateFileW");

// API hashing
DWORD hash_api(char *name) {
    DWORD hash = 0;
    while (*name) {
        hash = ((hash >> 13) | (hash << 19)) + *name++;
    }
    return hash;
}
// Resolve by hash comparison instead of string

Analysis Approach: Identify the hash algorithm, build a database of known API name hashes, use HashDB plugin for IDA, or run under a debugger to let the binary resolve calls at runtime.

Instruction-Level Obfuscation

; Dead code insertion — semantically inert but pollutes disassembly
push ebx / mov eax, 1 / pop ebx / xor ecx, ecx / add ecx, ecx

; Instruction substitution — same semantics, different encoding
xor eax, eax  →  sub eax, eax  |  mov eax, 0  |  and eax, 0
mov eax, 1    →  xor eax, eax; inc eax  |  push 1; pop eax

For advanced anti-disassembly tricks (overlapping instructions, junk byte insertion, self-modifying code, ROP as obfuscation), see references/advanced-techniques.md.

Bypass Strategies Summary

General Principles

Understand the protection: Identify what technique is used
Find the check: Locate protection code in binary
Patch or hook: Modify check to always pass
Use appropriate tools: ScyllaHide, x64dbg plugins
Document findings: Keep notes on bypassed protections

Tool Recommendations

Anti-debug bypass:    ScyllaHide, TitanHide
Unpacking:           x64dbg + Scylla, OllyDumpEx
Deobfuscation:       D-810, SATURN, miasm
VM analysis:         VMAttack, NoVmp, manual tracing
String decryption:   FLOSS, custom scripts
Symbolic execution:  angr, Triton

Ethical Considerations

This knowledge should only be used for:

Authorized security research
Malware analysis (defensive)
CTF competitions
Understanding protections for legitimate purposes
Educational purposes

Never use to bypass protections for: software piracy, unauthorized access, or malicious purposes.

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Security

No issues found by our static scan· No issues found

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by

Dávid Balatoni

Publisher

Seth Hobson (Community)

Distributed by

Seth Hobson

Source

wshobson/agents · plugins/reverse-engineering/skills/anti-reversing-techniques · commit 4bde316

Licence

MIT License (redistribution allowed)

LICENSE (added by AutoGineer from LICENSE)
ATTRIBUTION.md (written by AutoGineer)

Synced

2026-11-09

AutoGineer is not affiliated with or endorsed by Dávid Balatoni or Seth Hobson.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download anti-reversing-techniques.zip