Reducing Binary Entropy

Introduction

Entry is the amount of randomness within a data set. As randomness increases, so does entropy. There are many type of entropy. In cybersecurity, we're focused on Shannon's entropy.

The image shows how to utilize entropy while threat hunting.

Threat Hunting with File EntropyPractical Security Analytics LLC

Measuring Entropy

Several tools can determine the entropy of a given file such as pestudio and Sigcheck.

We can use SciPy's scipy.stats.entropy combined with matplotlib to create a visual of entropy.

def calc_entropy(buffer):
    if isinstance(buffer, str):
        buffer = buffer.encode()
    entropy = 0
    for x in range(256):
        p = (float(buffer.count(bytes([x])))) / len(buffer)
        if p > 0:
            entropy += - p * math.log(p, 2)
    return entropy

Choosing an Encryption Algorithm

When picking an encryption, encoding, or any other obfuscation algorithm. It's important to keep entropy in mind, that includes choosing the corrent algorithm but one that isn't too weak.

Another effective method to keeping entropy low is using the obfuscation algorithms.

IPv4fuscation, IPv6fuscation, Macfuscation, and UUIDfuscation** instead of using encryption algorithms**

Inserting English Strings

Another method we can use to reduce entropy is inserting english strings into the final implementation. It's recommended to use either all lower case or all upper case strings** to reduce the number of possibilities for every byte.**

Padding by Bytes

An easy way to reduce entropy is to pad the payloads ciphertext with the same byte repeatedly. Here is a MsfVenom payload before and after appending it with 285 bytes of 0xEA.

drastically dropping entropy from 5.88325 to 3.77597.

CRT Library Independant

The CRT, or C Runtime Library, is a standard interface for the C programming language. The CRT contains a collection of functions & macros. These functions are meant for managing memory, opening and manipulating files, etc. (memcpy, fopen, strcpy).

Removing CRT can drastically reduce the entropy of the final implementation.

Tool - EntropyReducer

It's also possible to reduce a payload's entropy using EntropyReducer, a tool developed by the MalDev Academy team.

EntropyReducer uses a custom algorithm that utilizes linked lists to insert null bytes between each BUFF_SIZE byte chunk of the payload.