How defenders detect malware

Why layers

Attackers change their malware all the time. One method alone will miss things. Defence in depth means using several detection methods, so something that slips past one is caught by another.

1. Signatures and hashes

Antivirus started with signatures: a known pattern of bytes or a known file hash. If a file's SHA-256 matches a hash from a threat report, it is the same known-bad file. This is fast and accurate for known malware, but a tiny change to the file gives a new hash.

PS> Get-FileHash C:\Users\ravi\Downloads\invoice_update.exe -Algorithm SHA256 | Select-Object Hash

Hash
----
<64 hex characters>

2. YARA rules

YARA is a free tool that matches files against rules made of text or byte patterns and conditions. Analysts and threat researchers share YARA rules to find families of malware, not just one exact file.

rule Suspicious_Invoice_Dropper
{
    meta:
        description = "Example: flags files containing both strings"
    strings:
        $a = "powershell -enc" nocase
        $b = "invoice_update.exe"
    condition:
        $a and $b
}

3. Behaviour (heuristics and EDR)

Endpoint detection and response (EDR) tools watch what programs do: which process started which, what files and registry keys changed, what network connections were made. Some patterns are rarely innocent:

• A Word or Excel process (winword.exe, excel.exe) starting PowerShell or cmd.exe.
• PowerShell with a long encoded command (-enc or -EncodedCommand).
• A process deleting shadow copies (Windows backups), which ransomware does before encrypting.
• Hundreds of files renamed in a minute.
• A program in a temp or downloads folder connecting to the internet every 60 seconds (beaconing).

4. Persistence: how malware survives a reboot

Malware wants to start again after a restart. Places to check:

PS> Get-ScheduledTask | Where-Object { $_.TaskPath -eq "\" } | Select-Object TaskName, State

TaskName              State
--------              -----
OneDrive Reporting    Ready
WinUpdateHelper       Ready

5. Sandboxing and reputation

A sandbox is an isolated virtual machine where an unknown file is run and watched. If it tries to encrypt files or call a known-bad server, it is malware. Reputation services check how common and how old a file or domain is.

Common mistakes

• Trusting "no antivirus alert" as proof of clean. Check behaviour and logs too.
• Opening a suspicious file on your own work laptop to see what it does.
• Removing the malware file but not the scheduled task that will download it again.
• Hunting only on Windows when the attacker is on a Linux server or in the cloud.