Precision vs Recall: The Detection Engineer's Trade-off

Two rules can both "detect" an attack and be worlds apart in quality. The difference is captured by precision and recall.

The definitions

Precision - of everything the rule fires on, how much is truly malicious? Low precision = alert fatigue.
Recall - of all the malicious activity, how much does the rule catch? Low recall = blind spots.
F1 score - the harmonic mean of the two: a single number that punishes being lopsided.

Fire on every PowerShell launch -> great recall, terrible precision (the SOC drowns and tunes you out).
Match one exact string -> great precision, poor recall (a one-character change evades you).

The craft is landing in the middle: catch the real thing, with few enough false positives that analysts trust the alert.

Anchor on robust attributes attackers can't trivially change (behavior + context), not one brittle string.
Enumerate variants (e.g. -enc, -e, -EncodedCommand).
Add filters for known-good actors instead of disabling the whole rule.
Measure against a labeled corpus so you know your precision/recall - don't guess.

A detection nobody trusts is worse than no detection, because it hides the real one.

Two rules can both "detect" an attack and be worlds apart in quality. The difference is captured by precision and recall.

Precision - of everything the rule fires on, how much is truly malicious? Low precision = alert fatigue.
Recall - of all the malicious activity, how much does the rule catch? Low recall = blind spots.
F1 score - the harmonic mean of the two: a single number that punishes being lopsided.

Fire on every PowerShell launch -> great recall, terrible precision (the SOC drowns and tunes you out).
Match one exact string -> great precision, poor recall (a one-character change evades you).

The craft is landing in the middle: catch the real thing, with few enough false positives that analysts trust the alert.

Anchor on robust attributes attackers can't trivially change (behavior + context), not one brittle string.
Enumerate variants (e.g. -enc, -e, -EncodedCommand).
Add filters for known-good actors instead of disabling the whole rule.
Measure against a labeled corpus so you know your precision/recall - don't guess.

A detection nobody trusts is worse than no detection, because it hides the real one.