Back
Interactive Explainer

Advanced Threat Detection

How to detect attacks that have bypassed preventive controls — covering SIEM architecture, UEBA, anomaly detection, threat hunting, and detection-as-code with Sigma rules.

🎯Key Takeaways
MTTD is the most important security metric — every day of undetected attacker dwell time means more damage.
SIEM correlates events; EDR detects endpoint compromise; GuardDuty catches IAM and API abuse; UEBA detects behavioural anomalies.
Enable AWS GuardDuty (or equivalent) in every account and every region — ~$3/month, catches real attacks automatically.
UEBA detects slow-and-low attacks that threshold rules miss — insider threats, compromised accounts using normal credentials.
Threat hunting is proactive: form a hypothesis, query data, document findings, convert to automated rules.
Detection-as-code (Sigma) version-controls detection rules, enables peer review, and allows vendor-neutral rule sharing.
Every successful threat hunt produces a new automated detection rule — hunting feeds detection engineering.
The SolarWinds 9-month dwell time is the cost of poor detection coverage.

Advanced Threat Detection

How to detect attacks that have bypassed preventive controls — covering SIEM architecture, UEBA, anomaly detection, threat hunting, and detection-as-code with Sigma rules.

~8 min read
Be the first to complete!
Why this matters

Preventive controls (firewalls, WAFs, MFA) are bypassed every day. The average attacker dwell time before detection is 207 days (IBM 2023). Without active detection, attackers operate undetected inside your environment for months.

Without this knowledge

Attackers move laterally, exfiltrate data, establish persistence, and prepare ransomware over weeks or months. By the time damage is visible, the attacker has long-term access.

With this knowledge

Anomalous behaviour is detected within hours or days. Threat hunters proactively search for indicators of compromise. Detection rules are codified as tests. MTTD shrinks from months to hours.

What you'll learn
  • MTTD is the most important security metric — every day of undetected attacker dwell time means more damage.
  • SIEM correlates events; EDR detects endpoint compromise; GuardDuty catches IAM and API abuse; UEBA detects behavioural anomalies.
  • Enable AWS GuardDuty (or equivalent) in every account and every region — ~$3/month, catches real attacks automatically.
  • UEBA detects slow-and-low attacks that threshold rules miss — insider threats, compromised accounts using normal credentials.
  • Threat hunting is proactive: form a hypothesis, query data, document findings, convert to automated rules.
  • Detection-as-code (Sigma) version-controls detection rules, enables peer review, and allows vendor-neutral rule sharing.
  • Every successful threat hunt produces a new automated detection rule — hunting feeds detection engineering.
  • The SolarWinds 9-month dwell time is the cost of poor detection coverage.

Lesson outline

The Detection Stack: SIEM, EDR, Cloud-Native, and UEBA

A complete threat detection stack combines several layers, each catching different classes of threat.

LayerWhat it detectsExamples
SIEMCross-source correlation: brute force then successful login from new geo, lateral movement patternsSplunk, Microsoft Sentinel, Elastic SIEM
EDRProcess injection, credential dumping, malware execution on hostsCrowdStrike Falcon, SentinelOne, Microsoft Defender
Cloud-NativeAPI abuse, IAM anomalies, resource misconfigurationsAWS GuardDuty, GCP SCC, Azure Defender
UEBAUnusual user behaviour: access at odd hours, large data downloads, new system accessSplunk UBA, Sentinel UEBA, Securonix
NDRLateral movement, C2 beaconing, data exfiltration over networkDarktrace, ExtraHop, Vectra AI
SOARAutomates response playbooks: isolate host, block IP, revoke credentialsPalo Alto XSOAR, Splunk SOAR

SIEM is only as good as the logs fed into it

Before investing in SIEM tuning, ensure you have: authentication logs (Okta, AD, CloudTrail), network flow logs, application logs (structured JSON with user IDs), endpoint telemetry (EDR), and cloud activity logs (CloudTrail, GCP Audit Logs).

High-Priority Detection Use Cases

Effective detection starts with a prioritised set of use cases based on your most likely threat scenarios.

Detection rules every organisation should have

  • Credential stuffing / brute force — >10 failed logins from the same IP in 5 minutes, or >3 failed logins to the same account from different IPs in 1 minute.
  • Impossible travel — Successful login from location A, then location B within a time window too short to physically travel between them.
  • Privilege escalation — IAM policy attaching AdministratorAccess, addition to admin group, new IAM user with console access.
  • Large data exfiltration — S3 GetObject >10,000 objects in 1 hour from a single identity, or unusual outbound data volume (>1GB) from a service.
  • Lateral movement — Service account authenticating to systems it has never accessed before, or unexpected service-to-service network connections.
  • C2 beaconing — Regular outbound connections to the same external IP at fixed intervals from backend services that should not initiate external connections.

AWS GuardDuty: Cloud-Native Threat Detection

AWS GuardDuty is a managed threat detection service that continuously monitors CloudTrail, VPC Flow Logs, and DNS logs — with no agents required.

Key GuardDuty finding types

  • Recon:IAMUser/UserPermissions — IAM user is making API calls to enumerate permissions — attacker reconnoitering blast radius
  • Backdoor:EC2/C&CActivity.B — EC2 instance communicating with a known command-and-control server
  • CryptoCurrency:EC2/BitcoinTool.B — EC2 instance connecting to cryptocurrency mining pools
  • Exfiltration:S3/ObjectRead.Unusual — IAM identity reading S3 objects at an unusual rate based on baseline behaviour
  • PrivilegeEscalation:IAMUser/AdministrativePermissions — IAM user creating policies to grant themselves admin access

Enable GuardDuty in every AWS account and every region

GuardDuty costs ~$3/month for most accounts and catches real attacks automatically. Enable it organisation-wide via AWS Organizations. Enable in all regions — attackers sometimes use inactive regions to avoid detection.

guardduty-response.tf
1# GuardDuty + EventBridge automated response — quarantine compromised EC2
2resource "aws_cloudwatch_event_rule" "guardduty_high" {
3  name        = "guardduty-high-severity"
4  event_pattern = jsonencode({
5    source      = ["aws.guardduty"]
6    detail-type = ["GuardDuty Finding"]
7    detail = {
8      severity = [{ numeric = [">=", 7] }]  # 7.0+ = HIGH severity
severity >= 7.0 = HIGH; 9.0+ = CRITICAL
9    }
10  })
11}
12 
13resource "aws_cloudwatch_event_target" "response_lambda" {
14  rule = aws_cloudwatch_event_rule.guardduty_high.name
15  arn  = aws_lambda_function.incident_response.arn
16}
17 
18# Lambda response (Python) — triggered on HIGH+ finding
19import boto3
20 
21def handler(event, context):
22    finding   = event['detail']
23    instance_id = (finding.get('resource', {})
24                          .get('instanceDetails', {})
25                          .get('instanceId'))
26    if instance_id:
27        ec2 = boto3.client('ec2')
Quarantine SG blocks all traffic while preserving instance for forensics
28        # 1. Apply quarantine SG — blocks all traffic, preserves for forensics
29        ec2.modify_instance_attribute(
30            InstanceId=instance_id,
31            Groups=['sg-quarantine-0abc1234']
Always snapshot before terminating — forensic evidence is critical
32        )
33        # 2. Snapshot root volume for forensic analysis
34        ec2.create_snapshot(
35            Description=f"Forensic: GuardDuty {finding['type']}",
36            VolumeId=get_root_volume_id(instance_id),
37        )

UEBA: Detecting Insider Threats and Account Compromise

UEBA builds a baseline of normal behaviour for each user and entity, then alerts when behaviour deviates significantly from that baseline.

What UEBA detects that rule-based systems miss

  • Slow-and-low data theft — A disgruntled employee downloads 100 files/day for 90 days — no single day trips a threshold, but UEBA detects the cumulative anomaly.
  • Impossible access patterns — A marketing employee accessing the payments database at 2am — normal credentials, abnormal behaviour for that identity.
  • Credential sharing — An account logging in from two cities simultaneously — impossible for one person.
  • Multi-signal anomaly — Unusual hours + new IP + sensitive resource access — each signal low individually, high risk combined.

UEBA needs 2–4 weeks to establish a baseline before generating reliable alerts

Enable UEBA on day 1 but do not act on findings for the first 2–4 weeks. The system is still learning. Acting on immature baselines produces alert storms from legitimate but unusual activity.

Threat Hunting: Proactive Detection

Threat hunting is the proactive search for threats that have evaded automated detection. Hunters assume attackers are already inside and look for evidence.

Threat hunting methodology

1

Form a hypothesis — based on threat intelligence ("APT29 uses registry run keys for persistence") or internal signals ("unusual outbound DNS to .ru domains")

2

Collect and query relevant data — search logs, EDR telemetry, network flows for the specific indicator or behaviour pattern

3

Analyse results — distinguish true positives from false positives; follow the kill chain for suspicious findings

4

Document findings — regardless of outcome: what was searched, what was found, what was ruled out

5

Automate — convert successful hunts into automated detection rules so the same pattern is caught automatically in future

Every successful hunt produces a new automated detection rule

Threat hunting feeds detection engineering. Over time, automation coverage improves and hunters can focus on novel TTPs rather than repeated manual searches.

Detection-as-Code with Sigma Rules

Detection-as-code treats detection rules as software — version-controlled, tested, peer-reviewed, deployed via CI/CD. Sigma is the YAML-based, vendor-neutral detection rule format.

Sigma rules compile to any SIEM query language

A Sigma rule written once compiles to Splunk SPL, Elastic EQL, Microsoft KQL, or CloudWatch Insights using the sigmac tool. Write once, deploy anywhere — no vendor lock-in.

sigma-iam-escalation.yml
1# Sigma rule — detect AWS IAM privilege escalation
2title: AWS IAM Privilege Escalation via Admin Policy Attachment
3id: a1b2c3d4-e5f6-7890-abcd-ef1234567890
4status: stable
5description: |
6  Detects when an IAM identity attaches AdministratorAccess or creates
7  a custom policy with iam:* permissions — common post-compromise escalation.
8references:
9  - https://attack.mitre.org/techniques/T1098/
10tags:
11  - attack.privilege_escalation
12  - attack.t1098.003
MITRE ATT&CK T1098.003 — Account Manipulation: Add Cloud Credentials
13 
14logsource:
15  product: aws
16  service: cloudtrail
17 
18detection:
19  selection_attach:
20    eventSource: iam.amazonaws.com
21    eventName: AttachUserPolicy
Catch AdministratorAccess attachment
22    requestParameters.policyArn|contains:
23      - 'AdministratorAccess'
24      - 'IAMFullAccess'
25  selection_create:
26    eventSource: iam.amazonaws.com
27    eventName: PutUserPolicy
Catch inline policy with iam:* wildcard
28    requestParameters.policyDocument|contains:
29      - '"iam:*"'
30      - '"Action":"*"'
31  condition: selection_attach or selection_create
32 
33falsepositives:
falsepositives — document expected triggers to reduce noise
34  - Legitimate IAM administration by security or infra teams
35  - Initial account bootstrap
36 
37level: high
38 
39# Compile to Splunk: sigma convert -t splunk <rule.yml>
40# Compile to KQL:   sigma convert -t microsoft365defender <rule.yml>

Key Metrics: MTTD and MTTR

Detection effectiveness is measured by two key metrics.

MetricDefinitionIndustry Average (2023)Target
MTTD (Mean Time to Detect)Time from breach to detection207 days (IBM 2023)< 24 hours
MTTR (Mean Time to Respond)Time from detection to full containment73 days (IBM 2023)< 4h critical / < 24h high
Alert PrecisionTrue positives / all alerts5–10% (most alerts are noise)> 30% before expanding coverage
Detection Coverage% of MITRE ATT&CK techniques with active rules< 20% for most orgs> 60% for critical techniques

Reduce MTTD, not just MTTR

Most orgs focus on MTTR (response speed). But MTTD is where attackers steal data — every day of undetected presence is a day of exfiltration. A breach detected in 6 hours with 2-hour MTTR beats one detected in 207 days with 1-hour MTTR.

How this might come up in interviews

Advanced threat detection appears in security engineer, SOC analyst, and DevSecOps architect interviews. Expect scenario questions: "An attacker has been in your environment for 3 months — how would you find them?"

Common questions:

  • What is MTTD and why does it matter more than MTTR?
  • How does UEBA differ from rule-based detection?
  • What is AWS GuardDuty and what does it detect?
  • Walk me through how you would threat hunt for a suspected credential compromise.
  • What is detection-as-code and what problem does it solve?

Strong answer: Knows MTTD vs MTTR difference, describes a threat hunting hypothesis, mentions Sigma for detection-as-code, and understands UEBA catches slow-and-low attacks rule-based systems miss.

Red flags: Thinking a firewall means you do not need detection, not knowing what GuardDuty or a SIEM is, or conflating MTTD with MTTR.

Quick check · Advanced Threat Detection

1 / 3

The SolarWinds SUNBURST attack was undetected for 9 months. Which capability would have had the best chance of detecting it earlier?

Key takeaways

  • MTTD is the most important security metric — every day of undetected attacker dwell time means more damage.
  • SIEM correlates events; EDR detects endpoint compromise; GuardDuty catches IAM and API abuse; UEBA detects behavioural anomalies.
  • Enable AWS GuardDuty (or equivalent) in every account and every region — ~$3/month, catches real attacks automatically.
  • UEBA detects slow-and-low attacks that threshold rules miss — insider threats, compromised accounts using normal credentials.
  • Threat hunting is proactive: form a hypothesis, query data, document findings, convert to automated rules.
  • Detection-as-code (Sigma) version-controls detection rules, enables peer review, and allows vendor-neutral rule sharing.
  • Every successful threat hunt produces a new automated detection rule — hunting feeds detection engineering.
  • The SolarWinds 9-month dwell time is the cost of poor detection coverage.
Before you move on: can you answer these?

A GuardDuty finding shows "Recon:IAMUser/UserPermissions" for a developer IAM user. What does this mean and what is your response?

This means the IAM user is making API calls to enumerate their own permissions (GetPolicy, ListPolicies, SimulatePrincipalPolicy) — reconnaissance behaviour: an attacker who has compromised the account is mapping what they can access. Response: (1) Immediately verify with the developer — if they did not make these calls, treat as account compromise. (2) If compromised: disable access keys, invalidate sessions, review CloudTrail for the past 24–72h. (3) Investigate how credentials were obtained. (4) Rotate all credentials for this identity.

What is the difference between a SIEM alert and a threat hunt?

A SIEM alert is a pre-written rule that runs continuously and fires automatically on known patterns — reactive, automated, catches known TTPs. A threat hunt is a proactive, hypothesis-driven manual investigation for attacker activity that may have evaded automated detection — exploratory, catches novel or subtle activity. Use SIEM alerts for high-confidence known patterns (brute force, privilege escalation). Use threat hunting when threat intelligence suggests a specific TTP, after a peer breach, or on a regular schedule to find slow-and-low attacks.

From the books

The Practice of Network Security Monitoring (Richard Bejtlich, No Starch Press)

Foundational guide to network security monitoring, threat detection, and incident response. Covers the analyst mindset and practical detection methodology.

Applied Network Security Monitoring (Chris Sanders & Jason Smith)

Practical NSM guide covering collection, detection, and analysis. Includes hands-on exercises with open-source tools (Zeek, Snort, Suricata).

🧠Mental Model

💡 Analogy

Security cameras and a 24/7 SOC — not just locks

⚡ Core Idea

Preventive controls (locks, firewalls, MFA) stop attackers at the door. Threat detection assumes the lock has been picked and asks: how quickly would we know? Detection is the security camera, the motion detector, and the SOC analyst who investigates the alert. Without detection, a burglar inside your house could stay for months.

🎯 Why It Matters

The SolarWinds attackers were inside for 9 months. The Equifax attackers exfiltrated data for 78 days. These were not discovered by preventive controls — they were discovered by detection (or reported externally). Breach cost scales linearly with dwell time. Cutting MTTD from 207 days to 24 hours reduces breach cost by an order of magnitude.

Ready to see how this works in the cloud?

Switch to Career Paths for structured paths (e.g. Developer, DevOps) and provider-specific lessons.

View role-based paths

Sign in to track your progress and mark lessons complete.

Discussion

Questions? Discuss in the community or start a thread below.

Join Discord

In-app Q&A

Sign in to start or join a thread.