SignalHunt
← All rules

RP-7004

high

Backend Origin Error Spike Behind Reverse Proxy

Detects a burst of 5xx responses from a single backend/upstream member within a short window. The reverse proxy sits directly in front of your origin, so a spike here means requests are actually reaching and breaking the backend - distinct from a WAF block (rejected before reaching the app) and worth treating as a possible application-layer DoS, resource exhaustion attack, or an attacker's fuzzer finding a crash path.

T1499.004
Test this rule in the simulator

Detection logic

Fires when at least 15 event(s) matching `upstream_status` is at least 500 happen from the same `upstream_addr` within 5 minute(s).

{
  "detection_type": "threshold",
  "selection": {
    "upstream_status|gte": 500
  },
  "groupby": "upstream_addr",
  "timeframe_minutes": 5,
  "threshold": 15,
  "condition": "gte"
}

Log source: category=reverse_proxy · Event ID(s): rp_access_log

Required log fields

  • upstream_addr
  • upstream_status
  • uri
  • client_ip

Enabling this in your environment

Where: Reverse proxy / load balancer access log configuration - log upstream_status and upstream_addr per request, separately from the status code returned to the client

nginx: log_format including $upstream_status and $upstream_addr; HAProxy: %ST (or %[fc_err]) and %si for the selected server.

Client-facing status and upstream (backend) status can differ - e.g. the proxy itself can return a cached 200 while the backend that filled the cache actually failed. Group by upstream_addr, not client_ip, to pinpoint which pool member is actually failing.

False positives

  • A genuine backend outage/deployment issue unrelated to attack traffic - check deployment logs/change history for the same window before treating this as adversarial
  • A single misbehaving legitimate client retrying aggressively against a real bug - correlate with client_ip diversity to distinguish "one confused client" from "distributed abuse"

References