SYN Flood Protection on Linux: What the Kernel Can and Cannot Stop

A SYN flood does not have to fill your port to take a service down, only one queue: the half-open TCP connections waiting on a listening socket. This guide to syn flood protection on Linux covers the kernel's defences, nftables synproxy, the counters that confirm an attack, and where only upstream filtering helps.

What a SYN flood exhausts

In the three-way handshake (RFC 9293), the client sends a SYN, the server answers with a SYN-ACK and enters SYN-RECEIVED, and the connection exists only when the client's ACK arrives. Until then the server holds state for a connection that may never happen.

RFC 4987 describes the attack: SYNs, often from spoofed addresses that never answer, sent to keep that backlog of half-open connections full so real clients are turned away. It targets one listening application; established and outgoing connections carry on, so a small flood can leave SSH working while the website refuses visitors.

SYN cookies: on by default

Linux's answer is net.ipv4.tcp_syncookies. The IP sysctl documentation (kernel 7.3.0) says it sends cookies "when the syn backlog queue of a socket overflows". The default is 1; 2 sends them unconditionally. The page does not define 0; in the kernel source, 0 drops the SYN when the queue is full, logging "Dropping request" instead of "Sending cookies".

A cookie encodes the connection's state in the SYN-ACK's sequence number instead of storing it (RFC 4987, section 3.6). If the client's ACK returns a valid cookie, the connection is rebuilt from it; a spoofed SYN costs one SYN-ACK and no stored state.

Leave it at 1. The documentation calls cookies a "fallback facility" and suggests 2 for testing their side effects.

The queue that triggers cookies

Three numbers matter:

• The application's listen() backlog. The listen(2) manual says it sizes the queue of established connections waiting for accept(), silently capped at net.core.somaxconn.
• net.core.somaxconn. Default 4096 since Linux 5.4, 128 before.
• net.ipv4.tcp_max_syn_backlog. The documented per-listener limit on SYN_RECV requests: at least 128, growing with memory, about 304 bytes each.

In the Linux 7.3 source, the SYN queue counts as full once it holds more requests than the listener's listen() backlog; tcp_max_syn_backlog is consulted only when cookies are off, and listen(2) says it is ignored with syncookies enabled. So with the default of 1, the application's backlog, capped at somaxconn, decides when cookies start. That holds since Linux 4.4; earlier kernels sized the queue from the smaller of the backlog and tcp_max_syn_backlog.

nginx's documentation gives its listen backlog a default of 511 on Linux, so at the default somaxconn of 4096, raising it changes nothing for nginx until you also set backlog=.

A conservative sysctl file

Save as /etc/sysctl.d/60-syn-flood.conf and load with sysctl --system:

net.ipv4.tcp_syncookies = 1
net.core.somaxconn = 4096

From Linux 5.4 both lines restate defaults, so nothing quietly changes them; on older kernels the second raises the cap from 128.

Guides often add net.ipv4.tcp_synack_retries = 3. With a 1-second initial timeout, the documented default of 5 puts the last SYN-ACK retransmission at 31 seconds (1 + 2 + 4 + 8 + 16) and the timeout at 63 (31 + 32); at 3, they become 7 (1 + 2 + 4) and 15 (7 + 8). RFC 4987 (section 3.3) calls shortening this timer flawed: attackers send proportionally faster, and some real connections fail to complete. The kernel already prunes under pressure: once a listener's queue is over half full, mostly with requests that have timed out once, it cuts their retries to as few as 2. Leave it at 5.

With cookies off, the backlog still caps the queue, and tcp_max_syn_backlog adds one rule: past three-quarters of its value, the kernel admits only peers it has seen before. A 4,096-request queue costs 4,096 × 304 = 1,245,184 bytes, about 1.2 MB per listener. RFC 4987 calls bigger backlogs, like shorter timers, "measurably problematic".

What SYN cookies cost

• TCP options. RFC 4987 warns that common cookie schemes clash with some TCP options. For IPv4, the current Linux source encodes one of four MSS values (536, 1300, 1440, 1460), so a client announcing 1,400 gets 1,300. Window scaling, SACK and ECN survive only with TCP timestamps on, stored in the timestamp's low bits.
• Server-speaks-first protocols. If the client's final ACK is lost, the server never learns the connection exists. RFC 4987 names SMTP; the kernel documentation warns of degradation "visible not by you, but your clients".
• Data on the SYN is not handled (RFC 4987).

So keep net.ipv4.tcp_timestamps at its default of 1, and treat cookies being sent as an alarm.

Firewall help: synproxy, conntrack and rate limits

A stateful firewall adds a second table to fill: conntrack. The nftables wiki's answer is synproxy, which completes the handshake with SYN cookies instead of conntrack, so a flood cannot exhaust conntrack and only clients that finish the handshake reach the service. The wiki's example, adapted to port 443:

table ip synflood {
  chain pre {
    type filter hook prerouting priority raw; policy accept;
    tcp dport 443 tcp flags syn notrack
  }
  chain in {
    type filter hook input priority filter; policy accept;
    tcp dport 443 ct state invalid,untracked synproxy mss 1460 wscale 7 timestamp sack-perm
    ct state invalid drop
  }
}

It needs nf_conntrack_tcp_loose at 0 and tcp_syncookies and tcp_timestamps at 1. The mss and wscale values must match your server's (nft(8) shows how to read them with tcpdump). The example is IPv4 only. Loose tracking off stops conntrack picking up existing connections, and the last rule drops invalid packets: apply it from a console, not over SSH. Choosing a firewall for a Linux server covers nftables. RFC 4987 (section 3.8) notes a proxy makes the flood the firewall's problem; here that firewall is your CPU.

Per-source rate limits (the first-hour DDoS runbook has an nftables example) stop a modest number of real addresses, not spoofed SYNs drawn, as RFC 4987 puts it, from a list of unresponsive addresses. A hypothetical 1,000,000 SYNs a second over 100,000 forged sources is 1,000,000 ÷ 100,000 = 10 per address, half of a 20-a-second limit. A global SYN limit trips, but drops real visitors too.

How to tell you are under a SYN flood

ss -s
ss -Htn state syn-recv | wc -l
nstat -az 'TcpExtSyncookies*' 'TcpExtListen*'

The first two give the socket summary and half-open count. Cookie connections hold no state and never appear in that count, so with cookies on, the source suggests it levels off near the attacked listener's backlog instead of climbing with the flood. nstat -az prints absolute values, zeros included; the kernel's SNMP counter documentation defines them:

• TcpExtSyncookiesSent rising: a listener's queue overflowed.
• TcpExtSyncookiesRecv: cookie replies received. If Sent climbs far faster than Recv, most cookie SYN-ACKs go unanswered: consistent with spoofing, not proof of it.
• TcpExtListenOverflows: connection attempts dropped because the accept queue was full. That usually points at an application not calling accept() fast enough, which no cookie fixes.

dmesg also logs "Possible SYN flooding on port", but only once per listening socket, so a quiet log proves little. If the warning comes from legitimate load, the documentation says, the server is misconfigured, not attacked.

Where the server loses anyway

Everything above runs after the packet has crossed your port. RFC 4987 leaves high packet-rate attacks out of scope: they target the network, and their rate matters more than packet type. SYNs are small, so a flood can exceed the packets per second your NIC and kernel can process, or fill the port, before any queue matters (Gbps vs Mpps has the arithmetic). Then filtering has to happen upstream.

NexonHost's DDoS protected server page lists "SYN floods and connection exhaustion" among the attacks its included 100 Gbps Layer 4 tier covers, and says floods "are dropped upstream, before they reach your port".

SYN flood protection on NexonHost servers

Host settings and upstream filtering fix different failures; you want both. For syn flood protection beyond what one server can absorb, every NexonHost DDoS protected dedicated server, from €129 a month, and every Linux VPS, from €9 a month, includes 100 Gbps of Layer 4 DDoS protection; the dedicated server page says there is nothing to enable. Upgrades run from 500 Gbps at €149 a month to 10 Tbps with Layer 7 filtering at €4,499 a month (price per Gbps of each tier).

What it does not solve: HTTP floods on the included tier, and an application that drains its accept queue too slowly. If TcpExtListenOverflows climbs under normal traffic, a bigger tier will not help; fix the application or its backlog. The new server security checklist covers the rest of a baseline. Start on a one-month term.

Prices as of 26 October 2026, in euros excluding VAT; check the product page before you order.

Sources

• RFC 4987 — TCP SYN Flooding Attacks and Common Mitigations, IETF, retrieved 26 October 2026: high packet-rate attacks out of scope (1), attack description (2.2), SYN-RECEIVED timer (3.3), SYN cookies and their TCP option and SMTP limits (3.6), firewalls and proxies (3.8), analysis (4).
• RFC 9293 — Transmission Control Protocol, IETF, retrieved 26 October 2026: connection states (3.3.2) and the three-way handshake (3.5).
• IP Sysctl — Linux kernel documentation, version 7.3.0, retrieved 26 October 2026: tcp_syncookies (default 1, 2 = unconditional, fallback facility, degradation warning), tcp_max_syn_backlog (minimum 128, about 304 bytes per request), tcp_synack_retries (default 5, 31 s and 63 s), somaxconn (4096, 128 before 5.4), tcp_timestamps (default 1).
• SNMP counter — Linux kernel documentation, version 7.3.0, retrieved 26 October 2026: TcpExtSyncookiesSent, TcpExtSyncookiesRecv, TcpExtListenOverflows.
• Netfilter conntrack sysctl — Linux kernel documentation, version 7.3.0, retrieved 26 October 2026: nf_conntrack_tcp_loose.
• Linux kernel source, 7.3 (master branch; the cited functions match the v7.3 tag): tcp_input.c, syncookies.c, inet_connection_sock.h, inet_connection_sock.c and proc.c, retrieved 26 October 2026: "Possible SYN flooding" messages for 0 and 1, logged once per listening socket; SYN queue full at the listen() backlog; tcp_max_syn_backlog used only without cookies (three-quarter rule for proven peers); SYN-ACK retries cut when the queue is over half full; the four-value IPv4 MSS table; options in timestamp bits.
• Linux kernel source, v4.3 request_sock.c and v4.4 inet_connection_sock.h, retrieved 26 October 2026: up to 4.3 the SYN queue was sized from the smaller of the backlog and tcp_max_syn_backlog; from 4.4 it is the listen() backlog.
• listen(2) — Linux man-pages 6.19, retrieved 26 October 2026: backlog semantics, somaxconn cap and defaults, tcp_max_syn_backlog ignored with syncookies.
• Synproxy — nftables wiki, retrieved 26 October 2026: operation, prerequisites, example ruleset, mss and wscale must match.
• nft(8) — nftables manual page, Debian, retrieved 26 October 2026: synproxy statement, reading backend options with tcpdump.
• ngx_http_core_module — nginx documentation, retrieved 26 October 2026: listen backlog default 511 on Linux.
• ss(8) and nstat(8) — iproute2 manual pages, and sysctl(8) — procps manual page, retrieved 26 October 2026: -s, -H, the syn-recv state filter, -a, -z, wildcards, --system.
• NexonHost DDoS protected dedicated servers and the NexonHost store, retrieved 26 October 2026: attacks covered by the included tier, upstream filtering, tier prices, VPS and dedicated starting prices.
• Computed: 1 + 2 + 4 + 8 + 16 = 31 s and 31 + 32 = 63 s; 1 + 2 + 4 = 7 s and 7 + 8 = 15 s; 4,096 × 304 = 1,245,184 bytes; 1,000,000 ÷ 100,000 = 10 SYNs per address.