The ticket always reads the same: "VPN connected, can ping, cannot open shares." The tunnel is up, IKE is happy, small packets flow, and anything larger than a few hundred bytes hangs forever. Nine times out of ten the cause is MTU, and it takes under a minute to prove.
IPsec adds 50 to 73 bytes to every packet. A 1500 byte packet from a workstation becomes 1550 to 1573 bytes on the wire. If the path only carries 1500, the gateway has to fragment, and modern hosts set the Do Not Fragment bit, so the packet is silently dropped instead.
Step 1Prove it is MTU in 30 seconds
Ping across the tunnel with the DF bit set and a payload that would make a full size packet. On Windows the payload size excludes the 28 byte ICMP and IP headers, so 1472 tests a 1500 byte packet.
ping -f -l 1472 10.2.0.10
# Reply: "Packet needs to be fragmented but DF set" = MTU problem
# Linux
ping -M do -s 1472 10.2.0.10
Now step the size down until it succeeds. The largest payload that works, plus 28, is the effective path MTU. For an IPsec tunnel over a 1500 byte link it usually lands between 1400 and 1420.
This is the whole diagnosis. If ping succeeds at 1300 and fails at 1472, the firewall rules are not the problem and neither is the tunnel. Stop reading logs and go to step 2.
Steps 2 to 5The checklist
The steps below are in order and each one is independent, so a partial fix still improves things. The first two are the actual repair. The third is what stops the problem coming back somewhere else, and the fourth is what tells you it is really fixed.
Every vendor names these differently. On Cisco the clamp is ip tcp adjust-mss, on Fortinet it is a per policy tcp-mss-sender setting, on a Linux gateway it is one iptables rule in the mangle table with --clamp-mss-to-pmtu. The number is the same everywhere.
ReferenceThe clamp on each platform
The number is 1360 everywhere. Only the syntax changes, and the setting always belongs on the gateway rather than on the hosts, because a gateway rule covers the laptop nobody manages.
| Platform | Where it goes | The setting |
|---|---|---|
| Cisco IOS | Tunnel interface | ip mtu 1400 and ip tcp adjust-mss 1360 |
| Cisco ASA | Global | sysopt connection tcpmss 1360 |
| FortiGate | Per firewall policy | set tcp-mss-sender 1360 and set tcp-mss-receiver 1360 |
| pfSense and OPNsense | IPsec advanced settings | Enable Maximum MSS, value 1360 |
| Linux, iptables | mangle FORWARD | -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu |
| MikroTik | Firewall mangle | chain=forward protocol=tcp tcp-flags=syn action=change-mss new-mss=1360 |
Two of these deserve a note. The Linux rule clamps to whatever the path discovers rather than to a fixed number, which is the better behavior when the overhead varies, and it only works if path MTU discovery is not blackholed.
The FortiGate settings are directional and both are needed: a clamp applied in one direction fixes half the sessions and produces a fault that looks intermittent.
BackgroundWhy it happens at all
Ethernet carries 1500 bytes of IP. IPsec in tunnel mode wraps the original packet in a new IP header (20 bytes), an ESP header (8), an initialization vector (8 to 16), padding, an ESP trailer (2) and an integrity check (12 to 16).
Behind NAT, a UDP header (8) is added for NAT traversal. A full size packet no longer fits, and the DF bit means it cannot be cut into two.
What should happen next is path MTU discovery: the gateway sends back an ICMP "fragmentation needed" message naming the size that would fit, and the sender adjusts.
What usually happens instead is that somebody has blocked ICMP at a firewall, the message never arrives, and the sender keeps retrying the same doomed packet until the application gives up. That is why the symptom is a hang rather than an error.
| Header added | Bytes | When |
|---|---|---|
| New IP header | 20 | Always, tunnel mode |
| ESP header and IV | 16 to 24 | Always |
| ESP trailer and ICV | 14 to 18 | Always |
| UDP header, NAT traversal | 8 | Behind NAT |
| Total overhead | 50 to 73 | Why 1400 is safe |
EvidenceWhat it looks like in a capture
Ping is enough to diagnose this, and a capture is what convinces somebody else. Take it on the client, start the transfer that hangs, and look for three things.
The handshake completes. SYN, SYN and ACK, ACK are all small packets, so they cross a constricted path without trouble. This is why the connection appears to open and the application appears to be at fault.
Then one full size segment is sent and sent again. Wireshark marks it as a retransmission, the sequence number does not move, and the interval between attempts doubles each time. Nothing is acknowledged because nothing arrived, and the sender has no way to know the packet was too large rather than lost.
And the message that would have explained it is absent. Filter for icmp.type == 3 && icmp.code == 4 and expect nothing. If one does appear, read the next hop MTU field inside it: that is the real path MTU, measured rather than guessed.
A capture with no such packet anywhere is the signature of a discovery blackhole, and it means somebody upstream is dropping ICMP.
One more filter is worth knowing once the clamp is in place. tcp.options.mss_val on the SYN shows the value each side announced. If the gateway rewrote it, the capture reads 1360 rather than 1460, which is how you prove the clamp is applied in the direction you thought it was.
The hard caseWhen a second tunnel is stacked on yours
An MTU of 1400 assumes your tunnel is the only one. It often is not. A GRE tunnel inside IPsec adds another 24 bytes, an SD-WAN overlay adds its own header, and a carrier can encapsulate the whole thing again without telling anybody. Each layer takes its bite out of the same 1500.
| What is in the path | Overhead | Tunnel MTU | MSS clamp |
|---|---|---|---|
| IPsec tunnel mode | 50 to 73 | 1400 | 1360 |
| GRE inside IPsec | 74 to 97 | 1400 | 1360 |
| IPsec over a PPPoE circuit | 58 to 81 | 1392 | 1352 |
| IPsec inside an SD-WAN overlay | varies | measure it | MTU minus 40 |
The last row is the honest one. When something else in the path is encapsulating traffic and you do not control it, the ping test from step 1 is not a diagnostic any more, it is the measurement. Find the largest payload that crosses, add 28, and that number is your MTU. Subtract 40 and that is the clamp.
PPPoE is worth calling out because it is easy to miss. It takes 8 bytes before your tunnel starts, so a circuit that looks like plain Ethernet is already carrying 1492, and every calculation above shifts down by 8.
Ruling it outWhen it is not MTU
Three cases look identical from the help desk and are not this.
Everything fails, including small pings. That is routing or a policy, not MTU. MTU problems always let small packets through, which is the whole reason they are confusing.
Only one application hangs. If large pings pass at 1472 and only one program stalls, look at that application. MTU does not select which program it breaks.
It works from one site and not another. Possible either way. Test with the same ping from both. If the failing size differs between sites, one path has extra overhead, often a carrier tunnel or an SD-WAN overlay stacked on top of yours.
- Set the tunnel interface MTU to 1400
On a VTI or tunnel interface, 1400 leaves room for every ESP and NAT traversal combination. Lower it only if a carrier adds overhead of its own.
- Clamp TCP MSS to 1360
MSS is MTU minus 40 bytes of IP and TCP headers. Clamping on the gateway fixes every TCP session without touching a single workstation, including the ones you do not manage.
- Allow ICMP type 3 code 4 through the firewall
Fragmentation needed messages are how path MTU discovery works. Blocking all ICMP breaks it and hides the real MTU from every host on both sides.
- Re-test with a real transfer, not just ping
Copy a 100 MB file across the tunnel. If it moves at line rate, close the ticket. If it stalls, check for a second tunnel or an overlay adding more overhead.
The numbers to remember
They are the same on every vendor. Only the command names change.
FAQQuestions from the comments
Does this apply to WireGuard and SSL VPN too?
Yes. WireGuard defaults to an MTU of 1420 for exactly this reason. An SSL VPN over TCP hides the symptom because TCP handles segmentation, but it shows up as poor throughput instead of hangs.
Why not just set MTU to 1400 on every workstation?
You can, and it works. The gateway clamp fixes all of them at once and keeps working for laptops nobody manages, which is why it is the better place to do it.
Ping works with large packets but SMB still hangs. Now what?
Check for a second tunnel or an SD-WAN overlay adding more overhead, and confirm the MSS clamp is applied in both directions. A clamp on one side only fixes half the sessions.
Is a lower MTU bad for performance?
Slightly, and far less than the alternative. Dropping from 1500 to 1400 costs about seven percent of header efficiency. A tunnel that hangs costs everything.
What size should I actually test with?
Start at 1472 on both Windows and Linux, because both of those tools take a payload size that excludes the 28 bytes of ICMP and IP header, so 1472 tests a 1500 byte packet. Halve the difference each time after that: if 1472 fails and 1200 works, try 1336. Four or five attempts land on the exact figure.
Does any of this apply to IPv6?
The symptom is the same and the mechanism is stricter. Routers are not allowed to fragment IPv6 at all, so only the sender can, which makes path MTU discovery mandatory rather than an optimization. The message to allow is ICMPv6 type 2, Packet Too Big, and the MSS arithmetic subtracts 60 rather than 40 because the IPv6 header is larger.
Will jumbo frames on the LAN help?
No, and they can hurt. A 9000 byte frame is a local agreement between two devices on the same switch. The moment the traffic reaches a router heading for the tunnel it has to fit inside 1500 again, so the only thing a jumbo LAN changes is where the packet gets too big.
Why does the transfer sometimes work for the first few seconds?
Because TCP grows its window from small segments. The handshake and the first exchanges fit, so the connection opens and a little data moves, and the stall arrives when the sender reaches full size segments. That pattern, a start followed by a freeze, is close to diagnostic on its own.
Is the MSS clamp a workaround or a real fix?
A real one, and the one every vendor recommends. Path MTU discovery is the designed mechanism and it depends on an ICMP message that firewalls across the internet drop, so relying on it means relying on strangers. Clamping sets the ceiling at the only point that knows the true overhead, which is your own gateway.
The clamp is applied and one direction is still slow.
Clamping only touches TCP. UDP has no handshake to rewrite and no segment size to negotiate, so an application sending large UDP datagrams, which is most VoIP media and some backup software, still needs the tunnel MTU set correctly and still needs ICMP type 3 code 4 to reach it.