OpnSense 22.7 and 23.1.x Hung Telnet and STARTTLS SMTP

Mindwatering Incorporated

Author: Tripp W Black

Created: 06/25/2023 at 10:00 PM

 

Category:
Linux
Other

Issue:
"Hung" Telnet and STARTTLS SMTP connection when behind OpnSense Firewall 22.7 or 23.1_6.

Telnet:
Telnet connects, commands can be sent, but servers are not closed when they issue the QUIT command.
The connection is eventually dropped by the back-end server, and no email is received.

SMTP Insecure:
Servers can make connection, run through the EHLO, MAIL FROM, RCPT TO, and DATA. The entire email is sent but the the "." is not replied to, and the connection "hangs" until timeout. The mail may or may not be received depending if the receiving server accepts the mail queued for delivery w/o a QUIT sent at the end.

START TLS:
The connect is successful. However, the STARTTLS negotiation does not occur and the certificates are never presented to the SMTP client.


Causes:
We have found two causes for this condition:
1. 9000 MTU is not liked by all public ISP routers. For one of our ISPs, we have to use 2000, for another 1800, and our fiber one works best with 9000 MTU or 1500 MTU. Otherwise "small" packets are lost leaving the firewall (successfully), but redirected back by the ISP routers, and then the firewall sees them as in improperly tagged packet and blocks them. This does not happen with all set-ups, but only happens after we upgrade from 21.x to 22.7 and 23.x versions of the firewall.

or

2. The Anti DDOS "Enable syncookies" option.
This seems to be very effective for HTTP protocols, but it causes the last packet of the SMTP send or the STARTTLS renegotiation to fail. Again, this was only a problem after we upgraded our firewalls to 23.1_6.


Solution for Cause #1:
Update the MTU to the recommended ISP default for their router. Then when you find out their supplied information is incorrect, try other numbers. In other words, one said it was 1500 MTU, but only 1800 worked. The other said anything up to 9000 MTU works, but only 1500 MTU and 2000 MTU worked. The fiber ISP's endpoint had no issue with 1500 MTU or 9000 MTU.


Solution for Cause #2:
Log into the OpnSense appliance. Navigate to Firewall --> Settings --> Advanced. Change the Enable syncookies field from Enabled to never (default).







previous page

×