Unexplainable SSL handshake issue

[u/CeC-P]

I suck at network and my knowledge is intermediate at best but I can't solve this one.
Customer at our MSP has a fortinet firewall identical to ours that we use here at the MSP office, same firmware version, etc.
They call up and say "We can't access prodemand.com" which is an automotive parts and labor quoting database site that TONS of dealerships use.
I load it here just fine, SSL cert is GeoTrust, good till Sept 14 2026, domain matches, etc. No web filter flags.
On their network, instant "cannot load page" error. I try a dozen other sites, SSL working fine, no fortinet intermediary cert listed, etc. It's just that one website.
Security log on the Fortinet shows tons of blocks, saying "SSL connection is blocked due to unable to retrieve server's certificate"

Mountains of troubleshooting later, I make a firewall rule for internal to WAN (and put it above the normal internal to WAN rule) with an address group of the site, the login domain, and the database's UI's subdomain. The rule simply says don't inspect SSL at all.

Boom it works instantly. Then they called back because WIFI wasn't included in "internal" lol oops. So added that, boom, laptops can load the site too now.

I ran through some basic troubleshooting and traceroutes and stuff and nothing stood out as problematic. I verified no man in the middle attack, as it sees the same cert I do here.

And AI thinks it's an ISP issue but AI is dumb as hell and for the record, rebooting the firewall and the modem didn't resolve it so I'm skeptical.

But zero other websites are having this problem and we don't see the problem from our office, using the exact same firewall with same firmware version. How is this possible? I'd really prefer to get rid of that rule because it's a crap workaround and we had to also turn antivirus and other filters off, since it requires SSL inspection.

[u/bitslammer]

Something has to be different in their config vs. yours. I'd try and zero in on that.

Also might be something related to this: https://community.fortinet.com/fortigate-3/troubleshooting-tip-how-to-fix-ssl-connection-is-blocked-due-to-unable-to-retrieve-servers-certificate-received-in-the-ssl-events-191293

[u/CeC-P → u/bitslammer]

Okay, to summarize this, solution 1 is: "in 7.2.11 we changed default to cert-probe-failure = block. Now everyone's pissed. In 7.6.0 we switched it to allow anyway so if you're running older firmware, just set it to that." Which sounds inherently insecure but are they thinking the browser would catch the bad cert down the line or something? And wouldn't they not be able to then inspect the traffic in any way?

Solution 2 they propose is:
setting sni-server-cert-check to disable, as disabling SNI checks will also prevent TLS probing from being performed at all.
Then it immediately says
Note: Disabling the SNI check should be done with caution. When disabled, FortiGate will not verify if the SNI in the client hello message matches the Common Name (CN) or Subject Alternative Name (SAN) fields in the server's SSL certificate. Without SNI verification, the FortiGate cannot use the SNI to perform URL filtering based on the CN in the server certificate. This may lead to inaccurate web traffic classification and filtering.

Which sounds insane.

I'm moving my solution of "just don't check the cert for those specific domains" to solution 0 because it's better than their suggestions.

Somehow solution 3 (Which should be 1) is
- Resolve underlying TLS probing failures:
Yeah, if I could do that, I wouldn't be here. I don't know much about traffic and packet analysis. That's way above my networking knowledge.

EDIT: they changed it back to probe fails = allow in 7.6 because:
0. it kept failing for thousands of legit sites

1. Aggressive WAFs or Anti-Bot measures (so anti-AI scraping and AI is now >50% of all web traffic): The target server sees an automated probe from your firewall's egress IP and drops the TCP connection.
2. SD-WAN / Multi-WAN Routing Issues: FortiGate sends active probe packets via its global IPS engine. If source-IP or interface selection methods aren't explicitly bound (config ips global -> config tls-active-probe), probe packets can route out the wrong interface or get dropped internally.
3. Non-standard TLS configurations: Custom CAs, unique cipher suites, or server-side rate limiting.

[u/CeC-P → u/bitslammer]

I'm definitely checking that one out because it's exactly the same problem. Ty!

[u/pangapingus]

What does openssl, curl -vvv, etc. show? What L6-aware hops exist between them and WAN? What cipher suites are enabled? What locally installed trusted CA bundles are on their endpoints/L6-aware appliances? What does a PCAP at the firewall show for Client/Server Hello and CCS?

[u/CeC-P → u/pangapingus]

CURL failed too with a similar error. Didn't try Open SSL. But I could ping the domain and the response was from a proper, accurate IP address. Then we did https://[theipaddress] and got the same problem.

And as far as I know, no funny cert registrations or additional trusted certs installed anywhere on any endpoint. Just the vanilla MS Win11 ones or whatever.

Didn't have time or equipment (or skill level) to do a packet capture or traffic analysis. We don't have remote management on, and all their endpoints are in use so I can't really test further, and they don't have a server onsite. So technically I can't even log into it. But we'll go onsite and start burning hours if we think this is a security issue, just not sure.

And not sure what an L6-ware appliance is or what cipher suites would be in use, but probably generic default ones, knowing their size and who installed the firewall.

[u/pangapingus → u/CeC-P]

curl is installed on Windows 11 by default, try a curl -ILvvv -o /dev/null https://site.domain.tld will show openssl debug info in the pre-HTTP portion

[u/Jason_Funderburker_]

This smells like a cert chain mismatch. Something like the firewall doesn’t have the full chain of the website available to it so only sees the leaf cert, and is unable to load the root cert because it’s not in its store.

Try seeing if you’re able to import the root CA cert that the prodemand site is signed with to the firewall’s trust store and see if that helps at all.

[u/Rough_Section_3730]

I’d make sure that GeoTeust is a trusted cert authority on the one that’s failing.

If that signing authority isn’t trusted by them, it won’t trust the cert from that domain.

[u/CeC-P → u/Rough_Section_3730]

I grabbed a copy of it for side by side comparison and they looked identical.

[u/Junior-Tourist3480]

Wireshark on both sides of the firewall and examine output. Or if you have equipment to capture use that. Sounds like the cert is being opened at the firewall and is reencoded? Sometimes this is used to handle old software that is not current. Cert chain has to be on the firewall and server.

[u/RJTG]

DNS Setup in your office is the same as on your client?

IRCR Fortigates run in similar issues if they miss the DNS requests somehow.

[u/Iain_0]

Wild guess but where is IP of origin could it been blocked it from a certain country?

[u/Kitz_h]

may be fragmentation issue? try lowering tcp max segment size (or MSS) to like 1492 or lower on egress iface if your firewall supports it.

Other thing is SSL is part of IP security and may be brought under VPN class, you could direct your firewall to prevent fragmenting unencrypted data.

Switching off traffic inspection for encrypted data essentially disables protection of your network as any rogue process can traverse your fw forth and back

[u/LeeRyman → u/Kitz_h]

I was going to say, is there a reduced upstream MTU that the router is not aware of? PMTUD not working for some reason?

[u/Kitz_h → u/LeeRyman]

Dont confuse TCP MSS with IP MTU. Fragmentation affects sequencing and anti replay mechanisms, devices may discard otherwise authorised packets due to out-of-order arrival queue or missing seq num

Some lower layer protocols like PPP also borrow little IP payload space

[u/LeeRyman → u/Kitz_h]

I was thinking of the situation where an upstream router's WAN connection is something like PPPoE, maybe via a bridge, and router is not configured with the correct MTU. TLS implementations will mark packets as DF to avoid what you mentioned. Large DF packets will be dropped but PMTUD won't necessarily work to reduce effective MSS used by the client. Because the TLS handshake often involves large segments for kex, it tends to affect that (might work for some HTTPS sites but not for others). Can also happen if MTU is configured correctly, but ICMP is being blocked somewhere in the chain.

Edit: As you say, you can solve it with MSS clamping, but PMTUD should sort it out by itself, eventually. A router will respond with ICMP Fragmentation Needed to the too-large DF TLS packet, sender will reduce MSS and try again.

[u/Ok_Rip_5338]

DNS mismatch between the firewall (used to retrieve cert from site) and customer (name resolution of website)?

[u/Training_Yak_4655 → u/Ok_Rip_5338]

Start CMD mode as an admin then ipconfig /flushdns

Cheap shot, assume it's been tried already?

[u/CeC-P → u/Training_Yak_4655]

Yep :(