Tuesday, November 4, 2014

Salesforce integration failures due to disabled SSL 3.0 (POODLE)

In response to the published security vulnerability on SSL 3.0 encryption (POODLE) Salesforce has put out the Knowledge Article Salesforce disabling SSL 3.0 encryption.

One important section with regards to web services that are called by Salesforce: (my emphasis)

Additionally, Salesforce recommends customers disable SSL 3.0 encryption in their own IT environment as soon as possible, unless they use call-out integrations. If a customer uses call-out integrations, and they have not already disabled SSL 3.0 in their own environment, then Salesforce recommends that they wait until after Salesforce has disabled SSL 3.0 for outbound requests.

The current timeline for outbound requests dropping support for SSL 3.0 is the 3rd of December 2014 for sandboxes and the 10th of December 2014 for production orgs.

If you work with a web service that disables SSL 3.0 before then you can start seeing exceptions like:

Message:  IO Exception: Remote host closed connection during handshake

I've also seen it reported manifesting in Batch Apex as:

Received fatal alert: handshake_failure

This can occur even if the web service supports TLS1.0 or higher.

Quotes attributed to support:

"We will use TLS with callouts but if it fails. We drop down to SSL and hard code to send it via SSLv3 for 24 hours or an app restart. Which ever comes first. This should address any changes that occur with the way other companies integrate with Salesforce until we completely disable SSL 3.0 on December 10th." Source
"I am from Salesforce Developer Support Team. I have taken ownership of your case regarding POODLE vulnerability. At present some outbound calls are initiated using SSLv3 ClientHello, so if this is disabled on your server, there'll be a handshake failure. Until then, it is advised that you support this for incoming calls (received from Salesforce). At present R&D and our Tech Ops organization are aggressively working on a strategy around this. Once this is finalized, there will be a tech comm broadcast as expected."
"The R&D team shall be releasing this on 11/4/2014. After that you may turn off SSLv3 without running into the handshake failures"

This presents a bit of an immediate problem. Salesforce is trying to fallback to SSL 3.0 on services that only support versions of TLS. At this stage, if you don't have control of the web service or the ability to get it to accept SSL 3.0 again until mid December the only option might be to wrap it in a proxy service that does support SSL 3.0 encryption.

Another fun part of the change is that Inbound SSL 3.0 support will start phasing out from the 7th of November. Between then and the 10th of December there will be cases where Salesforce servers won't be able to call Salesforce hosted web services. At least some of the time anyway when they aren't successfully using TLS.

Update 6/11/2014: Indications are that Salesforce have been silently updating servers to prevent the fallback to SSLv3 if the target server doesn't support it. It's hard to confirm what is going on as there isn't an official known issue that can be linked to.

Update 7/11/2014: Salesforce support responded to some of the outstanding issues:

All I see you guys have various questions and I think I can answer quite a few of them, I manage part of the technical support security team at salesforce. Email me at bestebez@salesforce.com (you can look at my linkedin profile if you question who i am).

Questions that have been raised around Salesforce’s support of SSL 3.0 and TLS 1.0. While we are in the process of disabling SSL 3.0, Salesforce currently supports TLS 1.0 and TLS 1.2 for inbound requests and TLS 1.0 for outbound call-outs.
Our Technology Team has been actively working to address an issue that causes outbound call-outs to use SSL 3.0 more frequently than they should, given we have TLS 1.0 enabled. We understand that this may have caused issues for customers who have already disabled SSL 3.0 in their call-out endpoints. We released a fix to Sandboxes last Friday, October 31, and plan to release the fix to production instances during off-peak hours on Wednesday, November 5, 2014.
Many customers and partners who have tested this fix in their Sandboxes have reported successful connections using TLS 1.0. A few customers continued to experience TLS 1.0 issues on their Sandboxes, and our Technology team is working with them to find a solution.

There was an issue specifically to Na14 that was generating more outbound messages that were using SSLv3 but that has since been fixed. That is probably why a few of you guys saw an issue with.

See Also: