5

I have small Rails app that performs various checks on our platform and sends me an email in case of an issue. Everything was running fine until today i started getting alerts about the following error:

SSL_connect returned=1 errno=0 state=error: certificate verify failed (certificate has expired)

Now the problem is the certificate in question is valid, it gets automatically renewed (Let's encrypt) and this code has been untouched for a couple of years and never had any issues before and suddenly this started to happen.

The code that throws the exception:

def get_request url
  uri = URI.parse(url)
  http = Net::HTTP.new(uri.host, uri.port)
  http.use_ssl = true

  #more than 10 seconds this is too slow
  http.open_timeout = 10
  http.read_timeout = 10

  request = Net::HTTP::Get.new(uri.request_uri)
  response = http.request(request)

  if response.code.to_i == 200
    return true
  else
    puts "Failed to GET #{url}: #{response.code.to_i}"
    return false
  end
end

If i open the site in the browser, it shows the secure connection without issues and shows that is using a valid certificate, furthermore if i check with certbot i get the following: Expiry Date: 2021-11-22 17:48:58+00:00 (VALID: 52 days) so clearly the certificate is valid, why suddenly rails is throwing a tantrum about it?

Note that i have restarted Nginx just in case, that didn't help.

Additional info: Ubuntu 16.04.5, OpenSSL 1.0.2g 1 Mar 2016, Rails 4.2, Ruby 2.6.5

EDIT:

This error also happens with a different url, which also has a valid certificate.

EDIT 2:

I've isolated the problem, it is related to Let's Encrypt DST Root CA X3 that has expired. A lot of people are dealing with this issue, i'll report my solution once i find one.

Julien
  • 2,217
  • 2
  • 28
  • 49
  • Does `http.verify_mode = OpenSSL::SSL::VERIFY_NONE` work? – razvans Oct 01 '21 at 11:11
  • Does this help - https://stackoverflow.com/questions/7969688/how-to-connect-to-an-https-server-with-an-invalid-certificate-using-the-latest-v ? – razvans Oct 01 '21 at 11:14
  • @razvans yes `VERIFY_NONE` prevents the issue but that's not exactly a solution, now if one of the certificates actually expires the system wont emit an alert about that, the problem here is to understand why this is happening and to fix it without "ignoring" it. – Julien Oct 01 '21 at 11:32
  • A LetsEncrypt root cert expired yesterday. See: https://www.openssl.org/blog/blog/2021/09/13/LetsEncryptRootCertExpire/ – Matt Caswell Oct 01 '21 at 14:43
  • @MattCaswell yep that was exactly the problem – Julien Oct 01 '21 at 14:44

2 Answers2

7

So after reading through this long thread of the Let's Encrypt community, the solution for my case ended up being to remove the DST Root CA X3 certificate:

sudo rm /usr/share/ca-certificates/mozilla/DST_Root_CA_X3.crt
sudo update-ca-certificates

After that no more errors from openssl.

Julien
  • 2,217
  • 2
  • 28
  • 49
  • This worked for me. I decided to `mv` the file rather than `rm` just in case things broke. – Darren Greaves Oct 02 '21 at 12:50
  • I'm having the same issue, but when I run these commands on my console (running on a Macbook Pro with the new M1 chip in case that matters), I get "No such file or directory" and "Command not found" respectively. – Etan Mizrahi-Shalom Oct 14 '21 at 21:08
  • @etan that's because my answer is for Ubuntu 16, try reading the thread i linked in my answer, the solution for Mac os should be there as well. – Julien Oct 15 '21 at 22:17
0

I had this exact issue. I finally tracked it down to this gem: https://github.com/stevegraham/certified. Unfortunately, another gem we had included in our Gemfile had listed certified as a dependency. This gem provides certificate bundle back from 2014 and is extremely out of date. Check your Gemfile.lock to see if certified is listed. If it is, remove the gem(s) that require this dependency and see if that solves your issue.

Eric Terry
  • 364
  • 2
  • 10