[Solved] urllib.error.URLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed

Today when I using pytube python package to download the YouTube video, I got the following error message:

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:847)>

I never got this error in the past, and it seems not a problem with YouTube page version. After I check on the Internet, I find maybe the version of urllib module is different from the previous version.

Simply put, the error seems an error in verifying the SSL certificate. Maybe the target website uses a certificate signed by itself, I’m not really sure.

Sample Program With Error

Let’s take an example:

import urllib.request
r = urllib.request.urlopen('https://google.com')

Output:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 407, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 817, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1077, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 689, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:847)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:847)>

Solution

However, if we choose not to authenticate SSL certificate, we can make urllib work.

# coding: utf-8
import urllib.request
import ssl


def main():
    ssl._create_default_https_context = ssl._create_unverified_context
    r = urllib.request.urlopen('https://google.com')
    print(r.status)
    print(r)


if __name__ == '__main__':
    main()

Output:

200
<http.client.HTTPResponse object at 0x7fc1af051ef0>

In addition to this method, if you just want a specific connection not to be verified, and the rest of the connection is to verify the SSL certificate, you can write:

# coding: utf-8
import urllib.request
import ssl


def main():
    context = ssl._create_unverified_context()
    r = urllib.request.urlopen('https://google.com', context=context)
    print(r.status)
    print(r)


if __name__ == '__main__':
    main()

Output:

200
<http.client.HTTPResponse object at 0x7fc1af051ef0>

References

• https://stackoverflow.com/questions/27835619/urllib-and-ssl-certificate-verify-failed-error
• https://moreless.medium.com/how-to-fix-python-ssl-certificate-verify-failed-97772d9dd14c

Read More

• [Python] Using package pytube to download YouTube videos
• [Python] Using “GoogleNews” package to get the Google News