Use without having to use verify=False

[Zorkai]

Hello, I wanted to know if it was possible to use this project without having to use verify=False.
I heard this was possible by installing a certificate.
Not using verify=False while doing requests will make my program crash because of SSL errors

[synchronizing]

You are indeed right: verify=False needs to be set to essentially tell requests to trust the bogus certificate that mitm generates to create a TLS/SSL connection with the client.

To give a little bit of context, computers are shipped with certificates (public keys) that are signed by certificate authorities (CA). These CAs hold huge events where they generate the private key and certificates, and make a huge effort to make sure the private keys are secure. If a private key leaks people can then intercept SSL/TLS traffic at will.

Therefore, to not use verify= you need to install the certificate on your computer. To do this, there is three specific settings within mitm.Config to allow you to generate the certificate outside of your data folder:

from mitm import Config
import pathlib

dt = pathlib.Path.home() / "Desktop"
config = Config(rsa_key=dt / "mitm.key", rsa_cert=dt / "mitm.crt", rsa_generate=True)

The above code should generate two files on your desktop, mitm.key and mitm.crt. The .key file is the private key, and the .crt file is the certificate. The certificate is what you want to install. Depending on platform (Win, Mac, Linux) there are different steps for installing it - whatever the case, once you do install it (and make sure to set trust settings), you will no longer need to set request's verify= flag. Do remember to load the certificates the next time you boot-up mitm, and set rsa_generate=False.

Note, however, that installing certificates like these open your computer to future dangers as anyone with the .key file can then use that to decipher your internet traffic. Another (less invasive) way to not have to set verify= in requests is by using the REQUESTS_CA_BUNDLE env variable (see here). Simply doing:

export REQUESTS_CA_BUNDLE=/path/to/your/certificate.crt
python script.py

Should be enough. Hopefully that helps!

[synchronizing]

Oh, also - you can generate your own custom certificates if you would like. Take a look at:

mitm/mitm/crypto.py

Lines 12 to 21 in b53efea

	def new_RSA(bits: int = 1024) -> crypto.PKey:
	"""Generates an RSA pair.
	Note:
	This function is intended to be utilized with :py:func:`new_X509`. See function
	:py:func:`new_pair` to understand how to generate a valid RSA and X509 pair for
	SSL/TLS use.
	Args:
	bits: Size of the RSA key. Defaults to 1024.

mitm/mitm/crypto.py

Lines 29 to 58 in b53efea

	def new_X509(
	country_name: str = "US",
	state_or_province_name: str = "New York",
	locality: str = "New York",
	organization_name: str = "mitm",
	organization_unit_name: str = "mitm",
	common_name: str = socket.gethostname(),
	serial_number: int = random.getrandbits(1024),
	time_not_before: int = 0, # 0 means now.
	time_not_after: int = 5 * (365 * 24 * 60 * 60), # 5 year.
	) -> crypto.X509:
	"""
	Generates a non-signed X509 certificate.
	Note:
	This function is intended to be utilized with :py:func:`new_RSA`. See function
	:py:func:`new_pair` to understand how to generate a valid RSA and X509 pair for
	SSL/TLS use.
	Args:
	country_name: Country name code. Defaults to ``US``.
	state_or_province_name: State or province name. Defaults to ``New York``.
	locality: Locality name. Can be any. Defaults to ``New York``.
	organization_name: Name of the org generating the cert. Defaults to ``mitm``.
	organization_unit_name: Name of the subunit of the org. Defaults to ``mitm``.
	common_name: Server name protected by the SSL cert. Defaults to hostname.
	serial_number: A unique serial number. Any number between 0 and 2^159-1. Defaults to random number.
	time_not_before: Time since cert is valid. 0 means now. Defaults to ``0``.
	time_not_after: Time when cert is no longer valid. Defaults to 5 years.
	"""

mitm/mitm/crypto.py

Lines 74 to 81 in b53efea

	def new_pair() -> Tuple[bytes, bytes]:
	"""
	Generates an RSA and self-signed X509 certificate for use with TLS/SSL using the
	default mitm settings.
	Returns:
	tuple: Key and certificate bytes ready to be saved.
	"""

new_pair is what mitm uses to generate the cert. You can generate your own, and then load it into mitm.

[synchronizing]

Let me know if anything else. Closing this for now.

[Zorkai]

This doesn't seem to work on windows. After installing the certificate, I'm getting [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate

Code :

import requests
proxies = {"http": "http://127.0.0.1:8888", "https": "https://127.0.0.1:8888"}
requests.get("https://google.com", proxies=proxies)

[synchronizing]

Reopening issue to continue discussion.

On Windows you'll likely have to install the certificate, and set trust settings. The Google query should be "windows trust self-signed certificate" - perhaps this might work? I'm not on Windows myself, so I can't test it out.

[Zorkai]

That's how I initially did it. It doesn't work for some reason.

[synchronizing]

It should work in theory. I don't know if there is anything else I can do to help 😕

[Zorkai]

And this works on Mac / Linux?

[synchronizing]

In theory it should work in all systems.

[areebbeigh]

In theory it should work in all systems.

Hey. I'm also trying this. Going through the code, it seems like mitm creates only one certificate. What should be happening is, mitm should create a separate certificate for every domain using the CA certificate it generates right now.

Right now even by using the generated certificate for verification, the request will fail:

➜  ~ curl -x 127.0.0.1:8888 https://google.com --cacert ~/Library/Application\ Support/mitm/mitm.crt
curl: (60) SSL: certificate subject name 'MBP-H744TTW0YW' does not match target host name 'google.com'

Note how it expects the certificate sent by the proxy subject to be google.com but it isn't. The proxy is just sending the CA certificate instead of a google.com one.

I might be wrong about something here. Let me know what you think.

[synchronizing]

In theory it should work in all systems.

Hey. I'm also trying this. Going through the code, it seems like mitm creates only one certificate. What should be happening is, mitm should create a separate certificate for every domain using the CA certificate it generates right now.

Right now even by using the generated certificate for verification, the request will fail:

➜  ~ curl -x 127.0.0.1:8888 https://google.com --cacert ~/Library/Application\ Support/mitm/mitm.crt
curl: (60) SSL: certificate subject name 'MBP-H744TTW0YW' does not match target host name 'google.com'

Note how it expects the certificate sent by the proxy subject to be google.com but it isn't. The proxy is just sending the CA certificate instead of a google.com one.

I might be wrong about something here. Let me know what you think.

You are indeed right. I made a note of it here and will be pushing a patch up soon. 😄