memory leak with each SSL web request #380
Description
I'm attempting to diagnose a memory leak with each treq.get call on a HTTPS website leaking small amounts of memory. This is my simple reproducer.
import treq
from twisted.internet.defer import inlineCallbacks
from twisted.internet import reactor
@inlineCallbacks
def main():
for i in range(1000):
# author owns this website, so hitting it is fine :)
_req = yield treq.get('https://mesonet.agron.iastate.edu/robots.txt')
reactor.stop()
reactor.callLater(0, main)
reactor.run()running this with memray seems to indicate a SSL verify_paths leak in OpenSSL/SSL.py set_result = _lib.SSL_CTX_set_default_verify_paths(self._context)
def set_default_verify_paths(self):
"""
Specify that the platform provided CA certificates are to be used for
verification purposes. This method has some caveats related to the
binary wheels that cryptography (pyOpenSSL's primary dependency) ships:
* macOS will only load certificates using this method if the user has
the ``[email protected]`` `Homebrew <https://brew.sh>`_ formula installed
in the default location.
* Windows will not work.
* manylinux cryptography wheels will work on most common Linux
distributions in pyOpenSSL 17.1.0 and above. pyOpenSSL detects the
manylinux wheel and attempts to load roots via a fallback path.
:return: None
"""
# SSL_CTX_set_default_verify_paths will attempt to load certs from
# both a cafile and capath that are set at compile time. However,
# it will first check environment variables and, if present, load
# those paths instead
set_result = _lib.SSL_CTX_set_default_verify_paths(self._context)Is there a means to workaround this within treq? Perhaps create some context that can be reused throughout the lifetime of the running app?