Cluster server client - TLS certificate is read once per-transaction.
thomaschristopherking opened this issue · 0 comments
thomaschristopherking commented
Description
When opening multiple transactions, in parallel, a UNIX system can raise an 'OSError: [Errno 24] Too many open files' error. This is because the TLS certificate file is opened once per-transaction, instead of once per-client.
Additional information
I am foregoing the reproducible steps, and instead going to provide some code changes I think are sufficient.
In typedb.connection.cluster we have the following method:
def _new_channel(self) -> grpc.Channel:
if self._credential.tls_root_ca_path() is not None:
with open(self._credential.tls_root_ca_path(), 'rb') as root_ca:
channel_credentials = grpc.ssl_channel_credentials(root_ca.read())
else:
channel_credentials = grpc.ssl_channel_credentials()
combined_credentials = grpc.composite_channel_credentials(
channel_credentials,
grpc.metadata_call_credentials(_CredentialAuth(credential=self._credential, token_fn=lambda: self._stub.token()))
)
return grpc.secure_channel(self._address, combined_credentials)
The line with open(self._credential.tls_root_ca_path(), 'rb') as root_ca
is causing the issue.
I propose instead, making this the init:
def __init__(self, address: str, credential: TypeDBCredential, parallelisation: int = 2):
super(_ClusterServerClient, self).__init__(address, parallelisation)
self._credential = credential
self._channel, self._stub = self.new_channel_and_stub()
self._databases = _TypeDBDatabaseManagerImpl(self.stub())
if self._credential.tls_root_ca_path() is not None:
with open(self._credential.tls_root_ca_path(), 'rb') as root_ca:
self._root_ca = root_ca.read()
And replacing the _new_channel method with this:
def _new_channel(self) -> grpc.Channel:
if self._root_ca is not None:
with open(self._credential.tls_root_ca_path(), 'rb') as root_ca:
channel_credentials = grpc.ssl_channel_credentials(root_ca.read())
else:
channel_credentials = grpc.ssl_channel_credentials()
combined_credentials = grpc.composite_channel_credentials(
channel_credentials,
grpc.metadata_call_credentials(_CredentialAuth(credential=self._credential, token_fn=lambda: self._stub.token()))
)
return grpc.secure_channel(self._address, combined_credentials)