G-Research/spark-dgraph-connector

Make gRCP max message size configurable

Opened this issue · 0 comments

Currently, gRPC maximum message size is hard set to 24 MiB at

def toChannel(target: Target): ManagedChannel = NettyChannelBuilder.forTarget(target.toString).usePlaintext().maxInboundMessageSize(24 * 1024 * 1024).build()

Make this number configurable via spark.read.option.

Further, distinguish between the channel used in SchemaProvider and DgraphExecutor, and provide individual settings (a general max size and two specific sizes).