Virtual Schema uses too many workers for small amount of files
Closed this issue · 0 comments
jakobbraun commented
If only a few files e.g: 2 are loaded the virtual schema still uses all available workers. Starting them is a huge overhead and leads to a bad performance.
Workarround
set the property MAX_PARALLEL_UDFS = 1
.
This wil force the virtual-schema to only use one worker.
Example:
CREATE VIRTUAL SCHEMA FILES_VS_TEST USING ADAPTER.S3_FILES_ADAPTER WITH
CONNECTION_NAME = 'S3_CONNECTION'
MAPPING = '/bfsdefault/default/path/to/mappings/in/bucketfs'
MAX_PARALLEL_UDFS = 1;