exasol/virtual-schema-common-document-files

Virtual Schema uses too many workers for small amount of files

Closed this issue · 0 comments

If only a few files e.g: 2 are loaded the virtual schema still uses all available workers. Starting them is a huge overhead and leads to a bad performance.

Workarround

set the property MAX_PARALLEL_UDFS = 1.
This wil force the virtual-schema to only use one worker.

Example:

CREATE VIRTUAL SCHEMA FILES_VS_TEST USING ADAPTER.S3_FILES_ADAPTER WITH
    CONNECTION_NAME = 'S3_CONNECTION'
    MAPPING         = '/bfsdefault/default/path/to/mappings/in/bucketfs'
   MAX_PARALLEL_UDFS = 1;