hifly81/1brc_streaming

Producer: threads running < number of csv files

Closed this issue · 0 comments

There are 10 csv files in folder but Producer is using only 4 threads; I see that this is hard-coded in producer java file

#! /usr/bin/java --class-path ./kafka_2.13-3.6.1/libs/* --source 17 -XX:ActiveProcessorCount=4

is there a way to determine that value from the hosting machine?

Run producer...
Producing...

Number of files:10

Thread ID: 24 Processing file: data_1.csv

Thread ID: 25 Processing file: data_10.csv

Thread ID: 1 Processing file: data_6.csv

Thread ID: 26 Processing file: data_3.csv