nismod/smif

Performance: Trying to read model parameter default from key <*>.csv seems slow

willu47 opened this issue · 1 comments

With the large number of parameters in the digital comms model, reading from multiple columns of a single default.csv file is very slow.

DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv
DEBUG    Trying to read model parameter default from key ../digital_comms/parameters/defaults.csv```

Could cache the csv file or call to store.read_parameter_file using functools.lru_cache