HicServices/RDMP

Poor performance

jas88 opened this issue · 2 comments

jas88 commented

Very slow reading data from SQL compared to SSMS - possibly DataTable/pipelining bottleneck?

Investigate batching DataTable writes, clone instead of repeating chunk metadata, refactor as IEnumerable instead of repeatedly calling DataTable GetChunk.

tznind commented

Also check which database you are connecting to. If the query has any collation in it and RDMP is connecting to master or another db with a different default collation this can be a problem.

  • 👍1
jas88 commented

Same SQL executed via SSMS runs in a tenth of the time, so not an SQL level issue :(
Synchronous logging looks like at least part of the culprit - single-row inserts on ProgressLog are sometimes timing out since it's over 100m rows. Will investigate further and see what can be done there.