Feature Request: Ability to only include tables that match a given pattern or name
Arnagos opened this issue · 1 comments
It would be great if we could tell the reader to only include tables that match a given pattern or name. This would improve the loading overhead substantially.
Loading each required table individually does work but has worse performance than just loading all tables.
MSSQL
Method | Mean | Error | StdDev | Median | Gen0 | Gen1 | Allocated |
---|---|---|---|---|---|---|---|
ReadAll | 765.5 ms | 18.91 ms | 54.85 ms | 750.3 ms | 2000.0000 | 1000.0000 | 14.47 MB |
AllTables | 456.9 ms | 12.77 ms | 37.45 ms | 457.6 ms | 2000.0000 | 1000.0000 | 13.98 MB |
ReadTablesIndividually | 5,840.3 ms | 57.10 ms | 50.61 ms | 5,836.7 ms | 8000.0000 | 1000.0000 | 51.11 MB |
PostgreSQL
Method | Mean | Error | StdDev | Gen0 | Gen1 | Allocated |
---|---|---|---|---|---|---|
ReadAll | 2.835 s | 0.0536 s | 0.0638 s | 1000.0000 | - | 7.03 MB |
AllTables | 2.653 s | 0.0485 s | 0.0454 s | - | - | 4.48 MB |
ReadTablesIndividually | 3.478 s | 0.0527 s | 0.0467 s | 6000.0000 | 1000.0000 | 39.33 MB |
Use Case:
We have a database with some 300 tables that belong to different projects (bad design, but I cannot change that) and want to load only a subset of these tables. Luckily all of our tables match either a given pattern (prefixed with some name like ProjectName__
) or are known by name.
I guess we could do this with a table like - something like var tables = reader.TablesLike("Prefix_%");
This would be a huge change- new overloads for every method (indexes, constraints, columns...) and implement the supporting sql in each database.