microsoft/DacFx

better handling of deployments to database with large volumes of stored procedures

Opened this issue · 0 comments

Is your feature request related to a problem? Please describe.
We have a legacy database that has between 50k-100k stored procedures in it, sqlpackage out of a box is unusable currently with it. those procedures are autogenerated, a software that we use requires them to run. In all of those stored procedures some of them are written by developers.

Describe the solution you'd like
We would like to be able to better filter stored procedures out of our deployment, handle only those in dacpac that are not autogenerated. we wound like to get results from sqlpackage in a much shorter time counted in minutes not hours.

Describe alternatives you've considered
Currently we are using DeploymentPlanModifier to filter out those autogenerated but there are some issues with it, we have to turn off DropObjectsNotInSource flag in our deployment profile, because if its turned on, then those excluded procedures are a part of analysis somehow and script generation it that case takes don't remember exactly but its like 4-7h

Additional context
From my investigations, I have profiled, and debugged sqlpackage, it looks like part of those excluded procedures or permission objects related to them are present in a list in memory (I thing there was over 600k object because in this list one procedure is more than one object) and since there is one for loop inside the other with such huge list the result crazy amount of iterations, during those iterations nothing is emitted in logs so from logs perspective looks like sqlpackage has hanged, but it is just calculating a lot, and with enough patience and free memory it will create a script. those autogenerated procedures are not present in this generated script.