Is there a way to execute the query but not fetch the results?
Closed this issue · 1 comments
Hello,
We are trying to optimize complex calculations where we need to first run one query and then use the RESULT_SCAN
to perform additional work on top of the previous results.
However, I haven't found any way to execute the query and get queryId without fetching the results. The first query we do produces a huge result and the process just crashes hitting the RAM limit. I could increase the RAM limit but that seems wrong since I don't need the result in the first place. 🤔
Here's what I tried so far.
This way I get the statement ID after the query is executed successfully and I can use it right away but the results are still fetched and memory usage is huge.
const connection = await this.getConnection();
const statement = connection.execute({
sqlText: preparedSql,
binds: parameters,
complete: (error, statement) => {
const statementId = statement.getStatementId();
},
});
This way is even worse. Not only the rows are still fetched but also the statementId
appears before the query fully executed so there's still some gap before I can actually use it in RESULT_SCAN
.
const connection = await this.getConnection();
const statement = connection.execute({
sqlText: preparedSql,
binds: parameters,
});
// Here I'm basically checking every second if statementId exists or not.
let statementId;
while (!statementId) {
statementId = statement.getStatementId();
await new Promise((resolve) => setTimeout(resolve, 1000));
}
I feel like there must be a way to execute the query without fetching results (especially since it's a use-case mentioned in the documentation: https://docs.snowflake.com/en/user-guide/querying-persisted-results#post-processing-query-results) but I can't find it.
Sorry, seems like I created this in the wrong repo. I wanted to report to the official SDK. 🤦♂️