Adding a few thousands of geometries causes memory corruption errors
duvifn opened this issue · 8 comments
Hello again,
I'm trying to add a few thousands of records to my db (in browser) and getting some errors:
const db = <...>;
const srid = 4326;
let script = `
BEGIN TRANSACTION;
SELECT InitSpatialMetaData();
CREATE TABLE ${tableName} (
id TEXT NOT NULL
PRIMARY KEY);
SELECT AddGeometryColumn('${tableName}', '${GEOM_FIELD_NAME}', ${srid}, 'GEOMETRY', 'XY');
`;
const geoJson = '{"type":"Polygon","coordinates":[[[35.172522345781324,31.807637007387367],[35.1730225777626,31.807379406789376],[35.17296088695526,31.807292779878278],[35.17246065497398,31.807550380717725],[35.172522345781324,31.807637007387367]]]}';
let geometryAddition = `SetSRID(GeomFromGeoJSON('${geoJson}'), 4326)`;
for (let i = 0; i < 5000; ++i) {
script += `INSERT OR REPLACE INTO ${tableName} VALUES ('a${i}',${geometryAddition});`;
}
script += "COMMIT;"
await db.read(script);
With 5000 rows I got:
table index is out of bounds
And with 10000 rows:
memory access out of bounds
This seems related to this chromium issue, but I have an up-to-date version (Version 103.0.5060.134).
Can you confirm this bug?
If it's only on my desktop machine I'm OK with this but if this is a common issue perhaps this chromium bug should be reopened.
Happens also in Firefox
That's a bug! Thanks, I didn't test such large scripts. The solution in is to put the script on the heap otherwise it is too large for emscripten's stack memory.
I hope I correctly understood you.
I have tested this also with several insert transactions (not a one giant script, but few ones) and also got similar results.
So not sure it's a script size issue.
Sorry.
Dividing the above script to several transactions (limiting script size, 2000 rows per transaction) does solves this issue.
Thank you very much, again!
Ok! I have just fixed the script bug and added a test here: https://github.com/jvail/spl.js/blob/main/test/browser.js#L242-L272
At least in my test both seem to work: large scripts and a large exec with 10000 rows.
Tanks a lot. Do you know when you publish this to npm?
It's on npm already.
Ah OK. Thanks again!