Adding a few thousands of geometries causes memory corruption errors

[duvifn]

Hello again,
I'm trying to add a few thousands of records to my db (in browser) and getting some errors:

const db = <...>;
const srid = 4326;
 
let script = `
BEGIN TRANSACTION;
SELECT InitSpatialMetaData();
CREATE TABLE ${tableName} (
    id TEXT NOT NULL
      PRIMARY KEY);
SELECT AddGeometryColumn('${tableName}', '${GEOM_FIELD_NAME}', ${srid}, 'GEOMETRY', 'XY');
`;
const geoJson = '{"type":"Polygon","coordinates":[[[35.172522345781324,31.807637007387367],[35.1730225777626,31.807379406789376],[35.17296088695526,31.807292779878278],[35.17246065497398,31.807550380717725],[35.172522345781324,31.807637007387367]]]}';
let geometryAddition = `SetSRID(GeomFromGeoJSON('${geoJson}'), 4326)`;
for (let i = 0; i < 5000; ++i) {
   script += `INSERT OR REPLACE INTO ${tableName} VALUES ('a${i}',${geometryAddition});`;
}
script += "COMMIT;"
await db.read(script);

With 5000 rows I got:

table index is out of bounds

And with 10000 rows:

memory access out of bounds

This seems related to this chromium issue, but I have an up-to-date version (Version 103.0.5060.134).

Can you confirm this bug?
If it's only on my desktop machine I'm OK with this but if this is a common issue perhaps this chromium bug should be reopened.

[duvifn]

Happens also in Firefox

[jvail]

That's a bug! Thanks, I didn't test such large scripts. The solution in is to put the script on the heap otherwise it is too large for emscripten's stack memory.

[duvifn]

I hope I correctly understood you.
I have tested this also with several insert transactions (not a one giant script, but few ones) and also got similar results.

So not sure it's a script size issue.

[duvifn]

Sorry.
Dividing the above script to several transactions (limiting script size, 2000 rows per transaction) does solves this issue.

Thank you very much, again!

[jvail]

Ok! I have just fixed the script bug and added a test here: https://github.com/jvail/spl.js/blob/main/test/browser.js#L242-L272

At least in my test both seem to work: large scripts and a large exec with 10000 rows.

[duvifn]

Tanks a lot. Do you know when you publish this to npm?

[jvail]

It's on npm already.

[duvifn]

Ah OK. Thanks again!