Supper slow to run even on local host

[zdravko123]

Hi,

I love this software its needed so so much! thanks for writing it.

It however is super slow to run, is there any ways to speed it up?

Cheers

[nisbus]

Thanks for testing it!

Which part is slow for you?
Deleting the documents shouldn't take too long but compacting the database and the views could be the culprit.
Of course if there are a lot of deleted documents this might take very long.
The only advice I could give would be to run it frequently so that there are fewer changes on each run.

Thanks

[zdravko123]

Hey nisbus,

It’s just the running of it, I think due to the fact you do it 1 document at a time....... Compacting the db and views i also do and they are fast always...

so the good news is - i don’t know any Python and I’m too old to learn it, im a decent C# dev though... so your code, worked but super slow.. but that was enough to inspire me to write my own in C# land...

After trying a few times I have got it to how i want, your script would haev taken about 2 days to run on a database with 100,000 deleted documents, mine that i did now takes just 20 seconds or so......... This is due to me batching it, originally my script took 2 hours without batches.

What i do is this,

1. query the changes feed.
2. for each deleted document, i batch up to 100 Documents, this seems to the prefect size , 200 , 500 or 1000 doesn’t work well and causes error due to the payload size i think.
3. i then make the call to the purge, it accepts arrays of ids not just id.
4. it takes about 2 minutes to run then I move on to the next database.

I ran it last night and managed in 5 hours to purge about 1000 databases out of 3000 im going to do the rest tonight, once im happy with the script i will share it online.

Thanks for this!

[nisbus]

Hey,
Great to hear this inspired you and thanks for sharing your work!
I'll see about changing this in my script also (when I have time) and if you want you can try it out and let me know.

Thanks