Address Orthanc@1.9.2 DB plugin refactoring

Question

Address Orthanc@1.9.2 DB plugin refactoring

Opened this issue 3 years ago · 10 comments

Hello everyone.
Well as you know orthanc has just released somewhat major db refactoring release,:

Starting with Orthanc 1.9.2 and PostgreSQL plugins 4.0, Orthanc will be compatible with multiple writers/readers sharing the same database. One Orthanc server will also manage several connections to PostgreSQL, in order to improve performance by adding concurrency.
Orthanc was originally designed as a mini-DICOM server in 1-to-1 relation with a SQLite database. Supporting multiple writers/readers necessitated a full refactoring of the database engine, so as to be able to replay transactions in the case of collisions. This refactoring should enable hospital-wide VNA scenarios.

So could we discuss how we can move the plugin forward with this release.
Also I still have a pending pull request I would be glad if could merge it (so every one benefit from the performance boost)

Answer 1 · 2021-09-14T09:51:00.000Z

Hello there.
I update the plugin file structure (inspired by orthanc-database one), and now the plugin support the new orthanc@1.9.2+ SDK.
This is Still a WIP (need to update the README section for the build) but you can give it a try.
Here is my fork repo.

Answer 2 · 2021-11-20T04:17:31.000Z

HI @Betalos ,
What version of mongocxx are you using ? I am using version 3.4.2 seems like your code is not compatible with that version at these lines in MongoDB/Plugins/MongoDBIndex.cpp

971 if (hasServer) {
972 query.reset(make_document(kvp("property", property), kvp("server", serverIdentifier)));
973 pDocument.reset(
974 make_document(kvp("property", property), kvp("value", utf8), kvp("server", serverIdentifier))
975 );
976 }

Answer 3 · 2021-11-20T09:45:18.000Z

Hello @phtran-dev ,

Well you should use the latest version of mongoc and mongocxx. I didn't had time to finish the building process, but basically you need to build and install mongoc and mongocxx.

Answer 4 · 2021-11-22T04:05:41.000Z

Thanks @Betalos ,

I manged to compiled the plugin. However when I test those dicoms (find in the attachment), I see that the plugin generates 3 studies instead of one (which SQLite
test_dicom.zip
Plugin generates). Can you please help to verify and test ?

Answer 5 · 2021-11-22T10:46:10.000Z

Hello @phtran-dev.

That's a weird issue, could you tell me how you did the upload of these instances into the pacs.
Also is the studies are identical (check the orthanc's rest api for the studyUID if it's the same)

Answer 6 · 2021-11-22T10:57:51.000Z

Hi @Betalos ,
I uploaded dicom instances via orthanc GUI. Yes the studies are identical since studyInstanceUID, PID of those dicom instances are the same.

Answer 7 · 2021-11-22T11:27:03.000Z

It's weird that sometimes it creates 3, sometimes it creates only 1. Not sure if there is missing database locking in the plugin'code ?

Answer 8 · 2021-11-22T12:34:11.000Z

hello @phtran-dev.

Well this issue is with http client (posting the dcm vi http), if you used dcmsend (which emulate dicom c-store scu) it will work.
I didn't have the time to test it with postgress plugin to see if it related to orthanc or the plugin.
well the plugin have db locking, here the issue has to do with http multi threads somehow.

Answer 9 · 2021-11-26T11:02:57.000Z

Thanks @Betalos ,
From your experience, do you think this mongo plugin can handle hundreds of millions of dicom instance (or even billion of instances)?

Answer 10 · 2022-03-02T11:59:33.000Z

hello @phtran-dev, sorry for taking too much time to replay.
To answer your question, we have some systems in production with 2to of data, and the system is stable and the requests not very slow.
but this version of the plugin still have some issues with locking mechanism. when we deployed it to prod, it had some performance degradation. So I need to look it up.