Achieve compatibility with WeKan
Opened this issue · 11 comments
What should be done?
They use Meteor which often tails the oplog for performance benefits but tailing the oplog is an optional setting for WeKan defined here. It uses Meteor's poll-and-diff method by default which just uses find under-the-hood.
They also use GridFS to store avatars and attachments.
https://github.com/wekan/wekan/tree/master/models
We just released a new version with OpLog tailing support, which was a major blocker for supporting Meteor apps: https://github.com/FerretDB/FerretDB/releases/tag/v1.18.0
When I try FerretDB SQLite, I get this exception:
Exception from sub board id wGvX5gYqr2vaFFisK Error: Exception while polling query {"collectionName":"boards","selector":{"_id":"gzST3j8ZiW78FndH7","archived":false,"$or":[{"permission":"public"},{"members":{"$elemMatch":{"userId":"iZYN9KrYfJNG3i9sQ","isActive":true}}},{"orgs.orgId":{"$in":[]}},{"teams.teamId":{"$in":[]}}]},"options":{"limit":1,"sort":{"sort":1}}}: unknown operator: userId
at PollingObserveDriver._pollMongo (packages/mongo/polling_observe_driver.js:165:11)
at Object.task (packages/mongo/polling_observe_driver.js:93:12)
at Meteor._SynchronousQueue.SQp._run (packages/meteor.js:1026:16)
at packages/meteor.js:1003:12
At browser right click console:
Error invoking SubsManager.subscribe("board", "gzST3j8ZiW78FndH7", false): Internal server error ab3fce4c494d44ba7eec83cd68639d536a5c1f47.js:746:2638
At docker-compose.yml of https://github.com/wekan/wekan/blob/main/docker-compose.yml#L91-L110 I have changed to FerretDB SQLite like this, then docker compose up -d and registered at https://localhost/sign-up and logged in at https://localhost/sign-in
services:
wekandb:
image: ghcr.io/ferretdb/all-in-one
networks:
- wekan-tier
restart: on-failure
expose:
- 27017
environment:
- FERRETDB_HANDLER=sqlite
- FERRETDB_SQLITE_URL=file:/data/
- DOCKER_DEFAULT_PLATFORM=linux/x86_64/v8
volumes:
- ./data:/data/
- /etc/localtime:/etc/localtime:ro
- wekan-db:/data/db
- wekan-db-dump:/dump
#- /etc/timezone:/etc/timezone:ro # Do not use https://github.com/wekan/wekan/issues/5123
Hmm, I think I did not yet add those OpLog settings from https://github.com/FerretDB/FerretDB/releases/tag/v1.18.0
I'll try
In Progress. Just guessing.
I don't have any of these working yet, I'm just guessing how these could work. I don't have any working examples yet.
How to set OpLog URL setting?
MONGO_OPLOG_URL , with MongoDB, was previously made this way https://github.com/wekan/wekan/blob/main/docker-compose.yml#L218 . Does Meteor also need MONGO_OPLOG_URL ?
For FerretDB, it looks like FERRETDB_REPL_SET_NAME should be something like this:
- MONGO_URL=mongodb://wekandb:27017/wekan
- MONGO_OPLOG_URL=mongodb://wekandb:27017/local?authSource=admin&replicaSet=oplog.rs
services:
wekandb:
image: ghcr.io/ferretdb/all-in-one
networks:
- wekan-tier
restart: on-failure
#ports:
# - 27017:27017
expose:
- 27017
environment:
- FERRETDB_HANDLER=sqlite
- FERRETDB_SQLITE_URL=file:/data/
- FERRETDB_REPL_SET_NAME=oplog.rs
- DOCKER_DEFAULT_PLATFORM=linux/x86_64/v8
volumes:
- ./data:/data/
- /etc/localtime:/etc/localtime:ro
- wekan-db:/data/db
- wekan-db-dump:/dump
#- /etc/timezone:/etc/timezone:ro # Do not use https://github.com/wekan/wekan/issues/5123
How to create oplog.rs capped collection at Snap?
https://github.com/wekan/wekan/blob/main/snap-src/bin/mongodb-control#L148
For creating oplog.rs capped collection, I guess it would be something like this:
mongosh wekan --eval 'db.createCollection("oplog.rs", { capped : true, size : 5242880, max : 5000 } );' $BIND_OPTIONS
Previous MongoDB OpLog at ToroDB/PostgreSQL
Previously, with ToroDB to PostgreSQL https://github.com/wekan/wekan/tree/main/torodb-postgresql , replicaset was previosly made this way https://github.com/wekan/wekan/blob/main/torodb-postgresql/docker-compose.yml#L129-L142
mongodb:
image: mongo:3.2
networks:
- wekan-tier
ports:
- "27017:27017"
entrypoint:
- /bin/bash
- "-c"
- mongo --nodb --eval '
var db;
while (!db) {
try {
db = new Mongo("mongodb:27017").getDB("local");
} catch(ex) {}
sleep(3000);
};
rs.initiate({_id:"rs1",members:[{_id:0,host:"mongodb:27017"}]});
' 1>/dev/null 2>&1 &
mongod --replSet rs1
But I don't know does FerretDB all-in-one container have mongosh command? MongoDB 3.x MongoShell command mongo is at MongoDB 6.x commmand mongosh.
Maybe creating oplog.rs with Meteor?
I don't know, would it be possible to create replicaset using Meteor commands? https://docs.meteor.com/api/collections
Maybe creating oplog.rs at models?
In theory, like at wekan/models/ferretdb-oplog.js, maybe something like this, not tested:
if (Meteor.isServer) {
Meteor.startup(() => {
// Attachments.collection.createIndex({ 'meta.cardId': 1 });
"oplog.rs".collection.createCollection({ capped : true, size : 5242880, max : 5000 });
});
}
Does Meteor also need MONGO_OPLOG_URL ?
I think so.
FERRETDB_REPL_SET_NAME=oplog.rs
That should be just a name like rs0, not an OpLog collection name (that is fixed). The same name should be used in MongoDB URI's replicaSet parameter.
But I don't know does FerretDB all-in-one container have mongosh command?
It does. But rs.initiate will not work. Instead, just create a capped collection with db.createCollection.
Yes, Meteor can use the environment variable MONGO_OPLOG_URL https://docs.meteor.com/environment-variables#MONGO-OPLOG-URL
MONGO_OPLOG_URL=mongodb://localhost:27017/local?replSet=rs01
Excited to see progress on FerretDB and Meteor!
Is there any working code example of Meteor working with FerretDB/SQLite ?
For example, can you make modifications to Dockerfile and docker-compose.yml at https://github.com/wekan/wekan and create GitHub Gist or repo, where WeKan would work with FerretDB/SQLite ? And if possible, also example for FerretDB/PostgreSQL ?
Is there any working code example of Meteor working with FerretDB/SQLite ?
For example, can you make modifications to Dockerfile and docker-compose.yml at https://github.com/wekan/wekan and create GitHub Gist or repo, where WeKan would work with FerretDB/SQLite ? And if possible, also example for FerretDB/PostgreSQL ?
Absolutely! There's a PostgreSQL example here https://github.com/wreiske/meteor-ferretdb I made around 2 years ago. It looks like FerretDB forked it and updated it a bit https://github.com/FerretDB/meteor-ferretdb
Maybe with small tweaks it can work with SQLite.