FoundationDB/fdb-document-layer

Support wire protocol 4 (Server v3.2)

apkar opened this issue · 9 comments

apkar commented

Wire protocol 4 adds

  • find command
    • It is likely this is just same as OP_QUERY. If so, it's a relatively easy task.
  • getMore command
    • It is likely this is just same as OP_GETMORE. If so, it's a relatively easy task as we already support OP_GETMORE.
  • OP_COMMAND
    • This is an internal message sent between Mongo servers. mongos doesn't implement this as well. The client never sends this message. We don't have to implement this.
apkar commented

This is important to be able to use mongo-go-driver with Document Layer. It only supports server versions >= 3.2.

bartle-stripe commented

There's also the killCursors command to replace OP_KILL_CURSORS. The other thing that 3.2 adds is support for readConcern on query commands (find, count, etc...), though I imagine the document layer can happily ignore that particular option.

One important difference with OP_QUERY and find is when cursors are automatically closed. OP_QURY doesn't support expressing a limit directly (numberToReturn indicates how many documents to return in the initial batch, but doesn't prevent drivers from requesting additional documents in OP_GET_MORE). On the other hand, find has a limit argument that represents the hard limit on the cursor (i.e. the cursor itself will not return more than limit results), and in practice, MongoDB will automatically close cursors when that limit is reached (this saves a roundtrip of having to call killCursors in many cases).

  • 👍1
apkar commented

Mongo Swift driver, MongoKitten, seems to have the same issue. The latest release, 5.0, has removed support for MongoDB 3.0.

likeabbas commented

Any updates on this? I'd be happy to help work on this, just having trouble parsing through the cpp codebase as it's been a few years since I've worked with the language

apkar commented

That would be great @likeabbas. We are planning on finishing this before 1.7.0, end of this month. I am just about to start on this issue. If you want to contribute we can find pieces of this issue you can take on. There are so many other issues you can contribute as well.

To start, you could take on couple of simple ones, like #84 and #92. That will give you introduction into code and also gives you chance to setup your development environment.

I am thinking of adding some design documentation. I can use this as an opportunity as well.

dongxinEric commented

Just wanna clarify that, according to my local tests, mongo-go-driver:v1.0.2 does work with both server 3.0 and 3.2.

dongxinEric commented

I tested the latest MongoKitten lib by running some basic operations and it just worked fine. Will open PR today or tomorrow.

  • 👍1
oemdaro commented

Any updates on this? The support for server v3.2 will be release soon?

Thank for hardworking on this. Love this project.

  • 👍1
akira-kurogane commented

As of v5.0 MongoDB Community deprecates the legacy OP_QUERY, OP_UPDATE, etc. write protocol message formats. https://jira.mongodb.org/browse/SERVER-55788

The OP_COMMAND + OP_COMMAND_REPLY ones were an inbetween attempt to have a generic command message type. They were never used by clients, instead OP_MSG was the next began support with v3.6. Related blog post: https://emptysqua.re/blog/driver-features-for-mongodb-3-6/.

To put it another way it was Jan 2020 when v3.4, the last OP_MSG-incompatible server version, was EOL'ed.

So although MongoDB drivers will probably continue to support OP_QUERY etc. for older mongodb server versions for a while yet a MongoDB compatible wire protocol should just move forward to OP_MSG. Offering backward compatibility has been N/A for a year or so already.

  • 👍2