Error: field selection and slice cannot be used with count - permissions too strict for count?

[benmurrell]

To begin, I saw your response on the mongoose issue here: Automattic/mongoose#1772

However, I don't understand how using count on a query with limited or sliced fields is nonsensical or invalid in a way that justifies throwing an Error and refusing to continue. Regardless of the fields selected (or how the selected fields are sliced), the count should remain the same.

mongo seems to act how I'd expect:

> db.events.find().count()
454

> db.events.find( {}, { name: true } ).count()
454

> db.events.find( {}, { name: true, presentations: { $slice: 1 } } ).count()
454

My use case where this matters is that I have created an API that takes a Query. I take that query and turn it into a constructor, then use the constructor to build a query for the count and a query for a limited range of data (this is ultimately for a pagination middleware):

var queryConstructor = aQuery.find( req.query.criteria ).toConstructor();

var countQuery = queryConstructor().count();

var dataQuery = queryConstructor().sort( req.query.sort );
dataQuery = dataQuery.limit( req.query.limit );
dataQuery = dataQuery.skip( ( req.query.page - 1 ) * req.query.limit );

Everything is fine until I receive a query that selects fields or slices. I've come up with a few workarounds:

• Accept a separate, nearly identical Query that is used for the count - a bit ugly and repetitive for calls to the API
• Clear out the query's internal _fields value before calling count on the query - seems unnecessary, and it touches internal state directly, but there isn't any API for this currently that I saw
• Create a new query using the query's internal _conditions value - again, with the internal state manipulation

I'm also having trouble finding documentation (mongodb, mongoose, or otherwise) indicating that some of the other operations that are disallowed in permissions.js for the count operation actually need to be disallowed. In particular, select, slice, sort, comment, and hint all seem fine for use with count. I appreciate the effort to keep people from shooting themselves in the foot, but the current permissions seem a little too strict.

> db.events.find( {}, { name: true, presentations: { $slice: 1 } } ).comment('hi').sort({ name:1 }).count()
454

If you can point me to any documentation that justifies these particular cases, I'd be happy to review it and submit a PR if the justification has changed since the current permissions were implemented.

[vkarpov15]

Actually, sort() doesn't make sense with count() at all. For one thing, the count command doesn't take a sort parameter or a select parameter, so there's simply no way to translate these fields into a sensible command to the mongodb server. The only other option is to silently ignore the sort() and select() with count(), which IMO is more reasonable. Do you think that's a good option?

[benmurrell]

I agree that using sort on a count query serves no purpose - but as you can see from the examples above, mongo has no qualms with a query that contains a sort and that ultimately ends up being a count query.

Logically, ignoring a sort, select, or slice for a count query makes sense, and I think it's a good way to handle this case. Our math libraries don't balk at us when we continue to multiply a variable equal to 0 by other numbers, even if those multiplications don't do any actual work :)

[aheckmann]

It's programming error. Someone reading your code later will be confused bc it won't work as it reads, then they'll open a ticket and ask why, and we'll have to explain it again.

This module took a hard stance on this after seeing the same "bug" reports repeatedly come up due to this confusion: you could create queries which MongoDB would fail to process. Once this change was put in place, there have been fewer questions,
presumably bc you can't create invalid queries.

[benmurrell]

I'm confused - what would the error be in this case? If I comment out line 66 of permission.js (the check for selected fields), no error occurs and the query executes how I would expect it to.

[aheckmann]

the current goal of this module is to be strict everywhere. trying to identify which combos do or do not cause errors in mongodb creates too tight a coupling between this module and mongodb releases IMO. how do you propose we make this a little easier to work with without tightly coupling to mongodb or causing inconsistent behavior between methods?