Negative values in NUMBER_RANGE

[tonypizzicato]

Working with negative numbers is broken in NUMBER_RANGE filters.
$main.filter($time.in([2015-12-17T00:01:00.000Z,2015-12-18T00:01:00.000Z)).and($Latitude.in([-1,12]))).split($Region,SEGMENT,main).apply(count,$main.count()).sort($count,descending).limit(101) gives empty result, while same request with $Latitude.in([0,12]) not. Such requests worked in 0.8.12 version, for example.

[evasomething]

Hi!
Latitude.in([-1,12]) is actually treated as “in the set of [-1, 12](that is, will return false unless the Latitude. is either -1 or 12) It's possible you saw results with [0, 12] because there were matches for 0?

If you want to see if it’s in a range you’ll want to use Latitude.in(-1, 12)

I'm a bit stumped as to why this would have worked in 0.8.12 though..

[tonypizzicato]

Actually i'm working with pivot and i've made a range filter menu where i use NumberRange filter and its fromJS() method

  bounds = { start: searchFrom, end: searchTo, bounds: '[]' };

  var clause = new FilterClause({
    expression: dimension.expression,
    selection:  r(NumberRange.fromJS(bounds))
  });

  return filter.setClause(clause);

and the result request from pivot is

{
  "version":    "0.8.24",
  "dataSource": "static",
  "expression": {
    "op":         "chain",
    "expression": { "op": "literal", "value": [{}], "type": "DATASET" },
    "actions":    [{
      "action":     "apply",
      "expression": {
        "op":         "chain",
        "expression": { "op": "ref", "name": "main" },
        "action":     {
          "action":     "filter",
          "expression": {
            "op":         "chain",
            "expression": { "op": "ref", "name": "time" },
            "actions":    [{
              "action":     "in",
              "expression": {
                "op":    "literal",
                "value": { "start": "2015-12-17T00:01:00.000Z", "end": "2015-12-18T00:01:00.000Z" },
                "type":  "TIME_RANGE"
              }
            }, {
              "action":     "and",
              "expression": {
                "op":         "chain",
                "expression": { "op": "ref", "name": "Latitude" },
                "action":     {
                  "action":     "in",
                  "expression": {
                    "op":    "literal",
                    "value": { "start": -1, "end": 100, "bounds": "[]" },
                    "type":  "NUMBER_RANGE"
                  }
                }
              }
            }]
          }
        }
      },
      "name":       "main"
    }, {
      "action":     "apply",
      "expression": { "op": "chain", "expression": { "op": "ref", "name": "main" }, "action": { "action": "count" } },
      "name":       "count"
    }]
  },
  "timezone":   "Etc/UTC"
}

so, the plyql query above is just my debug output on server side, i mean it was converted to

$main.filter($time.in([2015-12-17T00:01:00.000Z,2015-12-18T00:01:00.000Z)).and($Latitude.in([-1,12]))).split($Region,SEGMENT,main).apply(count,$main.count()).sort($count,descending).limit(101)

by ex = Expression.fromJS(expression); console.log(ex.toString())

Actually, i've encountered an error after updating pivot from 0.7.28 to 0.8.24 and plywood 0.8.12 to the latest.

Sorry if it's not the right place for the issue, but i just checked my prev version on the app and it makes exactly the same request to the server, but receives non-empty result, as it should be.

[tonypizzicato]

Is there a chance that

"action":     {
    "action":     "in",
    "expression": {
        "op":    "literal",
        "value": { "start": -1, "end": 100, "bounds": "[]" },
        "type":  "NUMBER_RANGE"
    }
}

was mistakenly converted to $Latitude.in([-1,12]) (a set type) instead of .in(-1, 12) on server by plywood and there is a bug in Expression.fromJS(expression) for NUMBER_RANGE?

[evasomething]

Hi! Sorry for the late response to this, was able to reproduce this and am looking into it!

[evasomething]

Hi just an update on this:
.in(NUMBER_RANGE) translates to a filter in Druid that uses alphanumeric comparisons, and alphanumeric comparisons struggle with making sense of “-“.

We’ve requested to allow for numeric comparisons here:

apache/druid#2989

In the meantime we’ll change plywood so it doesn’t translate to the filter that uses alphanumeric comparisons.

[vogievetsky]

hi this should be fixed in plywood 0.10.21. Could you try and reopen this issue if it does not work

[tonypizzicato]

Works great, thanks!