Extensible Functions

[lbosson]

Hello,

First off I want to say great stuff! This library has really helped enhance an internal tool here at Jawbone where we used to require mongo query language.

Perhaps I missed something but I couldn't figure out how to add generic functions or types short of using a schema or monkey patching. The find method instantiates either SchemaFreeParser or the SchemaParser so there is no way to extend those (without patching). Perhaps find could take a parser as a parameter instead of instantiating one? Also, the issue I had with using the scheme is I don't have a complete schema so I only wanted to define certain fields with a specific type. Perhaps SchemaParser could fall back to GenericField/Type when not found in the map?

We store our timestamps in unix epoch. So I wanted a function to allow a user to compare a timestamp to a human readable date. Here's how I ended up adding EpochFunc to pql:

class EpochFunc(pql.matching.Func):
    def handle_epoch(self, node):
        return self.parse_arg(node, 0, EpochField())


class EpochField(pql.matching.AlgebricField):
    def handle_Str(self, node):
        dt = dateutil.parser.parse(node.s)
        return float(dt.strftime('%s.%f'))

    def handle_Num(self, node):
        return node.n

    def handle_Call(self, node):
        return EpochFunc().handle(node)


class GenericFunc(pql.matching.StringFunc, pql.matching.IntFunc, pql.matching.ListFunc, pql.matching.DateTimeFunc, pql.matching.IdFunc, EpochFunc):
    pass


# Monkey Patch GenericFunc with our GenericFunc which includes EpochFunc
pql.matching.GenericFunc = GenericFunc

Possible Asks:

1. Allow a parser to be passed to find.
2. Have SchemaParser fall back to Generic when field map misses.
3. Have a way to extend generic functions.

Cheers,
Lew

[alonho]

1. I'll do it.
2. Could you do it by passing in a defaultdict?
3. I'll think about this when I get back from vacation.
   I'll also add support for epochs (not sure about the dateutil format though, will have to see which timestamp format feels better)

Thanks for the ideas!

[lbosson]

1. Excellent!
2. I don't believe this will work because resolve_field in SchemaAwareOperatorMap checks to see if the field is in the map with "in". Also, I am using python 2.6 (shame on me).
3. The dateutil parser parses a bunch of various string date formats. Very similar to what you have done with your date parser. I was just lazy, I haven't compared the two.

If you would like me to work on a patch for anything let me know. I'd be glad to help out.

Cheers,
Lew

P.S. Enjoy your vacation!

[alonho]

I ended up doing the following:

1. 'find' is just a shortcut for the trivial use cases. perhaps advanced users should use parsers themselves. I kinda prefer leaving it like it, not sure.
2. It now does a getitem so defaultdict will work
3. replaced to dateutil and added your epoch field.

Thanks again!