Pipe, thread

[berrytj]

Pipe and thread are constructs I use frequently to organize my transformations. I'm wondering if there's a reason you don't appear to have them. They differ with compose in that 1) the order of transformations is reversed, and 2) the transformations are invoked by the call.

[Suor]

I can see how pipe is useful. But what's your use cases for thread_*?

[berrytj]

So you don't have to curry your functions. Although you could extend pipe to parse tuples and partial them for you.

That reminds me -- I use the toolz.curried namespace extensively, and rolled my own in funcy (can't live without walk_keys). That would complement pipe nicely.

[Suor]

I was thinking of adding an easier way to partial apply functions to funcy, haven't settled on an interface yet. I feel curried and normal functions would intermix creating a mess. How do you use that? Can you show some code samples?

[berrytj]

The curried namespace is one way to minimize mixing. That being said, I haven't run into any mixing issues. That might be because toolz's curry is more lenient than yours -- it allows you to give multiple args at a time (so you could do curry(add)(5)(6) or curry(add)(5, 6)).

Here's a simplified version of the script I'm currently working on. The _| ... |_ stuff is just syntactic sugar for pipe.

<script src="https://gist.github.com/berrytj/e83e56a52f1737e7ac5a.js"></script>

[berrytj]

from __future__ import division                            

from funcy.curried import *                                
from fn import _ as __                                     
from bookends import _                                     

import survey                                              


def total_probability((outcome, pmf)):                     
  return (_| pmf                                           
           | select_keys(__ == outcome)                    
           | __.values                                     
           | call                                          
           | sum                                           
           |_)                                             


def prglength_pmf(records):                                
  return (_| records                                       
           | count_by(__.prglength)                        
           | walk_values(__ / len(records))                
           |_)                                             


@curry                                                     
def born_on_or_after(records, week):                       
  return filter(__.prglength >= week,                      
                records)                                   


def main():                                                
  births = survey.read_records()                           
  outcomes = range(60)                                     
  prglength_groups = map(born_on_or_after(births),         
                         outcomes)             
  print (_| prglength_groups                         
          | map(prglength_pmf)                       
          | partial(zip, outcomes)                   
          | map(total_probability)                   
          |_)                                                

[Suor]

funcy has autocurry, which is like that curry. Works poorly with functions with optional arguments, though, so I rely more on partial.

[berrytj]

Also, regarding funcy's curry, I've run into this:

>>> from funcy import curry, walk_keys
>>> curry(walk_keys)(sum)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/t/.virtualenvs/test/lib/python2.7/site-packages/funcy/funcmakers.py", line 41, in <lambda>
    return wraps(func)(lambda f, *seqs: func(make_func(f, builtin=builtin, test=test), *seqs))
TypeError: walk_keys() takes exactly 2 arguments (1 given)

But haven't had the chance to dig into yet.

[berrytj]

Shouldn't autocurry be able to leverage anything partial can do?

[Suor]

I think you are abusing all these stuff. You code could be rewritten into:

from __future__ import division                            

from funcy import *
from whatever import _

import survey                                              


def total_probability((outcome, pmf)):
    return sum(v for k, v in pmf if k == outcome)


def prglength_pmf(records):              
    counts = count_by(_.prglength, records)
    return walk_values(_ / len(records), counts)


def main():                                                
    births = survey.read_records()      
    outcomes = range(60)                                     
    prglength_groups = [filter(_.prglength >= week, births) for week in outcomes]
    pmfs = map(prglength_pmf, prglength_groups)
    print map(total_probability, zip(outcomes, pmfs))

[Suor]

Regarding partial and autocurry. First one always works in 2 steps: take partial, call a function. The second one needs to guess whether I mean calling it or partial applying, which is not generally possible when optional arguments are involved.

[berrytj]

My abuse consists of 1) having shorter lines that do one thing each, rather than nesting, 2) not mixing list comps with functions, which I find mentally muddling, and 3) to put my data structure before my transformations, rather than after. I stand by those.

[berrytj]

WRT optional arguments, that makes sense.

[berrytj]

Although you're right about simplifying total_probability.

[berrytj]

Feel free to close this if you're not convinced.

[Suor]

I'll probably add pipe(). Threads are superseded with pipe + partial/curry, so no point in them.

[Suor]

After implementing and testing it I came to a conclusion it duplicates compose in an alternative way. So I ended up not including it.