CPPCON video

[RPeschke]

Hi,

Really great library just finished watching your cppcon presentation. really nice.

A while back I toyed myself with the push model and pipes etc.
https://github.com/RPeschke/ArggLib

since I never planed on really publishing it its not very well documented. But anyway I came across some of the same problems that you are mentioning on your limitation slide.

• reverse etc.

operations like this require that you have knowledge about when the stream is over and then act on this knowledge.

To achieve this I implemented my processor with "onStart", "onStop" functionality. The mechanism which called the processor/pipe just checks if a onStart/onEnd function exists. It starts the last processor in the pipe first and moves to the front.
it ends with calling the onEnd function of each processor front to back.

exameple:
these are example processors which only implement one part of the processor. OnStart only implements the on start function OnEnd only implements the on end function and evaluate only implements the on eval function.

  param() | proc() 
	  >> OnStart([&]  { out << "1\n"; }) 
	  >> OnEnd([&] ( ) { out << "2\n"; }) 
	  >> OnStart([&] {out << "3\n"; }) 
	  >> OnEnd([&]() {out << "4\n"; })
	  >> Evaluate([&] { out << "eval\n"; return 1; });

// "3\n1\neval\n2\n4\n");

with this model, you can buffer the input stream until you reach the end of the stream and then run it in reverse.

I am using the right shift operator to create pipes and then run the pipe by using the | operator. if the input is void I pipe in the param() helper.

• end pipe early.

Each processor in my pipeline has the possibility to send a stop signal back to the pipeline therefore each operator() of the processor has to return an enumeration which tells success, skip or stop,

• split
  I am not entirely sure I got it correctly but basically, I see two alternatives either you want to process all parts of the split simultaneously.

example:

"a string with multiple words" | magic_split | use_parts

where use_parts has a operator() which takes an "almost" arbitrary amount of input parameters. and it receives

use_parts ("a", "string", "with", "multiple", "words").
which is an exercise in the template of function. I can point you to my implementation.

Or you want to split it by time .

example:
"a string with multiple words" | magic_split2 | use_parts

use_parts("a")
use_parts("string")
use_parts("with")
use_parts("multiple")
use_parts("words")

which would be just calling the output pipe multiple times.

magic_split2 (str , next){

auto sp = split(str);
for (auto s : sp){
 next(s);
}

}

• fragments

inputs >>= transform(f)

this can be done by building an adapter which is basically a reference to the input and then calling it with a special void input type which just tells the machinery to start working.

auto fragment = ref_input(input);

void_input >>=  fragment >>= Long_and_complicated_pipeline

in my library, I generally do it by separating the creation of a pipeline and its invocation. Therefore I build the pipeline up with the right shift operator and execute it with the bit or operator.

auto x1 = param() | Import_CSV()->fileName("fileNames.txt") >> out_stream();

this example stores the string "fileName.txt" and is executed once "param() | " is applied to it.

• creating the result of the pipe

for this again I am using the onEnd function. the result of this function will be the result of the entire pipeline. In my implementation, the last something, that is returned from one of the onEnd functions will be the return of the entire pipe.

	auto x1 = param() | Import_CSV()->fileName("fileNames.txt") >> out_stream();

in this example, x1 is a smart pointer to the string streamer which is created in out_stream.

let me know if this is useful for you. I am sorry that the library is not more organized but so far it has only been my personal sandbox. Also i could not use new features since it had to compile with cern root.

Again great talk. Cheers,
Richard

[joboccara]

This is very interesting. Thanks for sharing Richard!