This crate was an experiment to work with generators as iterators within for loops.
Generators in Rust are a kind of stackless coroutine, which are written like closures and can function like iterators. For example:
let generator = || {
for i in 0..10i32 {
yield i;
}
}
This generator returns the values 0
through 9
before returning.
It would be great if you could drive an iterator in a for loop.
for value in generator {
println!("{}", value);
}
But this will fail to compile, as generators are not iterators and don't implement IntoIterator
. geniter
provides a macro to help
for value in geniter!(generator) {
println!("{}", value);
}
Generators don't implement IntoIterator
for good reason: generators can be resumed with arguments, unlike iterators.
Take this example:
let mut read_line = |mut arg| {
let mut buf = String::new();
loop {
match arg {
'\0' => {
// break on null characters and end sequence
arg = yield Poll::Ready(buf.clone());
return;
},
'\n' => {
// break on new lines and clear buffer
arg = yield Poll::Ready(buf.clone());
buf.clear();
}
_ => {
// buffer character and yield pending
buf.push(arg);
arg = yield Poll::Pending;
}
}
}
}
This is an asynchronous version of read_line
as a generator. This is how we drive it today:
let text = "first line\nsecond line\nlast line\0";
for ch in text.chars() {
match Generator::resume(Pin::new(&mut read_line), ch) {
GeneratorState::Yielded(yielded) => {
match yielded {
Poll::Ready(line) => println!("{}", line);
Poll::Pending => (), // nothing else to do
}
},
GeneratorState::Returned(_) => break,
}
}
This example highlights how geniter
allows generators in for loops even if they don't take ()
as their resume argument: the macro requires you to provide an iterator to bind to the resume arguments of the generator.
for line in geniter!(text.chars() => read_line) {
println!("{}", line.await); //note: this isn't valid, read_line doesn't return a future, just a Poll result. It could though!
}
But there's another good reason that generators aren't IntoIterator
: they can return anything, iterators only every return Some(Item)
or None
. geniter
supports this using a callback called then
, which is executed when the generator is returned.
let then = |returned| println!("returned: {}", returned);
for line in geniter!(text.chars() => read_line, then) {
println!("{}", line.await);
}
- I'm convinced that we can use the result of a
for
loop expression more gracefully, eg allowingbreak
to return a value that can be assigned to a variable binding or passed into a function. For example, consider an iterator bound to generator that returns an iterator of futures to evaluate the loop concurrently or to schedule their evaluation. - Generators have to be pinned. This seems to put limits on how useful generators can be as general purpose iterators or streams, but I don't know all of the implications.
- It would be great to pipe generators from one to the other, eg
for data in geniter!(tcp_stream => decode => parse)
. - This is tightly coupled to async iteration semantics and for await loops.