Manual
TLDR: Python / C# yield
with performance matching plain Julia iterators (i.e. unbelievably fast)
Continuables are generator-like higher-order functions which take a continuation as an extra argument. The key macro provided by the package is @cont
which will give access to the special function cont
within its scope and wraps the computation in a special Type Continuables.Continuable
. It is best to think of cont
in the sense of yield
from Python's Generators. It generates values and takes feedback from the outer process as return value.
If you come from Python, use Continuables wherever you would use generators. If you are Julia-native, Continuables can be used instead of Julia's Channels in many place with drastic performance-improvements (really drastic: in the little benchmark example below it is 20 million times faster!).
This package implements all standard functions like e.g. collect
, reduce
, any
and others. As well as functionalities known from Base.Iterators
and IterTools.jl
like take
, dropwhile
, groupby
, partition
, nth
and others.
For convenience, all methods also work for plain iterables.
Example of a Continuable
Let's define our fist continuable by wrapping a simple range iterator 1:n
.
using Continuables
# new Continuable ---------------------------------------------
corange(n::Integer) = @cont begin
for i in 1:n
cont(i)
end
end
That's it. Very straight forward and intuitive.
Many standard functions work seamlessly for Continuables.
using Continuables
collect(corange(10)) == collect(1:10)
co2 = map(corange(5)) do x
2x
end
collect(co2) == [2,4,6,8,10]
foreach(println, corange(3)) # 1, 2, 3
foreach(chain(corange(2), corange(4))) do x
print("$x, ")
end # 1, 2, 1, 2, 3, 4,
reduce(*, corange(4)) == 24
all(x -> x < 5, corange(3))
any(x -> x == 2, corange(3))
map(corange(10)) do x
corange(x)
end |> flatten |> co -> take(co, 5) |> collect == Any[1,1,2,1,2]
collect(product(corange(2), corange(3))) == Any[
(1, 1),
(1, 2),
(1, 3),
(2, 1),
(2, 2),
(2, 3),
]
collect(partition(corange(11), 4)) == [
Any[1,2,3,4],
Any[5,6,7,8],
Any[9,10,11],
]
using OrderedCollections
groupbyreduce(isodd, corange(5), +) == OrderedDict{Any, Any}(
true => 9,
false => 6,
)
nth(3, ascontinuable(4:10)) == 6
nth(4, i2c(4:10)) == 7
nth(5, @i2c 4:10) == 8
# further defined are `takewhile`, `drop`, `dropwhile`, `repeated` and `iterate`, as well as `groupby`.
Importantly, Continuables do not support Base.iterate
, i.e. you cannot directly for-loop over a Continuable. There is just no direct way to implement iterate
on top of Continuables. Give it a try. Instead, you have to convert it into an Array first using collect
, or to a Channel using aschannel
.
The same holds true for zip
, however we provide a convenience implementation where you can choose which interpretation you want to have
# uses Channels and hence offers lazy execution, however might be slower
zip(i2c(1:4), i2c(3:6), lazy=true) # Default
# uses Array, might be faster, but loads everything into memory
zip(i2c(1:4), i2c(3:6), lazy=false)
Last but not least, you can call a Continuable directly. It is just a higher order function expecting a cont
function to run its computation.
continuable = corange(3)
foreach(print, continuable) # 123
# is the very same as
continuable(print) # 123
The @Ref
macro
As you already saw, for continuables we cannot use for-loops. Instead we use higher-order functions like map
, foreach
, reduce
or groupbyreduce
to work with Continuables. Fortunately, julia supports beautiful do
syntax for higher-order functions. In fact, do
becomes the equivalent of for
for continuables.
However, importantly, a do
-block constructs an anonymous function and consequently what happens within the do-block has its own variable namespace! This is essential if you want to define your own Continuables. You cannot easily change an outer variable from within a do-block like you may have done it within a for-loop. The solution is to simply use julia's Ref
object to get mutations instead of simple variable assignments. For example instead of var_changing_every_loop = 0
, and an update var_changing_every_loop += 1
you use var_changing_every_loop = Ref(yourvalue)
and var_changing_every_loop.x += 1
.
(If you would use something mutable instead like an Vector instead of the non-mutable Int here, you of course can directly work in place. I.e. say a = []
, then push!(a, i)
will do the right thing also in a do-block).
For convenience, Continuables comes with a second macro @Ref
which checks your code for variable = Ref(value)
parts and replaces all plain assignments var = newvalue
with var.x = newvalue
. This makes for beautiful code. Let's implement reduce with it:
using Continuables
@Ref function myreduce(continuable, merge, init)
accumulator = Ref(init)
continuable() do x
accumulator = merge(accumulator, x)
end
accumulator
end
myreduce(i2c(0:5), +, 0) == 15
Let's check that @Ref
indeed only replaced accumulator
with accumulator.x
. Run @macroexpand
on the whole definition, i.e. @macroexpand @Ref function myreduce(....
, which returns
:(function myreduce(continuable, merge, init)
accumulator = Ref(init)
continuable() do x
accumulator.x = merge(accumulator.x, x)
end
accumulator.x
end)
When combining @cont
with @Ref
do @cont @Ref ...
, i.e. let @cont
be the outer and @Ref
be the inner macro.