FGenerators.jl

FGenerators: foldl for humans™

FGenerators.jl is a package for defining Transducers.jl-compatible extended foldl with a simple @yield-based syntax. Here are a few examples for creating ad-hoc "generators":

julia> using FGenerators

julia> @fgenerator function generate123()
           @yield 1
           @yield 2
           @yield 3
       end;

julia> collect(generate123())
3-element Array{Int64,1}:
 1
 2
 3

julia> sum(generate123())
6

julia> @fgenerator function organpipe(n::Integer)
           i = 0
           while i != n
               i += 1
               @yield i
           end
           while true
               i -= 1
               i == 0 && return
               @yield i
           end
       end;

julia> collect(organpipe(3))
5-element Array{Int64,1}:
 1
 2
 3
 2
 1

julia> @fgenerator function organpipe2(n)
           @yieldfrom 1:n
           @yieldfrom n-1:-1:1
       end;

julia> collect(organpipe2(2))
3-element Array{Int64,1}:
 1
 2
 1

FGenerators.jl is a spin-off of GeneratorsX.jl.

Use FLoops.jl to iterate over the items yielded from the generator:

julia> using FLoops

julia> @floop for x in generate123()
           @show x
       end
x = 1
x = 2
x = 3

Adding fold protocol to existing type

The foldl protocol can be implemented for an existing type T, by using the syntax @fgenerator(foldable::T) do .. end:

julia> struct OrganPipe <: Foldable
           n::Int
       end

julia> @fgenerator(foldable::OrganPipe) do
           n = foldable.n
           @yieldfrom 1:n
           @yieldfrom n-1:-1:1
       end;

julia> collect(OrganPipe(2))
3-element Array{Int64,1}:
 1
 2
 1

Note that inheriting Foldable is necessary only if using Base API such as collect. It is not necessary when using just Transducers.jl API (including FLoops.@floop).

Defining parallelizable collection

@fgenerator alone is not enough for using parallel loops on the collection. However it can be easily supported by defining SplittablesBase.halve and length (or SplittablesBase.amount if length is hard to define). Since halve and length has to be implemented on the same existing type, @fgenerator(...) do notation as above should be used. Extending OrganPipe example above:

julia> using SplittablesBase

julia> function SplittablesBase.halve(foldable::OrganPipe)
           n = foldable.n
           return (1:n, n-1:-1:1)
       end;

julia> Base.length(foldable::OrganPipe) = 2 * foldable.n - 1;

julia> @floop for x in OrganPipe(2)
           @reduce(s += x)
       end
       s
4

Using @floop in @fgenerator

@floop can be used inside @fgenerator

julia> @fgenerator function ffilter(f, xs)
           @floop for x in xs
               if f(x)
                   @yield x
               end
           end
       end;

julia> collect(ffilter(isodd, generate123()))
2-element Array{Int64,1}:
 1
 3

julia> collect(ffilter(isodd, organpipe(3)))
3-element Array{Int64,1}:
 1
 3
 1

julia> collect(ffilter(isodd, 1:5))  # fallback to `Base.iterate`
3-element Array{Int64,1}:
 1
 3
 5

@fgenerator function f(...) ... end
@fgenerator f(...) = ...
@fgenerator(generator::GeneratorType) do; ...; end

Define a function f that returns a "generator" usable with Transducers.jl-compatible APIs (akd foldable interface). In @fgenerator(generator::GeneratorType) do ... end form, define Transducers.jl interface for GeneratorType.

Use @yield in the function body for producing an item. Use return to finish producing items.

See also FGenerators.

Extended help

Examples

Defining a function that returns a generator:

julia> using FGenerators

julia> @fgenerator function generate123()
           @yield 1
           @yield 2
           @yield 3
       end;

julia> collect(generate123())
3-element Array{Int64,1}:
 1
 2
 3

Defining foldable interface for a pre-existing type:

julia> struct Counting end;

julia> @fgenerator(itr::Counting) do
           i = 1
           while true
               @yield i
               i += 1
           end
       end;

julia> using Transducers  # for `Take`

julia> Counting() |> Take(3) |> collect
3-element Array{Int64,1}:
 1
 2
 3

Note that function such as collect and sum still dispatches to iterate-based methods (above Counting example worked because Counting was wrapped by Take). To automatically dispatch to foldl-based methods, use Foldable as the supertype:

julia> struct Count <: Foldable
           start::Int
           stop::Int
       end;

julia> @fgenerator(itr::Count) do
           i = itr.start
           i > itr.stop && return
           while true
               @yield i
               i == itr.stop && break
               i += 1
           end
       end;

julia> collect(Count(0, 2))
3-element Array{Int64,1}:
 0
 1
 2

@yield item

Yield an item from a generator. This is usable only inside special contexts such as within @fgenerator macro.

@yieldfrom foldable

Yield items from a foldable. This is usable only inside special contexts such as within @fgenerator macro.

Examples

julia> using FGenerators

julia> @fgenerator function flatten2(xs, ys)
           @yieldfrom xs
           @yieldfrom ys
       end;

julia> collect(flatten2(1:2, 11:12))
4-element Array{Int64,1}:
  1
  2
 11
 12