avocADO - Safe compile-time parallelization of for comprehensions

Example

import cats.effect.IO

import avocado.*
import avocado.instances.cats.given

val run: IO[Int] =
  parallelize {
    for {
      a <- doStuff1
      b <- doStuff2(a)
      c <- doStuff3
      d <- doStuff4(a)
    } yield combine(a, b, c, d)
  }

avocADO will transform the above for-comprehension to code equivalent to:

for {
  a <- doStuff1
  (b, c, d) <- doStuff2(a).zip(doStuff3).zip(doStuff4(a))
} yield combine(a, b, c, d)

Description

avocADO is a small library that allows for automatic rewriting of for comprehensions to their parallel versions.

The name avocADO is a pun on the function that is the inspiration for this library - ado from Haskell's language extension ApplicativeDo.

Usage (with build tools)

sbt

libraryDependencies ++= Seq(
  "org.virtuslab" %% "avocado" % "version from the badge",
  "org.virtuslab" %% "avocado-cats" % "version from the badge", // for Cats
  "org.virtuslab" %% "avocado-zio-2" % "version from the badge", // for ZIO 2.x
  "org.virtuslab" %% "avocado-zio-1" % "version from the badge", // for ZIO 1.x,
  "org.virtuslab" %% "avocado-zio-query" % "version from the badge", // for ZIO Query
)

scala-cli

//> using lib "org.virtuslab::avocado:version from the badge"
//> using lib "org.virtuslab::avocado-cats:version from the badge" // for Cats
//> using lib "org.virtuslab::avocado-zio-2:version from the badge" // for ZIO 2.x
//> using lib "org.virtuslab::avocado-zio-1:version from the badge" // for ZIO 1.x
//> using lib "org.virtuslab::avocado-zio-query:version from the badge" // for ZIO Query

Usage (in code)

All you need to do in order to use avocADO is to import the parallelize function and an AvocADO instance for your effect system. i.e.

import avocado.* // This line exposes the `parallelize` function - entrypoint of the library
// You should choose one of the following imports depending on your effect system of choice
import avocado.instances.cats.given
import avocado.instances.zio2.given
import avocado.instances.zio1.given
import avocado.instances.zioquery.given

And that's it! All that's left is to wrap the for-comprehensions that you want to parallelize with a call to parallelize an watch your program run in parallel! Like so:

parallelize {
  for {
    ...
  } yield ...
}

Usage (custom monads)

On the off chance that avocADO doesn't provide an instance for your favourite effect monad, you might have to write an instance of our AvocADO typeclass yourself. Don't worry, it's relatively straightforward.

AvocADO is just a Monad with a changed name, so that it can be easily associated with avocADO. So if you want to write an instance for a class called Effect it might look like so:

given AvocADO[Effect] = new AvocADO[Effect] {
  def pure[A](a: A): Effect[A] = Effect.pure(a)
  def map[A, B](fa: Effect[A], f: A => B): Effect[B] = fa.map(f)
  def zip[A, B](fa: Effect[A], fb: Effect[B]): Effect[(A, B)] = fa.zipPar(fb) // This is the most important method
  def flatMap[A, B](fa: Effect[A], f: A => Effect[B]): Effect[B] = fa.flatMap(f)
}

Every parallel part of the computation will be rewritten to a call to zip, so in order to achieve any speedup, you have to provide a parallel implementation for zip.

References

Inspired by Haskell's Applicative do-notation

Similar project: kitlangton/parallel-for (only for Scala 2)