Add "immediate mode" algebra ("Doodle Canvas")

[noelwelsh]

Overview

Create an algebra that allows treating a picture as a bitmap on which we can perform basic bitmap operations (get pixel, set pixel, and more interesting things like drawing lines and circles). This would allow creating certain kinds of generative art that are more concerned with pixel editing than structured composition of shapes, and allow creating pictures that are too large to represent as a collection of data structures.

In computer graphics, drawing immediately to screen is known as "immediate mode", while constructing a data structure (a "scenegraph") that is then used to control drawing to screen is known as "retained mode". Doodle is retained mode; Picture is a data structure that describes what should appear on screen. This project adds immediate mode functionality to Doodle, while working with the existing retained mode framework.

Details

The current Doodle implementation is designed for pictures can be easily expressed by describing the relationships between elements. E.g. an element may be placed above or beside another element. This makes it easy to express certain types of pictures but hard to express others. The pictures that are hard to express are typically ones that have thousands or millions of elements, or ones where the relationships between elements are either random or derived from complex formula.

Generative art (see, for example, https://georgemsavva.github.io/creativecoding/posts/flowfields/) is one kind of picture that is difficult to express with Doodle. A plot with many data points is another. You certainly can describe these kinds of pictures in Doodle but they are tedious to write and inefficient to render. The majority of such pictures would consist of calls to at, to position elements relative to an origin, and on to give all the elements the same origin. This will build a very large data structure (a Picture) that will then be slow to draw.

An alternative is to represent a picture as a sequence of imperative commands that draw on the screen using a shared coordinate system. For example, instead of writing

val retained =
  Picture.circle(10).at(100, 100)
    .on(Picture.circle(10).at(-100, -100))

we could write

val immediate =
  (gc: GraphicsContext) => {
    // First parameter is diameter, second is x, third is y
    gc.circle(10, 100, 100)
    gc.circle(10, -100, -100)
  }

Here, a GraphicsContext is some object that provides methods for drawing on the screen.

Doodle takes the first approach (retained). A Picture is a data structure that represents what should be drawn on screen. The latter approach (immediate) is the one taken by most comparable graphics software, such as P5JS: a sequence of commands that draw directly to screen with no intermediate data structure.

Our goal is to add the advantages of the immediate mode, while retaining the advantages of retained mode. We can do this by extending Doodle with a new algebra that allows us to express immediate mode drawing. This would produce a Picture, that can then be laid out relative to other Pictures using above, on, and so on, but the contents of this Picture are created by calling a user supplied function. The user supplied function has access to a GraphicsContext that draws relative to the origin of the Picture.

Let's call this a Raster. We might define:

trait Raster extends Algebra {
  def raster(width: Int, height: Int)(f: GraphicsContext => Unit): Drawing[Unit]
}

which declares a width * height element that calls f to render it's content. GraphicsContext is some abstraction that captures standard 2D graphics operations. (The Java2D Graphics2D type is an example.)

The user can write (assuming we have syntax raster to convert a GraphicsContext => Unit to Picture[Unit])

val twoCircles =
 (gc: GraphicsContext) => {
    gc.circle(10, 100, 100)
    gc.circle(10, -100, -100)
  }

val masterpiece = twoCircles.raster(240, 240).beside(Picture.square(100))

which allows us to seamlessly integrate the two styles of drawing and use each where it is best suited.

(This is essentially a Church encoding. The Church / Free duality strikes again!)

Resources

• JDK Graphics2D
• p5js API
• HTML Canvas
• Skija