Generative Ink

Generative art is just writing down a mathematical function $f:{\mathbb{R}}^2\to {\mathbb{R}}^3$ that takes coordinates and returns a colour. The following is one such function:

This is a write-up of a skeuomorphic ink-on-paper generative art sketch I wrote. It is very much the spiritual descendant of dipolar, but faster and with a more realistic ink texture. It isn’t intended to be an introduction to generative art or WebGL, and assumes familiarity with both.

The rendering pipeline is roughly:

1. Paths are defined in javascript and triangulated.
2. The paper heightmap and gradients are calculated and cached to a texture.
3. A vertex shader calculates the coordinates for the vertices of this triangulated path, as well as varyings for a few quantities used in ink effects.
4. A fragment shader estimates the distance of each point in screenspace from the path.
5. The ink colour is computed as an additional pass on top of this per pigment.
6. The entire canvas is given a rough paper texture.

§ Lines

To start, let’s draw some lines on the canvas — the paths are defined in javascript, adding some noise/jitter to help make it look more organic.

To render these to the canvas I used Matt Deslauriers’s blog post Drawing Lines is Hard as a reference which covers the topic of rendering paths in WebGL in much more detail. In summary: the path is triangulated such that each line segment is 2 triangles. These triangles are rendered as a solid colour on top of the canvas.

§ Paper

To give the image some roughness I add some noise to the canvas. This is the same effect I used in a few of my older pieces, e.g. dipolar and abstraction.

The baseline noise texture is standard Perlin noise $p:{\mathbb{R}}^2\to \mathbb{R}$. This is layered together at a few frequencies to give fractal Perlin noise

for some choice of parameters $a$, $f$, $N$.

The heightmap is a sum of a larger low-frequency noise contribution, flattened with $\tanh$ to give the big soft grains, and a small high-frequency contribution which gives an additional rough texture. I think it’s quite effective. The first WebGL program here calculates and returns $(h(\overrightarrow{x}),\frac{\partial h}{\partial x}(\overrightarrow{x}),\frac{\partial h}{\partial y}(\overrightarrow{x}))$ on the red, green and blue channels. These are used to seed randomness/noise in other parts of the pipeline, and since this work only has to be done once it’s good to cache it into a texture.

A second program takes a baseline colour $\overrightarrow{v}$ (which may vary across the page) and generates the visible paper texture as $\overrightarrow{v}+(\overrightarrow{l}\cdot \overrightarrow{\mathrm{\nabla }}h)\overrightarrow{1}$, which is an approximation of what a textured surface would look like lit by soft white directional lighting. Play with it below, use your mouse to change the value of $\overrightarrow{l}$ (the lighting direction).

§ Inky effects

A few simple effects are added to the vertex shader which triangulates the path, these modify the width and opacity of the path.

For the following I parameterise the path as $\overrightarrow{X}(l)$ where $l\in {\mathbb{R}}^{+}$ is the “distance from the end of the path”.

Drip is intended to capture extra thickness caused by a droplet of ink being deposited by the nib as it is lifted. The two parameters are $(D_{\mathrm{w}},D_{\mathrm{l}})$ and the width of the line at $l$ is increased by $D_{\mathrm{w}}{\text{clamp}}_{[0,1]}(1-\frac{l}{D_{\mathrm{l}}})$. With a well chosen (small) value of $D_{\mathrm{w}}$ this gives some nice directionality when there are many short strokes.

Weight is intended to simulate the effect on a line of the weight of the hand and thickest part of the pen being aligned with a downstroke. This has four parameters: $({\theta }_W,W_{\mathrm{o}p},W_{\mathrm{width}}^1,W_{\mathrm{width}}^2)$.

${\theta }_W$ defines the angle of the downstroke and $w(l)\in [-1,1]$ is the dot product of the path direction with this downstroke direction. The width of the line at $l$ is increased by $W_{\mathrm{width}}^{1}w(l)+W_{\mathrm{width}}^{2}w(l{)}^2$, and the opacity of the ink by $W_{\mathrm{o}p}w(l)$. The idea here is that the directional $W_{\mathrm{w}idth}^1$ component captures something like a flexing nib, and the bidirectional $W_{\mathrm{width}}^2$ captures something like an italic nib.

Shading captures the higher density of ink at the end of a path, caused by ink being dragged by a nib across the page. This has three parameters $(S_{\mathrm{op}},S_{\mathrm{start}},S_{\mathrm{width}})$ and increases the ink opacity by $S_{\mathrm{op}}\cdot {\text{clamp}}_{[0,1]}(1-\frac{l-S_{\mathrm{start}}}{S_{\mathrm{width}}})$.

§ Ink bleed and pigments

Drawing the ink ended up quite a bit more complicated. I wanted an ink-drawing routine which gave nice feathered ink bleed edges, and I wanted this to behave well at corners and intersections. The approach I decided to go with was to use Signed Distance Functions, computed in a fragment shader. A trick here is that you can map a colour to $f(-\text{sdf}(\overrightarrow{x}))$ for some monotonic $f:\mathbb{R}\to [0,255]$, change the blending function to gl.MAX, and compose the sdfs from different triangles/paths together:

gl.enable(gl.BLEND);
gl.blendEquation(gl.MAX);
gl.blendFunc(gl.ONE, gl.ONE)

This works because in effect this is taking the minimum of two SDFs, which gives you (outside the shapes/sets) an SDF for the shape formed from the union of the two shapes/sets. (Strictly, inside the set you no longer have an SDF, but a function which returns a lower bound on the distance from the boundary).

In the final pass, we compose a translucent colour per pigment. A pigment is an rgba tuple and a bleed value which controls how soft the edge of the path is. Layering multiple pigments per line is expensive, but gives a nice subtle effect at the edge of the path where different pigments bleed further into the paper.

The main body of the fragment shader for this one is quite concise so I’ll show it here:

float cutoff(float v) {
    return 0.5 - 0.5 * tanh(v);
}

void main() {
    float sdfNoisy = sdf() - u_penNoiseScale * height();
    float alpha = cutoff(sdfNoisy / u_bleed) * (1.0 + density());
    outColor = vec4(u_pigment.rgb, u_pigment.a * alpha);
}

This has a single extra parameter, the “pen noise scale” $\beta$ which adds some noise to the path edge. Here, density() brings in the sum of the effects described in the previous section. This is seeded from the paper noise texture, both so the paper and ink look more like they are interacting, and to save some effort in recalculating the Perlin noise.

§ Conclusions

On a modern laptop it’s pretty fast — the bottleneck seems to be the js -> flat-format array conversions. For a simple sketch with a few lines it’s fast enough to be calculated in realtime. Try drawing on the sketch below:

I’m quite proud of it! I know not everyone is a fan of skeuomorphism, but I hope you agree that it achieves what I wanted it to effectively. I find it quite useful when I’m imagining a plotter as the canonical output target. It’s fast enough to do interactive or animated pieces. There are still some issues — it currently behaves badly at sufficiently sharp corners. I think it could be made even quicker, so it could support realtime on much more complicated sketches.