Bare bones generative image making

I first tried portable pixmap graphics going through The Ray Tracer Challenge in JS during batch at RC, came across it again on the way to translating Scott Draves' Fuse (1991) for the web, then a few months later as well researching image segmentation looking at e.g., Pedro Felzenszwalb's C++ graph cut solver (2004).

It kind of rocks being able to compose images straight up out of color values both in science and in practice respectively when a pixel is not a little square and when, similar to say .obj files, printf(1) and bc(1) alone can get you truly far.

I had fun putting together a /dev/urandom driven rainbow generator last year and so took up drawing Worley noise and byte beat motifs, which are more specific types of problem, in the same vein as a drill. It's refreshing that for building pipelines like those I can rely on reading the manual and not have to spend much time looking up answers online when stuck. I do have Copilot, but it almost never helps beyond auto complete to be honest.

Not that I could write the book on it, but since many AI processes like Stable Diffusion start off as noise, making something out of nothing so to speak, which is a broadly common theme in software if you analyze it, I wanted to come up with more meaningful and browser compatible results as a next step.

Chaos

I have fuse mechanics already figured out, let me start there. Inspired by GNU Emacs Dissociated Press, cannibalistic bluff teen film Society (1992), cut-up technique, and implicitly maybe if I had to guess Carolee Schneemann and James Tenney's Fuses (1964-67) home movie full of lust, the algorithm is pure joy for being on the empirical side and open ended:

1. Start with a series of images, divide them up into small tiles, typically 25x25px.
2. Shuffle those around and place one of them in the middle of canvas.
3. Fan out looking for locally coherent tiles array matches to assemble the final picture.

If the output can be merely as good as the input material, wicked kooky is a guarantee with this approach, when human parts like eyes and hands are involved in particular. I was wondering what happens processing text though. Noise, silent void, chaos and disorder, what's the difference? Joseph Haydn's The Creation (Die Schöpfung) Hob. XXI:2 (1797-98) strikes me as appropriate to be sampling from therefore.

I see Art Nouveau capital of the world Brussels Philharmonic have the libretto uploaded in both English and Old German. I expected it might have been symbolic to remove all the x's, but there are none, no joke. Too perfect, feeling irreverent and compelled to mangle up content to a degree:

\
# Download HTML.
curl -s https://www.brusselsphilharmonic.be/en/text-die-schöpfung |
# Target the first column for German, discard lint errors.
# XPath is like a jQuery for the command line.
xmllint --html --nowarning --xpath '//tr/td[1]//*/text()' 2> /dev/null - |
# Single space everything (squeeze).
tr -s [:space:] " " |
# Trim whitespace.
sed -e "s/ *$$//" |
# Introduce a few typos.
sed -e "s/ck/k/g;s/ht/th/;s/hr/rr/23" > libretto.de.txt

It may sound counterintuitive, but qlmanage(1) has a 800px soft cap when dealing with plain text files, but no issue rasterizing HTML or SVG <foreignObject>, which also means I can fit more letters into the master image using CSS, cool:

# Expanded for readability, should be line collapsed.
awk '{printf(
  "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"100%%\" height=\"100%%\">
    <style>
      pre {
        background: black;
        color: white;
        font-size: 5px;
        line-height: 1;
        white-space: wrap;
        word-spacing: -1ch;
      }
    </style>
    <foreignObject width=\"100%%\" height=\"100%%\">
      <pre xmlns=\"http://www.w3.org/1999/xhtml\">%s</pre>
    </foreignObject>
  </svg>",
$0)}' libretto.de.txt > master.svg

# Thumbnail in name only:
qlmanage master.svg -t -o . -s 2000

To draw samples from the master image on a Mac, sips(1) is agreeably JavaScript-able, for example:

// Basic sampler.js, call inside a for loop to derive fuse input images out of master:
// `for i in {1..8}; do sips -j sampler.js -o $i.png master.svg.png; done`
const w = Math.floor(Math.random() * 100) + 20
const h = Math.floor(Math.random() * 100) + 20
const canvas = new Canvas(w, h)

const [master] = sips.images
const x = Math.floor(Math.random() * master.size.width - w)
const y = Math.floor(Math.random() * master.size.height - h)

canvas.drawImage(master, x, y, w, h, 0, 0, w, h)

const tile = new Output(canvas, sips.outputPath)

tile.addToQueue()

Gibberish, happy to have traveled down this path, but would prefer some structure and some definition. Bank the templating via string interpolation and press on.

Formula

To have form, you want numbers and to arrive at numbers willy-nilly not, you have to have math, and to settle on a math you need to experiment a little. Would converting my .ppm byte beat maker to SVG lead to any gains?

As it happens no, I ran into floating point precision trouble with bc(1) using doubles internally, whereas float32 based patterns look unmistakably more absorbing in my opinion. I found no way of overriding the default setting and was convinced my calculator should be Perl instead, but was tired exploring more of this angle.

// TFW single vs. double precision floating point
// arithmetic will do your head in sometimes.
console.assert(42n ** 10n === BigInt(42 ** 10))
console.assert(43n ** 10n === BigInt(43 ** 10))

Alright, one less idea to worry about, making progress, but obviously not there yet. I had ten days budgeted for this baby doll, and about a week's worth of sandglass expenditure into my quest, crunch time panic is creeping in. The pressure is on, the heat is up, the stakes are high, the rest of my schedule is on the line, how do I manage? Give up? Cave, lose heart and duck out? Go mad? Throw the towel in? Wave the white flag? Because that would be very sad.

LOL, not so fast Mr. Doubting Thomas Sir, hold your horses, dial the negativity down a notch please, hang on a minute and please allow me a moment of pause to reflect. Okay, fortunately, I still have a bit of rope left, why not tie up any metaphorical loose ends with a stevedore knot stacking pieces of DOM up and down and around? Bingo! Killing two birds, design and interpretation, with one stone? There you go buddy, gagnant, let's ride!

I thought it was going to be easy, but failed to get a proper formula out of ChatGPT here surprisingly enough. There was some confusion about the parametric equation being in existence, but the math is in fact beautifully simple and child's play really. Stevedore's is naturally a special case of Lissajous. If supporting folks on older browsers was no requirement, I might have written the whole thing in CSS:

x = R cos(n_x t + φ_x), y = R cos(n_y t + φ_y), z = R cos(n_z t + φ_z)

Where R is the radius, n_x = 3, n_y = 2, n_z = 5, φ_x = 1.5, φ_y = 0.2, and φ_z = 0.

Transform

Brilliant, I have my data source decided fine and dandy and would now like to compile a web document, preferably portable and self-contained to house the visual. SVG comes to mind again, but the lack of CSS 3d transforms is a showstopper.

Well, I had always hoped the XSLT toil of restyling nginx index listings might prove useful one day. Browsers are limited to version 1.0 of the spec (1999), which may not be the latest and greatest, but you have to appreciate the foresight in promoting separation of concerns regardless, yes? For example a data object of:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="example.xsl" type="text/xsl"?>
<Article title="Die Schöpfung" lang="de">
  <Category hoboken="XXI:2">Oratorio</Category>
  <Composer>Joseph Haydn</Composer>
  <Description>
    Masterpiece! Depicts and celebrates the creation of
    the world as narrated in the Book of Genesis.
  </Description>
  <Librettist>Gottfried van Swieten</Librettist>
  <Completed>Herbst 1797</Completed>
  <Period>Classical</Period>
  <Premiere datetime="1798-04-30">30. 04. 1798</Premiere>
  <Premiere datetime="1799-03-19" type="public">19. 03. 1799</Premiere>
  <Section title="NR. 2. REZITATIV (RAPHAEL & CHOR)">
    <Lyric>
      <Character>RAPHAEL</Character>
      <Lines>
        Im Anfange schuf Gott Himmel und Erde,
        und die Erde war ohne Form und leer,
        und Finsternis war auf der Fläche der Tiefe.
      </Lines>
    </Lyric>
    …
  </Section>
</Article>

Yields:

Given an example.xsl stylesheet along the lines of:

<?xml version="1.0"?>
<xsl:stylesheet
  version="1.0"
  xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
  <xsl:output
    method="html"
    encoding="utf-8"
    doctype-system="about:legacy-compat"
    indent="yes"/>
  <xsl:template match="/">
    <html lang="{Article/@lang}">
      <head>
        <meta name="description" content="{Article/Description}"/>
        <title>
          <xsl:value-of select="Article/@title"/>
        </title>
        <style>…</style>
      </head>
      <body>
        <article>
          <h1><xsl:value-of select="Article/@title"/></h1>
          …
          <xsl:for-each select="Article/Section">
            <xsl:for-each select="Lyric">
              <p>
                <strong><xsl:value-of select="Character"/></strong>
                <span><xsl:value-of select="Lines"/></span>
              </p>
            </xsl:for-each>
          </xsl:for-each>
        </article>
      </body>
    </html>
  </xsl:template>
</xsl:stylesheet>

Fits like a glove. I can tell httpd.conf(5) to serve up the data file directly without HTML embedding even, grand.

Not entirely a must, but if the HTML is rendered in-browser and the data is static, no reason being blindly purist about it, kindly ask JS to automatically reload the page before lunch.

// Might as well for the sake of it, because of CDATA retro attraction
// and meta#http-equiv being dumb. Run a tab switching aware timer and
// reload the page noons daily in near sync with new product delivery.
const then = new Date()
const deadline = new Date(then)

if (then.getHours() < 12) {
  deadline.setHours(12)
} else {
  deadline.setHours(36)
}

deadline.setSeconds(0)
deadline.setMinutes(0)

document.body
  .animate(null, { duration: 500, iterations: 1 })
  .addEventListener("finish", function loop(e) {
    const now = new Date()

    if (now > deadline) {
      location.reload()
    } else {
      e.target.play()
    }
  })

Give it a schmancy color palette, oversaturated neon possibly, call it "Untitled (Reimagined) No. XXIII," put on a? 𝔫𝔣𝔱, Bob's Yer Uncle! Just kidding. No offense darling, what is that, some sort of ribbon? So you found declarative ways of sketching pseudo 3d shapes using ancient general commands readily available on most systems? Big deal, is that best you can do? How do you bring real life traces into it?

Kicker, The

Have you heard? Earth's rotation rate is variable and slowing down long term on average partly due to "tidal braking and the redistribution of mass within, including oceans and atmosphere," which is why our civilization invented and keeps track of leap seconds. Is that right? Our measure of time is inevitably convention? Lord have mercy, and besides undeniable uncertainty on other fronts the UTC scale differs in realization from lab to lab by a few nanoseconds? 🙀

# Probe for leap second updates in one fell swoop. Twice a year will suffice.
# NOTE: Remember to URL encode the @ sign when giving your email as password.
# Sourcing credentials from ~/.netrc here.
\
ftp -VMo - ftp://ftp.boulder.nist.gov/pub/time/leap-seconds.list | # Call NIST.
grep -o '^[^#]*' | # Strip out all the comments
tail -n1 | # Pluck out the last line
awk '{print $NF}' # Check current tally, game on if 38 or more.

Is there going to be further adjustments by 2035 when the new standard kicks in? Who knows. In the event that there is however, be advised, my code is watching and will be gladly taking a break to mark the occasion. With any luck then my efforts would not have been in absolute vain.

More

• Adam Finkelstein and Sandy Farrier's JFK / MM (1994)
• UK MSF radio time signal technical information
• The Creation recorded live for BBC Sounds (2015)