The terror of the sea

Let AI illustrate your sea shanties - what can possibly go wrong?

Apr 16, 2021

So I did an experiment earlier where I used Vadim Epstein’s CLIP+FFT program to illustrate each line of The Wellerman sea shanty. Since it treats each line independently and tries to produce images that the early 2021 internet would have paired with it, CLIP+FFT will do things like interpret “We’ll take our leave and go” as Brexit-related, and “The Wellerman” as an oil drilling supervillain.

It turns out making AI-illustrated sea shanties is a lot of fun.

I give you “Ten Thousand Miles Away,” as sung by Stephen Ware and illustrated by CLIP+FFT.

It crams each illustration full of detail, trying to interpret each line as strongly as it possibly can. It gets several tries at the chorus line “I’ll stay no more on England’s shore” and in each attempt it tries waves, rows of cottages, union jacks, ships, wistful figures on the seashore, wistful figures wearing felt hats, wistful figures with Beatles haircuts, and the occasional crowd of redcoat soldiers.

The image slides smoothly between seashore full of old tall ships, steep-roofed cottages overlooking the seashore, rolling waves, ships on rolling waves, more cottages, lots of wistful looking human figures (which have weirdly duplicated faces), all in a confusing collage whose perspective almost but not quite makes sense.

The illustrations for “And sing blow, ye bully boys, blow” are hilarious, as people wearing mean-kid haircuts and private school uniforms croon into microphones. (It’s going with the modern meaning of “bully” I guess.)

I also have been dying to illustrate Barrett’s Privateers, a Canadian classic, and Stephen Ware provided a version of this, too.

I think I’m most impressed by CLIP+FFT’s rendition of “Now I’m a broken man on a Halifax Pier”, atmospheric, rainy, and containing several prominent Halifax scenes.

There are sad-looking people with red hair in 18th century muttonchops, looking down at a rain-soaked pier from various angles. Some pavillions, sea, and low green islands in the distance.

Then sebastian büttrich sent me a doom metal version of Barrett’s Privateers, which was something I was definitely not expecting. I experimented with ways to get CLIP+FFT to produce more doom metal-appropriate illustrations, and the thing that worked best was simply to append “+ darkness” to the end of each line.

It changed the illustrations DRAMATICALLY. Here’s “How I wish I was in Sherbrooke now”, then “How I wish I was in Sherbrooke now + darkness”.

A gleaming city with spires and yellow-walled streets, with strange sparkling green crystals everywhere. Barely visible in several places is the word Sherbrooke.

There are spires and yellow-walled streets, but now everything is shrouded in snow and darkness. A moon, or possibly an eclipsed sun, shines in the sky.

It really leans into “For the scummiest vessel I’d ever seen”.

Skeletal faces grin above old-style sailing ships made of shadowy mist. In several places the mist is sucked into huge dark voids.

Also twitter user bagofmoons sent me a clean, nearly a cappella version of Barrett’s Privateers, so I decided to see what would happen if I ran the recording through a speech to text transcription program. It turns out that singing REALLY throws off the transcription. The only line that it consistently gets is “God damn them all” and even then it will sometimes read it as “God damn them ball” or even “God damn the mall”.

Here’s the full version.

I think my favorite misheard lyric illustration is “On the King’s birthday we put to sea” mistranscribed as “On the King’s birthday we put the CD.” There’s the CD, dutifully put front and center in the king’s birthday party.

Many copies of a person in ruffled collar and crown, surrounded by smiling crowds in more ordinary gear. There is a cake with lit candles, and also a single shining CD.

The full lyrics are in the YouTube video descriptions, and I also highlighted a few more of my favorite misheard frames in the bonus post that you can get if you’re an AI Weirdness supporter. Or become a free subscriber to get new AI Weirdness posts in your inbox!