about the work

I didn't particularly enjoy the 1922 classic film, Häxan. I had to watch it in several shorter sittings in order to finish the entire film. I think the standards for entertainment have come a long way in the past 100+ years. What was, at the time, horrifying and bizarre seems today to be trite and bland. Maybe that's just a sign of the gradual decay of society...

Anyways, I had thought before I even watched the film that it might be fun to introduce glitches into the datastream in order to achieve some interesting visual effect. After watching the film, I thought it would be fun to work by directly manipulating the hexadecimal representation of the binary data stream (Häxan -> hex -> hexadecimal). I also wanted specifically to try to introduce colour into the otherwise black-and-white film. I think I succeeded on both fronts.

My first attempt at glitching simply NULL-ed out every 10'th frame of the h264 data. Because h264 is encoded by using offsets and directions, this doesn't simply give a black screen every 10 frames. Instead, it introduces some crazy-looking smears, glitchy squares, and dragging of the image as playback goes on. Looks very cool, but does not introduce any colour.

My second attempt was way too over the top. It transcoded the h264 file into the raw yuv420p frame-by-frame data, and then drew some colourful noise and sine waves on top of it. It looked kind of cool, but also kind of boring. The same patterns repeated over and over. Also, due to the addition of noise, the recompression process was very inefficient - our little 650 MB mp4 blew up over 7 GB (not to mention the raw yuv420p binary data was about 60 GB before being recompressed). The worst part is that due to the inefficiency of the compression, my strategy of NULL-ing out every n frames couldn't reliably introduce any glitched into the video (the high entropy provided by the noise acts as a frame "anchor" and ensures that the playback remains smooth). This wasn't acceptable to me.

In my third and final attempt I went back to working with only the h264-encoded data. I did some research on how the frames work and came up with a strategy to systematically introduce glitches that should result in colour being introduced by the decoding errors. The trick was to only look at I-frames and rather than NULL-ing them out, to randomly twiddle the least-significant bits in a pattern that biases towards more mutations at the end of the frame (where chroma data is stored) than at the beginning of the frame. I was quite pleased with the results: in addition to flashes of colour being introduced, there's the "melting" effect that is applied to the text. I only modulated 1 out of every 4 frames, so that the movie would still be more-or-less "watchable" (on VLC, at least).

I find my version more enjoyable - psychedelics are optional but recommended. You can see exactly how I did this, provided you prove yourself by breaking the witch's hex that is protecting the source code 😉

how to run

Download the source code to a file called häx.py

Any modern python3 variant should be fine. You'll also need ffmpeg. No other requirements are needed. there's 3 steps: first we extract just the h264 file out of the mp4 using ffmpeg, then we glitch the x264 to create a new glitched h264 file, then we use ffmpeg to recombine the glitched h264 file with the rest of the original mp4 to generate a playable glitched mp4 file. In bash, we can do the following, starting from the original 'Häxan (1922).mp4' file from the Internet Archive :

$ ffmpeg -i Häxan\ \(1922\).mp4 -vcodec copy -bsf:v h264_mp4toannexb -f h264 haxen.h264
$ python3 häx.py haxen.h264 haxen_glitched.h264
$ ffmpeg -i haxen_glitched.h264 -i Häxan\ \(1922\).mp4 -map 0:v:0 -map 1:a? -map 1:s? -c copy -bsf:v h264_metadata=aud=insert haxen_glitched.mp4

and enjoy watching haxan_glitched.mp4. Due to the use of a random number generator, every run is subtly different!