PBS Off the Book has a nice short video on The Art of Glitch. It’s a fun story about a born-digital art phenomena, but aside from that, I think it’s useful at helping us better understand the nature of digital objects. In the video, artist Scott Fitzgerald  gives the following concise argument for the value of glitching, or breaking copies of digital files on purpose.

“Part of the process is empowering people to understand the tools and underlying structures, you know what is going on in the computer. As soon as you understand the system enough to know why you’re breaking it then you have a better understanding of what the tool was built for.”

I think we would all do well to develop a more visceral sense of what files exactly are,  and I think some of his tactics for glitching can help with that.

A different way to read an MP3

changing a file extension

Digital objects are encoded information. They are bits encoded on some sort of medium. We use various kinds of software to interact with and understand those bits. In the simplest terms software reads those bits and renders them. You can get a sense of how different software reads different objects by changing their file extensions and opening them with the wrong application.

For example, you can  listen to this performance of the West Virginia Rag from Fiddle Tunes of the Old Frontier: The Henry Reed Collection. From that page you can download a .mp3 and .wav copy of the recording. Once you’ve done that, instead of opening and playing the files with a media player, try changing the file extension to .txt and then open the file up in your text editor of choice.

Below you can see an example of the kind of mess you can create by changing a file extension. My text editor has no idea what to do with a lot of the information in this mp3. The text editor software is attempting to read the bits in the file as alphabetical characters and it isn’t having a lot of success.

The MP3 opened in a text editor

While is a big mess, notice that you read some text in there. Notice where it says “ID3″ at the top, and where you can see some text about the object and information about the collection. What you are reading is embeded metadata,  a bit of text that is written into the file. They are part of the  ID3 tags. We can read them in a text editor because the text editor can make sense of those particular arrangements of information as text.

Another way to view an MP3

Now, if you go back, and change the extension again, you can get something that looks a bit more interesting. This time, change it from .txt to .raw and open it in some image editing software. Here is what I saw when I did that with both a .mp3 version of the file and a .wav version. The black and white pixelated images below are screenshots of my image editing program attempting to read the MP3 as a RAW file. These are visual interpretations of the particular set of the information in those audio files.

viewing the .mp3 as a .raw

viewing a .wav of the same recording as a .raw

Look at the difference between the .mp3 on the left and the .wav on the right. What I like about this comparison is that you can see the massive difference between the size of the files visualized in how they are read as images. Notice how much smaller the black and white squares are. It’s also neat to see a visual representation of the different structure of these two kinds of files. You get a feel for the patterns in their data.

Beyond just incorrectly reading these kinds of files, we can use the same sort of tactics to start to incorrectly edit them and further expose the logic of how they are encoded.

Edit an Image with a Text editor

A similar approach works with digital images. For example,  start with this image, “Sod house, Grassy Butte, North Dakota, on Catherine Zakopayko farm.” If you download the .jpg version of the image, and change it’s file extension to .txt you can open it up in a text editor. It will look like gibberish. In this case, because of the way that compression works on .jpg files you can delete chunks of the file in the text editor, save the file, change the extension back to .jpg and see what would happen if the particular chunk of the file was lost.

You can see comparisons between the original image and two levels of degradation I created by cutting out chunks of the data in the file and copying and pasting parts of it into itself.

original image

Some degradation of the image

Extensively damaged image file

In the second image, notice how the removal of a block of information has degraded the image. The entirety of the image is still there, it’s just that a rectangular region is magenta and two slices across the image are grey. The compression algorithms used to create jpg files mean that removing a chunk of the file doesn’t necessarily remove a chunk of the image, it results in removing some of the information that is layered into the image. In the further degraded image you can see how additional removal can result in big stripes of grey and similar kinds of color problems.

What was that about Screen Essentialism?

New media and digital humanities scholars have coined the phrase “screen essentialism” to refer to a problem  in many scholarly approaches to studying digital objects. The heart of the critique is that digital objects aren’t just what they appear to be when they are rendered by a particular piece of software in a particular configuration. They are, at their core, bits of encoded information on media. While that encoded information may have one particular intended kind of software to read or present the information we can learn about the encoded information in the object by ignoring how we are supposed to read it. We can change a file extension and read against the intended way of viewing the object.

This might seem like a rather academic point, however, I think it suggests the value of understanding the integrity of digital objects not simply as “looking right” in one particular reading out to the screen. In many cases, the integrity of the objects is something that can be expressed through a range of software enabled readings of it.

I’m curious to hear what folks have to say about these glitched files? What other things can they tell us about how these files work? Are there other ways to glitch files that you know of that you think can facilitate the same kinds of understanding? Lastly, what do you make of screen essentialism?