“Bring a thing!” Such a simple and innocuous prompt to incite fear and apprehension in your humble free software developer. A thing? But I write software!

I had been exploring the idea that careful reading could be a creative process—that the traces from active navigation of media could themselves be media. And I had been writing software around that concept. Perhaps I had achieved at least a virtual tangibility for digital video collections that would otherwise float through the ether-net, leaving traces exclusively for the advertising and spy agencies with an interest in one form of targeting or another. With InterLace, the playback rectangle is always shown in context, and every attempt is made to treat the viewer as a full participant in determining their points of focus.

While my timelines gave graphic form to the passage of time—at least a shadow of the moment—I still didn’t think my software qualified for membership in the world of things. So was born Hyperopia Thing. Building on the Kiwix standalone Wikipedia server, I wrote an experimental wiki interface that would open links inline and implicitly build an “associative trail” in the right-hand sidebar with all of the links, selections, and images clicked on while browsing. To make it a “thing,” I looked to the past and appropriated the book-form of yesteryears’s reference collection: I loaded all of Wikipedia onto a microSD card in a Raspberry Pi, and carved it into an old Dutch encyclopedia. A new research lab should have a copy of Wikipedia for their shelves, I reasoned, even if the physical form was symbolic.

The thermal printer was a joke, but like many good jokes it turned out to be more “true” than expected. Housed inside the physical encyclopedia, I wired up a receipt printer to create an instant record of the reading process, emanating out from the soul of the book. At first I saw the receipts as a gimmicky echo of the digital sidebar, but as I spend more time with them and they outlive the server and database and disk migrations of their digital counterparts, I begin to wonder if there’s something to having a physical trace (a receipt, if you will) that’s worth, well, holding on to.

What traces of us will be left when machines have “learned” from our literature and seem capable of reading and writing and translating our languages with relative fluency? Paradoxically, the word for the digital data inputs fed into computers, the “corpus,” comes from the Latin word for “body.” I was thinking about the assumptions and impenetrable insides of machine learning with a study, PDF To Cognition, that trained a word-image based model of written text. Is a corpus strung together with characters isomorphic to ASCII character codes, or is a corpus compiled of typographic shape, reaching our eyes as image? And does the model ever transcend its inputs, or do its traces—us—forever lurk within?

InterLace (2012-)

The first use of InterLace was in collaboration with Eyal Sivan on his web documentary, Montage Interdit.

As a data-based film, continuity is not a linear narrative, but is rather achieved at “runtime” by the viewer, who can navigate by tag, source, or timeline, re-sorting to make new continuities and montage. The act of authorship, then, is in the creation of focus and metadata rather than narrative.

http://montageinterdit.net
http://interlace.videovortex9.net

Hyperopia Thing (2014)

Custom offline Wikipedia interface and thermal printer, embedded into an old encyclopedia. Offers wireless network with captive portal collaborative encyclopedia; tracks browsing and prints receipts of drifts.

Hyperopia is a device based on the transitive principle that “Reading is Writing” and that if whole is comprised of parts then the parts should stay connected to the whole: look at something closely enough and the truth of the universe is manifest in its every detail. The name “hyperopia” refers to a (supposed) defect of vision commonly known as farsightedness.

Online interface: http://hyperopia.meta4.org
Snapshots: http://rmozone.com/snapshots/2014/02/hyperopia

PDF To Cognition (2014)

Word-shape representation of text, trained on the Neuroscience literature. By taking PDF journal articles and splitting them into words—without breaking them down into letter—I saved each word as an image, and then built up a basis set of word-shapes. The distance between words is entirely a function of their visual similarity.

To encode a concept of difference and distance into these word-forms, I have used a statistical technique known as Principal Component Analysis (PCA), which is a mainstay of any modern compression and data analysis workflow. PCA takes data as input with any number of characteristics, and forms a basis set of abstract components in the same form as the input data, much like Gilbert Ryle’s Average Taxpayer. In this case, the components are statistically relevant word-forms that represent correlations between different parts of the shape, and PCA functions by enabling projection from any word-form into a combination of its basis set. Moving from a complex word form with, say, 1,500 pixels, and reducing it to a specific blending of 50 basis words is a dramatic dimensional reduction, and it is this reduction that allows for spatialization and comparison of words as shapes. If I take the liberty of calling this compression a “reading” of the word, then PCA is significant because its reading is bidirectional—to read implies an ability to write. Analysis and synthesis are deeply entwined.

http://readmes.numm.org/pdftocognition

Rendering, Pdf To Cognition. Words are digitally represented from an image-form basis (here trained on academic neuroscience papers) instead of character codes. The pca basis forms a continuous space of text-image. Screenshot, Pdf To Cognition. The training data (from critical theory reading lists) can reappear in unlikely arrangements when parsing new data.