An image is more than the sum of its pixels.

In the context of the traditional art school, we are taught to distrust the “effects” of photo editing software. Why use a digital simulation when you can work with the “true materials” of paint? What are the “true materials” of software, and hadn't we better use those when considering what painting means on a digital canvas? How can digital tools embrace the actual material of the algorithmic rather than merely simulating the analog?

The more we investigated “computer vision” techniques, however, the more we realized computer scientists are using the same techniques, and even employing an approach where techniques are composed almost as the cutups of a visual collage. No time to compute an actual Laplacian of gaussian? No problem, the textbooks offer, just gaussian blur at multiples of the square root of 2 and subtract the results to get a pretty good approximation.

A scanned image is more than a matrix of color values destined only to be displayed as pixels. Viewed through the lens of algorithms, an image is multiple layers of potential interpretations. Each layer tells different story, revealing some aspects, obscuring others. Algorithmic glitches are revealing: is this a letter, or the edge of a roof tile? What are the visual features of text when treated again as an image? The layers also speak to each other; how do SIFT features related to edges, and edges to the automatic detection of text in an OCR software?

http://sicv.activearchives.org

Contours: shows the contours detected on each image. In addition to tracing the edges, the algorithm connects the lines into a series of distinct segments represented by a different color.

Red, green, blue: What is significant color information? Contrary to human intuition, for a computer, a white image is an image saturated with red, blue and green. To find the images that look the most blueish, reddish or greenish, we counted only the color values that were superior to the others by a certain threshold.

Lexicality and Texture: These layers are produced using Tesseract, a software for Optical Character Recognition (OCR). An OCR program operates at different levels of granularity. It can detect lines, words, symbols.Lexicality shows the words detected in an image, while Texture shows the symbols detected. Texture is configured to be rather tolerant in its understanding of what a character is. It therefore tends to see characters in very unexpected places.Scale-invariant feature transform (SIFT): SIFT features are “interesting” points of an image that can be extracted to provide its “feature description.” This description can then be used to identify (parts of) an image even when rotated or changed.