Proto-Computing - an Interview with Ralf Baecker

At CODE2012 I presented a paper on "programmable matter" and the proto-computational work of Ralf Baecker and Martin Howse - part of a long-running project on digital materiality. My sources included interviews with the artists, which I will be publishing here. Ralf Baecker's 2009 The Conversation is a complex physical network, woven from solenoids - electro-mechanical "bits" or binary switches. It was one of the works that started me thinking about this notion of the proto-computational - where artists seem to be stripping digital computing down to its raw materials, only to rebuild it as something weirder. Irrational Computing (2012) - which crafts a "computer" more like a modular synth made from crystals and wires - takes this approach further. Here Baecker begins by responding to this notion of proto-computing.

MW: In your work, especially Irrational Computing, we seem to see some of the primal, material elements of digital computing. But this "proto" computing is also quite unfamiliar - it is chaotic, complex and emergent, we can't control or "program" it, and it is hard to identify familiar elements such as memory vs processor. So it seems that your work is not only deconstructing computing - revealing its components - but also reconstructing it in a strange new form. Would you agree?

RB: It took me a long time to adopt the term "proto-computing". I don't mean proto in a historical or chronological sense; it is more about its state of development. I imagine a device that refers to the raw material dimension of our everyday digital machinery. Something that suddenly appears due to the interaction of matter. What I had in mind was for instance the natural nuclear fission reactor in Oklo, Gabon that was discovered in 1972. A conglomerate of minerals in a rock formation formed the conditions for a functioning nuclear reactor, all by chance. 

Computation is a cultural and not a natural phenomenon; it includes several hundred years of knowledge and cultural technics, these days all compressed into a microscopic form (the CPU). In the 18th century the mechanical tradition of automata and symbolic/mathematical thinking merged into the first calculating and astronomical devices. Also the combinatoric/hermeneutic tradition (e.g. Athanasius Kircher and Ramon Llull) is very influential to me. These automatons/concepts were philosophical and epistemological. They were dialogic devices that let us think further, much against our current utilitarian use of technology. Generative utopia.

Schematic of Irrational Computing courtesy of the artist - click for PDF

MW: Your work stages a fusion of sound, light and material. In Irrational Computing for example we both see and hear the activity of the crystals in the SiC module. Similarly in The Conversation, the solenoids act as both mechanical / symbolic components and sound generators. So there is a strong sense of the unity of the audible and the visual - their shared material origins. (This is unlike conventional audiovisual media for example where the relation between sound and image is highly constructed). It seems that there is a sense of a kind of material continuum or spectrum here, binding electricity, light, sound, and matter together?

RB: My first contact with art or media art came through net art, software art and generative art. I was totally fascinated by it. I started programming generative systems for installations and audiovisual performances. I like a lot of the early screen based computer graphics/animation stuff. The pure reduction to wireframes, simple geometric shapes. I had the feeling that in this case concept and representation almost touch each other. But I got lost working with universial machines (Turing machines). With Rechnender Raum I started to do some kind of subjective reappropriation of the digital. So I started to build my very own non-universal devices. Rechnender Raum could also be read as a kinetic interpretation of a cellular automaton algorithm. Even if the Turing machine is a theoretical machine it feels very plastic to me. It a metaphorical machine that shows the conceptual relation of space and time. Computers are basically transposers between space and time, even without seeing the actual outcome of a simulation. I like to expose the hidden structures. They are more appealing to me than the image on the screen.

MW: There is a theme of complex but insular networks in your work. In The Conversation this is very clear - a network of internal relationships, seeking a dynamic equilibrium. Similarly in Irrational Computing, modules like the phase locked loop have this insular complexity. Can you discuss this a little bit? This tendency reminds me of notions of self-referentiality, for example in the writing of Hofstadter, where recursion and self-reference are both logical paradoxes (as in Godel's theorem) and key attributes of consciousness. Your introverted networks have a strong generative character - where complex dynamics emerge from a tightly constrained set of elements and relationships.

RB: Sure, I'm fascinated by this kind of emergent processes, and how they appear on different scales. But I find it always difficult to use the attribute consciousness. I think these kind of chaotic attractors have a beauty on their own. Regardless how closed these systems look, they are always influenced by its environment. The perfect example for me is the flame of a candle. A very dynamic complex process communicating with its environment, that generates the dynamics.

MW: You describe The Conversation as "pataphysical", and mention the "mystic" and "magic" aspects of Irrational Computing. Can you say some more about this a aspect of your work? Is there a sort of romantic or poetic idea here, about what is beyond the rational, or is this about a more systematic alternative to how we understand the world?

RB: Yes, it refers to an other kind of thinking. A thinking that is anti "cause and reaction". A thinking of hidden relations, connections and uncertainty. I like Claude Lévi-Strauss' term "The Savage Mind".

Read More...

Summary only...

Transduction, Transmateriality, and Expanded Computing

In common usage a transducer is a device that converts one kind of energy to another. Wikipedia lists a fantastic variety of transducers, mapping out links between thermal, electrical, magnetic, electrochemical, kinetic, optical and acoustic energy. In this form transducers are everywhere: a light bulb transduces electrical energy into visible light (and some heat). A loudspeaker transduces fluctuations in voltage into physical vibrations that we perceive as sound.

In analog media, transduction is overt (put the needle on the record...). But digital media are riddled with it too. Inputs and output devices all contain transducers: the keyboard transduces motion into voltage; the screen transforms voltage into light; the hard drive mediates between voltage and electromagnetic fields. A printer takes in patterns of voltage and emits patterns of ink on a page. Strictly transduction only refers to transformations between different energy types; here I want to extend it to talk about all the propagating matter and energy within something like a computer, as well as those between that system and the rest of the world. From this transmaterial perspective a computer is a cluster of linked mechanisms and substrates; a machine for shifting patterns through time and space.


If this sounds unfamiliar, it's only by historical accident. Mechanical computers, where these patterns are physically perceptible, predate electrical (let alone digital) ones, by centuries (above: a replica of Konrad Zuse's Z1, a mechanical computer from 1936. Image by rreis). Materially, our current computers are more or less black box systems. Their transductions come as a sort of preconfigured bundle or network, a set of familiar relations constructed again by mixtures of hard- and software, protocols, standards: generalising frameworks. I press a key, a letter appears; this is all I need to know. Click "OK". No user-serviceable parts inside.

Except that currently, across the media arts and a whole slew of other fields, the computer is undergoing a rich and productive decomposition. It's composting, to borrow a Sterlingism. This goes under all kind of different names: hardware hacking, device art, homebrew electronics, physical computing. Such practices mount a direct assault on the computer as a material black box, literally and figuratively cracking it open, hooking it up to new inputs and outputs, extending and expanding its connections with the environment. Microcontrollers like the Arduino present us with nothing but a row of bare I/O pins. Finally we can tackle the question of what should go in, and what should come out: of transduction. A whole generation of artists, designers, nerds and tinkerers are taking up soldering irons and doing just that. Below: the Spoke-o-dometer from Rory Hyde and Scott Mitchell's Open Source Urbanism project.


One side-effect of this decomposition of computing is that the ontological status of the digital starts to break down with it. As Kirschenbaum shows brilliantly, the digital is just the analog operating within certain tolerances or threshholds. Thomas Traxler's The Idea of a Tree (below) is a solar-powered system that fabricates objects from epoxy, dye and string, by turning a spindle. Solar energy generates electrical energy, which drives the motor, which draws the string through the dye and onto the spindle: a chain of analog transductions produce an object that manifests specific changes in its local environment. The work is a beautiful demonstration that variability doesn't have to be worked up with generative code: if the system is open to it, it's already there in the flux of the material field.

This is not to dismiss computing, only to recast it: an incredibly dynamic, pliable set of techniques for manipulating the material environment. Paradoxically the very generalities of computing - the abstractions and protocols that insulate it from local, material conditions - make it a powerful tool for transduction, that is, the propagation of specificities. Usman Haque's Pachube is a generalised infrastructure, a set of protocols and standards that rest in turn on wider standards like XML, and which assume a whole stack of functional layers: IP, HTTP, and so on. All in order to propagate material patterns and flows from here to there: this is an architecture of transduction whose utopian aim is to "patch the planet" into a translocal ecology of linked environments.

Digital fabrication is part of the same shift: an expansion and extension of the computer's range of material transductions. Digital pattern, to lasercutter instructions, to physical form. Fabbing shows how material matters. It's unsurprising that a piece of laser-cut ply is aesthetically different to a luminous pattern of pixels; more interesting is the way computation reaches out into the substrate's material properties, and the range of potential applications and domains it opens up. Fabbing has often presented itself with a narrative of materialisation, making the virtual real, translating bits into atoms - Generator.x 2.0 was subtitled "Beyond the Screen." Not so: because of course, the "virtual" never was, and the screen is material too. Fabbing does get us beyond the screen, but only because its processes and materials have different properties, different specificities, and they hook us up to new contexts, as well as new sensations. (Below: Andreas Nicolas Fischer & Benjamin Maus: Reflection - from 5 Days Off: Frozen)


Transduction suggests a way to link practices like physical computing, fabrication, networked environments, and many more. Data visualisation - in the broadest sense, from poetic to fuctionalist - is about creating customised transductions, sourcing new inputs and/or manifesting new outputs (even if they don't reach "beyond the screen"). We could add tangible interfaces, augmented reality, and locative systems. What does all this amount to? In 1970 Gene Youngblood observed a similar moment as the dominant cultural form diversified into a networked, participatory, interdisciplinary field of practices. He called it expanded cinema. So perhaps we can call this expanded computing: digital media and computation as material flows, turned outwards, transducing anything to anything else.

Read More...

Summary only...

There is no Software - Kittler and Evolvable Hardware

I'm slowly developing this notion of transmateriality; in this post, some media theory and a nice example from computer science. In the next I'll try to connect all this with some current art work, and back to the notion of the transmaterial.

Thanks to a prod from my friend Brogan Bunt, I've been reading Friedrich Kittler, a literary and media theorist who has made some striking forays into computational media. In a paper from 1995 he grapples, like Kirschenbaum, with the grounding of computation in matter, distilling this position to a wonderful aphorism: "there is no software." Kittler begins by announcing the end of writing; that texts "do not exist anymore in perceivable time and space" but have been miniaturised to the scale of integrated circuits. This miniaturisation, in which writing escapes the bounds of human perception, is facilitated by Turing's core principle of computing, which sets out minimal conditions for computation and proves its independence from hardware - the ability for any number of different physical machines to implement a universal computer. This principle, Kittler says, "has had the effect of duplicating the implosion of hardware by an explosion of software." "Ordinary language" is overtaken by a new hierarchy of programming languages, layers that reach from the command line down to assembler and the very protocols embedded in the silicon itself.

Kittler plays out this "descent" in which each layer depends on the one below it; the word processor depends on DOS, which in turn rests on the hardwired BIOS. Ultimately "[a]ll code operations, despite their metaphoric faculties such as "call" or "return", come down to absolutely local string manipulations and that is, I am afraid, to signifiers of voltage differences." In other words, although they resemble "ordinary" language, programming languages return us to the imperceptible, inaccessable, too-fast and too-small world of the microprocessor. So, "there is no software at all," except as defined by the "environment of everyday language" that surrounds computation.

In a paranoid turn Kittler analyses the tendency of computer culture to "systematically obscure hardware by software, electronic signifiers by interfaces between formal and everyday languages." In the GUI, but also in high level languages such as C (let alone Java), the physical machine is increasingly concealed from its users, and its programmers, in the name of functionality and "friendliness."


Finally Kittler considers the limits and thresholds of "programmable matter"; he points out that current computing hardware relies on the isolation of discrete elements from each other and thus a limit to connectivity. This contrasts with the "maximal connectivity" of the physical systems - "waves, weather and wars" outside the computer. The current approach to computing hardware is essentially more of the same: more transistors, more elements, smaller circuits with better isolation. Kittler instead suggests that the only way to "keep up" with the physical complexity of the world is to match it with "nonprogrammable systems" made of "sheer hardware": "a physical device working amidst physical devices and subjected to the same bounded resources." In such devices once again "software as an ever-feasible abstraction would not exist any more."

Interestingly Kittler ends up close to where my earlier work on art and artificial life (published as Metacreation) came to rest. In considering the desire for emergence in a-life art, I wondered about the constraints imposed by the physical substrate - the "coarse, rigid grammar" of digital electronics. My favourite demonstration of what lies beyond this grammar is the evolvable hardware work of Adrian Thompson, in which circuit designs for programmable chips (field gate programmable arrays - as in the image above) are evolved using a genetic algorithm and tested in hardware for their performance in a particular task. It's perhaps not surprising that successful circuits were evolved over many thousands of generations; but the fun part is that when analysed, these circuits were completely unlike any human-designed computing machine. In Kittler's words they were "sheer hardware," treating the chip as a "maximally connective" physical substrate rather than an abstracted set of discrete elements. Some chips drew on external influences, such as electromagnetic radiation, to achieve their evolved ends; so the chip is not formally isolated (but to quote Kittler again) "a physical device working amidst physical devices." Not only that, they were often "tuned" to the physical specificities of a single chip, despite the FPGAs being notionally identical - the exact opposite to the hardware-indpendence of the Turing Machine.

Read More...

Summary only...

ACMC07 - Warren Burt and Sebastian Tomczak

This year's Australasian Computer Music Conference is here in Canberra, hosted by Alistair Riddell at the CNMA. Though ironically I could only get there for the first day, here are a couple of choice morsels.

The opening keynote by Warren Burt took on the conference theme - "trans" - and delivered a dense core sample of transdisciplinarity in music, from the ancient Greeks to the West Coast musical avant-garde of the 70s, through to the present. Many of Burt's projects look fresh all over again - he's been doing audiovisual synthesis, sonification of complex systems, and bio-collaboration since back in the day. He also made some great points about the role of the avant-garde in transforming cultural systems, rather than just "playing new music in the same old venue" (a mistake he attributed to Richard Wagner and the Sex Pistols, among others). During the 70s and 80s Burt was involved with the Clifton Hill Community Music Centre, an experiment in new social contexts for music, affordable technology and anarchic DIY.

Meanwhile back in the present, Alex Thorogood presented some nifty hardware hacks splicing an Arduino board with the innards of a cheap MP3 player, for his Chatter and Listening sound sculpture project. Hardware of the day though was Sebastian Tomczak's amazing Toriton Plus, a homebrew controller based on lasers, photocells and water. I'll spare you a lengthy description, except to say that it's much more beautiful in live performance - here's the video.

Read More...

Summary only...