Abandoning any attempt at continuity or chronological order, here's another project from a while back that really should have been posted here months ago! Synchresis is a DVD I curated for the Australian Network for Art and Technology late last year; it's a survey of Australian audiovisual practice, with an emphasis on "fused" or "synaesthetic" AV - a niche where the Australian scene is very strong. The DVD features work by Jean Poole, Andrew Gadow, Robin Fox, Gordon Monro, Wade Marynowsky, Abject Leader, Pix + Delire, Ian Andrews and Botborg.
Below is the essay,"Monsters and Maps," that I wrote for the project. It introduces the works on the disc but also touches on some more abstract thoughts about synchresis, synaesthesia, cross-modal perception, the map, and audiovisual practice.
This DVD isn't available retail-wise, unfortunately; it was distributed with ANAT's Filter magazine to their members. However I have a bundle of copies that I'm happy to send out, especially if you're interested in writing a review - let me know. Otherwise, read on for the essay; you can also grab the whole Filter issue as a pdf (3.2Mb).
Sound and video are, on the face of it, the most unremarkable ingredients in the new media arts. Compared to smart textiles, bio-art, data visualisation or locative media, sound and moving image seem almost archaic. Yet as the work documented in this edition of Filter shows, these materials are anything but obsolete. This body of work reflects a dynamic practice, and one that has emerged in recent years as a real Australian strength. It's not video art, though it has been known to fly that label as a flag of convenience; if anything its ties are closer to sound and music. What distinguishes it is that it focuses on, to recycle another archaic word, the audiovisual. The relation between sound and image, and especially, in digital culture, the set of possible relations between sound and image, is central here. The soundtrack, largely neglected in video art, is amplified; the moving image, taken for granted as "content" in an era of desktop video, is obliterated into sheer texture, or synthesised from scratch. Most importantly, sound and image fuse into new, tightly articulated wholes; they are cross-wired, cut or moulded into blocks of something like raw sensation.
As the writings here show, this practice crosses, and links, diverse scenes and cultures; it draws on video and sound art, experimental music and improv, computer music and experimental film, live video and the VJ scene. It also reflects in part a rich historical tradition of abstract animation and "visual music", as well as a specifically Australian history of audiovisual practice; Stephen Jones' video work with Severed Heads looms large. But at the core of this collection is a focus on the point of audiovisual fusion - the moment of synchresis - and as I hope to explain below, that point of fusion, and the relation it describes, has a particular significance in digital media culture.
On the screen, pixels shift, a dark void moves, opening and closing. The speakers vibrate; pulses and lumps of tone and pitch, flecks of air and noise; a voice. The void belongs to a face; and the lumps, stops and pulses from the speaker meet in time with the openings and closings of the mouth. We sense a correlation, a tight coincidence between these disparate events, and we recognise, involuntarily, a common cause that seems to link them. A newsreader enunciates the headlines: her image and her voice, separated at the point of capture, technically distinct and independent in the video signal, re-embodied by different means, are finally reunited in our perception.
This is synchresis, in its everyday form: lip-sync, the perceptual trick at the representational core of screen culture. Film sound theorist Michel Chion, who coined the term, defined synchresis as "the spontaneous and irresistible weld produced between a particular auditory phenomenon and visual phenomenon when they occur at the same time" (see Chion, Audio/Vision) Lip sync is its most ubiquitous form, but as Chion shows there are many others. The cinematic punch, what Chion calls the "emblematic synch point," shows how synchresis can be fabricated, how a sound effect guarantees an event that "we haven't had time to see" - but also an event that, in fact, never occurred.
The audiovisual cliche of the punch also hints at the wider potential of this audiovisual fusion. As Chion observes, synchresis works even with "images and sounds that strictly speaking have nothing to do with each other, forming monstrous yet inevitable and irresistable agglomerations in our perception." With the perceptual impetus of synchronisation, anything will glue to anything, and in the process new, sometimes monstrous wholes are constructed. Scratch video, Milli Vanilli, and more recently the YouTube-powered renaissance in video ventriloquism have all demonstrated this at one level. Yet these cutups rely on synchresis reinforcing the cinematic representation of a body, a source, a redundant, all-too-obvious shared cause. The audiovisual practice presented here offers some powerful alternatives.
Ian Andrews' Spectrum Slice (above) is constructed on a temporal grid - a field of tripwires that trigger Chion's monstrous agglomerations. The video track is chaotically intercut, but the edits are locked to the grid's pulse. The same pulse structures the audio track, as Andrews punches live radio samples into a loop. Tiny, random chunks of degraded video and radio noise weld into new units. But in this recombinant cutup, staggered loops and refrains also move against each other: synchretic welds are formed and reinforced only to be broken and shifted. We feel synchresis at work; and though it's constantly apparent that sound and image are disparate, the jolt of fusion keeps working. In Spectrum Slice Chion's monsters multiply, are cut into pieces, disintegrate and re-form; a preacher with a fragment of deep talk-radio voice; then a crowd with white noise static, then video static and a snatch of orchestral strings. Associative chains and correlations form between the fragments; between the cosmic, molecular jitter of audio and visual noise, and faces in a crowd; crowd/audience and preacher/presenter.
Similar chains and networks emerge in Abject Leader's Bloodless Landscape, again created by combinations of synchretic fusion and disjunction; and as in Spectrum Slice, the material signatures of the medium are everywhere. These works take apart the conventional anchor of synchresis - the represented body - but instead create an embodiment of their own media substrates. Sound and image operate in indexical relays that ground media forms in a material continuum: dirt, bubbles, scratches, static, celluloid, light; synchresis becomes haptic, or tactile.
Peter Newman's work intensifies this material quality while dissolving the last fragments of indexical content into incandescent plasma (. The cinematic constituent, the edited shot, is likewise obliterated into long textural dissolves; in a sort of inverse of Spectrum Slice, which trips up synchresis by multiplying the edit, Newman's audiovisual textures tighten their relationship through immersive duration. A flickering, rhythmic synch point binds Rosebud (above) together from the outset; but the synchretic monster here is a single entity; it intensifies as it gradually reveals (or destroys) itself, a tactile or palpable presence.
The monstrous body makes a representational comeback in Wade Marynowsky's work. Like Newman, Marynowsksy uses digital processes to manipulate sound and image into abstraction; but where Newman's video seems to have been worn smooth over years, Marynowsky's retains a hard, self-referential edge. In a post-glitch performance parody, The_Geek_From_Swampy_Creek - another monster - sways gently at his Powerbook, calling up disintegrating images of his wetland home. Unlike most of his peers, Marynowsky puts his own body up as the synchretic anchor, the common cause for sound and image. Jean Poole's floating monkey in Cappadocia Skies is another audiovisual self-construction; author as both witness and performer, using the sure-fire synchresis of music video to re-frame experience and, again, landscape. Cappadocia Skies is also an emblem for a nomadic laptop-video culture; where images and sound are gathered, networked, recombined, processed, filtered, and beamed back into the environment. A kind of distributed, recursive, synchresis machine.
In most cases, synchresis is constructed; sound and image composed and arranged to form these spontaneous welds. However in one important strain of Australian AV, synchresis becomes automatic, inherent; and the synch point, the synchretic moment, is extended and intensified to become all-encompassing. Robin Fox's audiovisual practice began as he connected his laptop audio rig to an old analog oscilloscope. The scope renders audio signal as dynamic image, a phosphorescent trace generated automatically, according to a technically defined translation. The result is an audiovisual instrument; Fox's audio software becomes a sound-and-image-synthesiser. In the resulting work the synchretic weld acquires new power; image and sound are tightly and immediately coupled. The sense that Newman conveys in Rosebud, of an autonomous, coherent AV entity, is extended. There's a sense of something like revelation, as sound-forms unfold: Fox's Immaculate Infection (above) continues to investigate an audiovisual space which is totally self consistent, yet continually surprising.
Andrew Gadow's work approaches the same relation from the other side; his practice is based on rendering synthesised video signals as audio. Using a Fairlight CVI - a pioneering (but now archaic) Australian instrument - Gadow generates visual patterns and forms with an automatic double in modulated buzz-saw sound. Again there is a direct and immediate fusion, an intense coherence behind the abrasive surface of the work. In Gadow's more recent work he has introduced an analog audio synthesiser, creating loops in which video is generated by the CVI, filtered by the audio synth, and returned to video (as in Techne / Auxons, above). Botborg complexify these loops, wiring a video mixer into an unstable, psychotropic feedback system where the elements become indistinguishable; we see-and-hear video-and-sound in varying mixtures and causal relations (below: Principle 4, from their DVD Principles of Photosonicneurokinaesthography).
Synaesthesia offers a promising analogy for these works, as well as an alternative model of audiovisual relations. In synaesthetic perception one modality is automatically and involuntarily translated into another. Synaesthetes may perceive letters, numbers or days of the week as inherently coloured, associate shapes with smells or tastes, or see coloured forms in response to music. The analog crossovers of Fox and Gadow seem to echo the neural cross-wiring thought to cause synaesthesia; in both cases there's an immediate and automatic connection between sound and image. Is this some kind of machine synaesthesia? If so, can we learn it, internalise it and reconfigure our own sensorium? In the fictotheory of Arkady Botborger crosslinked sound and image promise to open "new neural pathways" that expands perception by overwhelming or bypassing our "linguistic centres."
Artificial or induced synaesthesia is possible, but not in a few minutes of AV: it would mean living with, and adapting to, rewired or augmented perception in the long term. An alternative model for these fused audiovisuals is both more realistic and more abstract. Synchresis is perceptually founded on the process of cross-modal binding, where percepts in different modes are "bound" into recognition as a single entity. Binding feels good - some neuroscientists theorise that it activates our limbic pleasure circuits (see Ramachandran and Hirstein, "The Science of Art" [pdf]). Binding is marked subjectively by a flash of recognition, an "a-ha" moment or a sense of noesis, or revelation. Perhaps the affect of tightly fused audiovisual experiences is related to the perceptual process of binding?
What some suggest as an adaptive rationale for cross-modal binding - and the reason it feels good - is its functional role in interpreting our environment. As the newsreader shows us every night, we link sound and image into a functional, coherent model of the environment; we recognise the common cause of these correlated perceptions. Which begs the question, what is that common cause, in the tightly correlated audiovisuals of artists such as Fox and Gadow? Unlike constructed forms of synchresis, in these works a single structure really does cause both sound and image: the signal, the abstract pattern of voltage fluctuations, is what we see-and-hear. The artists literally feel out the domain of the signal; but also, over time, a form more abstract still. Moment by moment we sense the signal's dual manifestations in sound and image; over time, we come to sense the pattern of correspondences between sound and image. This is the map, the relation between the signal domains, which for Fox and Gadow at least is a static, technical artefact.
This map is a space of transformation; it describes how one domain connects with another. Though conceptually abstract, it is completely ubiquitous in contemporary media. Digital media are founded on transformation, for the digital is imperceptible in itself; "digital imaging," "digital video" and "digital sound" are in a sense oxymorons, for what we sense is not the digital, but a mapping of the digital into one or more domains of sensation. These maps are essentially arbitrary, open to change; we can take image data and re-map it into sound, or text; map text into image, or three-dimensional form. If the digital has no inherent manifestation, no necessary relation to our perception, then the question of the aesthetics, or affect, of digital media, is a question of the map. In this context Gadow and Fox do something remarkable, and entirely contemporary, in their cross-wirings of image and sound; they give us a sense, literally, of the map.
Maps may be ready-made, technical artefacts, as they are for Fox and Gadow; or like Chion's synchretic monsters they can be invented, arbitrary, anything-at-all. Gordon Monro's Triangular Vibrations unfolds from a computational model that generates both sound and image, and so invents a distinctive correspondence between sound and image. Here too, cross-modal perceptual cues direct us to the "common cause" here - that computational model. That model - another abstract but ubiquitous figure in digital media - becomes literally sensible; as the system slowly changes it reveals its own logics, its internal patterns of conistency, its boundaries and centers.
Pix and Delire's Fijuu2 navigates a similar space - a computational map between sound and image - but here its exploration is turned over to the user. With a pallette of audiovisual forms and filters, gestural controls and a sequencer, Fijuu2 demonstrates a proliferation of digital maps, some of the range of possible audiovisual "anythings" this space contains. More monsters, and again they go for the throat, so to speak; the gestural distortions in Fijuu2 make a visceral choreography out of the abstract malleability of the digital. Drawing on the technical resources of open source game culture, this work reconfigures the default narratives of hardware-accelerated 3d into far more promising circuits of action and sensation.
Synchresis is, as Chion says, ubiquitous, commonplace. It's one of the mainstays of Western media culture, and we take its familiar manifestations entirely for granted. A mouth, a sound, a body, a figure, a punch, a story. Our sensoria are well-wired for finding these correlations, identifying those coherences in the incoming flux of reality that tell us, something's out there. But we're starved, at the same time, of those perceptual moments that really matter, marked by an unfamiliar coherence, an unknown cause: what's that? McLuhan told us years ago that art is perceptual training for the future; in this case, it's directing us towards powerful abstractions that define the digital present. Synchresis is ubiquitous and commonplace, yet when we focus, as this work demands, on that kernel of media and perception, it opens out in ways that are anything but ordinary.
Tuesday, June 24, 2008
Synchresis - Australian Audiovisuals
Sunday, November 18, 2007
Array Art (More Multiplicity)
Since that earlier post I've begun to notice another form of multiplicity. I'm seeing arrays everywhere, lately. Grids or articulated fields of points; substrates for transitory patterns and forms in light or sound. In United Visual Artists' recent Battles video, the band plays in a triangular grid of vertical LED strips. Patterns traverse the array, bathing the surrounding rocky landscape in flickering, articulated light. Of course UVA have got form with arrays; their 2006 Volume installation uses a similar configuration - and in a way there's a continuum between these grids and the more conventional (but equally effective) LED wall they used for the Massive Attack tour in 2003.
Edwin van de Heide's Pneumatic Sound Field - blogged earlier - echoes UVA's light arrays, but uses flickers of high-pressure air. There's another parallel here, in that the elements in the array - the "emitters" - are simple, physical things; points of energy. Other arrays I've noticed recently include Robert Henke and Christopher Bauder's Atom; a grid of LED-lit helium balloons that also move vertically under remote control. See also Artificiel's Condemned Bulbes (2003); and then it's a short hop to Rafael Lozano-Hemmer's recent Pulse Room at the Mexican pavilion in Venice. We could add a "mirror" sub-genre, with Aleph by Bengt Sjölén and Adam Somlai-Fischer and Daniel Lazin's beautiful articulated arrays. I'm sure there are many more.
UVA's body of work illustrates one of the reasons I'm interested in these arrays; they represent a kind of expansion, or explosion, of the screen. In part these arrays mimic everyone's favourite luminous grid, the digital display; but they literally take it apart; they expand it in size but also string it out through physical space. Instead of a vertical image (think cinema, painting, architecture, etc) we get an often horizontal array, a field to walk through. These arrays echo the display, especially its logics of modularity and generality - a logic shared by computational culture more broadly, where grids of uniform elements create wide spaces of potential. But these are not simply low-res displays. The visual unity of the screen is based on the merged imperceptibility of the pixel elements; by contrast these works expose those elements and emphasise their interrelations, making them available as kinesthetic as much as visual experience.
In "Notes on Sculpture 4: Beyond Objects" (1969) Robert Morris discusses the "anti-formal" move in sculpture of the time, where discrete geometric objects began to be replaced by wide, horizontal fields of undifferentiated stuff. He draws inspiration from Anton Ehrenzweig, a gestalt psychologist, and his notion of "synchretic" or "scanning" perception. This is an unconscious or "low level" mode of vision that Ehrenzweig claims provides access to richly detailed information in the perceptual field. To put it in modern (ie technological) perceptual terms, scanning occurs before recognition or gestalt formation, in the perceptual pipeline. As such, argues Ehrenzweig, scanning can easily accommodate "open structures" - complexity, contingency, chaos, the unformed or uncertain. Morris argues that the lateral, post-formal "fields" in sculpture of the time, build this mode of perception into their very structure; and ties this to a larger, McLuhan-like argument that "art itself is an activity of change ... of the willingness for confusion even in the service of discovering new perceptual modes."
Leaving the modernist declarations aside, there are some interesting links between this horizontality and the perceptual mode it demands, and the arrays of the works discussed here. Works such as Morris' 1968 Untitled (Threadwaste) (above - source) emphasise unformed materiality; in works like UVA's arrays or Lozano Hemmer's Pulse Room the material and the immaterial, or informational, play against each other in a very contemporary way. These explicit, low-res arrays reveal themselves as material structures (unlike the screen), but also as material substrates for dynamic, informational patterns and forms. The role of the light source is important here; in Condemned Bulbes and Pulse Room, archaic light technology is used for its material and sonic byproducts; the globe here isn't a pixel, it's a physical device, a buzzing, glowing object, a manifestation of electricity. Yet it's also a pixel, an abstract unit in a digital array; the two are complementary, co-constituents, rather than opposites.
Ehrenzweig's "scanning" perception also seems relevant all over again; it's exactly the mode of experience that a lot of data visualisation demands, and linked to what I've described as the "artist's squint" in data art. In a culture of digital multiplicity - where, as in these arrays, we are literally surrounded by digital grids - the gestalt or fixed image is impossible; "scanning" promises access to the pre-conscious information in these articulated masses. Sometimes these works offer a reassuring, unified image of the grid, where it's in sync, under control, centrally choreographed; but other times, especially in Lozano-Hemmer's work, it's a more complex, chaotic field.
What about the relation between this form of multiplicity and the generative variety? As well as an aesthetic interest in sheer "moreness" there's a conceptual connection. In a way these arrays are the inverse of generative multiplicities that sample wide spaces of potential. These grids partly act to manifest that space of potential explicitly; this is all there is, 64 balloons (as in Atom, above) or a few dozen LED strips. But what they reveal is how that explicit grid contains a far vaster, implicit space of potential, an unthinkable mass of relations, patterns and movements. So though the manifestation is very different they suggest the same dynamic - of the actual pointing to the virtual
Monday, October 29, 2007
More is More: Multiplicity and Generative Art
Douglas Edric Stanley wrote a nice post recently on complexity and gestalts in code and generative graphics. In it he wonders about "all those lovely spindly lines we see populating so many Processing sketches, and how they relate with code stuctures." I've been wondering about the same thing for a while, and Stanley's post has prodded me to chase up a few of these ideas.
Stanley makes some astute observations about the aesthetic economics of generative art; the fact that it costs almost exactly the same, for the programmer, to draw one, a hundred or a million lines. Stanley pursues the machinic-perceptual implications - how simple code structures contribute to the formation of gestalts; but he only hints at what seems like a more interesting question, of how these generative aesthetics relate to their cultural environment: "all of these questions of abstraction and gestalt are in fact questions about our relationship to complexity and the role algorithmic machines (will inevitably) play in negotating our increasing complexity malaise."
I actually don't think complexity is the right concept here. For me complexity refers to causal relations that are networked, looped and intermeshed (as in "complex systems"). These "lovely spindly lines", and Stanley's gestalt-clouds, show us multiplicity but not (necessarily) complexity. Simple, linear processes are just as good at creating multiplicity. There's certainly a relationship here - complex systems often produce multiplicitous forms and structures; and causal complexities embedded in "real" datasets seem to be a reliable source of rich multiplicities - but complexity and multiplicity aren't the same thing. For the moment I want to focus on the aesthetics of multiplicity.
Multiplicity is the uber-motif of current digital generative art - especially the scene around Processing. Look through the Flickr Processing pool and try to find an image that isn't some kind of swarm, cloud, cluster, bunch, array or aggregate (this one is by illogico). The fact that it's easy to do is a partial and not-very-interesting explanation; to go one step further, it's easy and it feels good. Multiplicity offers a certain kind of aesthetic pleasure. There's probably a neuro-aesthetics of multiplicity, if you're into that, which would show how and where it feels good. Ramachandran and Hirstein have suggested that perceptual "binding" - our tendency to join perceptual elements into coherent wholes - is wired into our limbic system, because it's an ecologically useful thing to do. Finding coherence in complex perceptual fields just feels good. The perceptual fields in generative art are almost always playing at the edges of coherence, buzzing between swarm and gestalt - just the "sweet spot" that Ramachandran and Hirstein propose for art in general.
I don't find this explanation very satisfying either, because it doesn't seem to tell us anything much about the processes involved - it's a "just because," and a fairly deterministic one. Another way in is to think formally about the varieties of multiplicity in generative art. I rediscovered Jared Tarbell's wonderful Invader Fractal (below) in the Reas/Fry Processing book recently. It shows a kind of multiplicity that's the same but different to the "spindly lines" aesthetic. Each invader is the product of a simple algorithm; the whole mass is a visualisation of a space of potential - a sample (but not an exhaustive display) of the space of all-possible-25-pixel- invaders. Multiplicity here is a way to get a perceptual grasp on something quite abstract - that space of possibility. We get a visual "feel" for that space, but also a sense of its vastness, a sense of what lies beyond the visualisation. John F. Simon's Every Icon points in the same direction; towards the vastness of even a highly constrained space of possibility (32x32 1-bit pixels).
Perhaps current aesthetics of multiplicity are actually doing something similar. The technical differences are fairly minor; basically a switch in spatial organisation from array to overlay; a compression of instances into a single picture plane. The shortest (and my personal favourite) path to multiplicity in Processing is aggregation: turn off background() and let the sketch redraw. Reduce the opacity of the drawing for an accumulating visualisation of the space of possibility that your sketch is traversing. Multiplicity here isn't an effect or aesthetic for its own sake; it's intrinsically linked to one of the defining qualities of generative systems - their creation of large but distinctive spaces of potential. Multiplicity is again a way to literally sense that space; but also, since it almost never exhausts or saturates that space, it points to an open, ongoing multiplicity; it actualises a subset of a virtual multiplicity, and shows us (as in Every Icon) how traversing that space is only a question of specifics and contingencies. Multiplicity says "and so on"; an actual gesture towards the virtual.
Multiplicity refers to the specific space of potential in any single system, by actualising a subset of points within it; but it also metonymically refers to an even wider space of potential, which is the one that all computational generative art - and in fact all digital culture - traverses. Because of course any system can be tweaked and changed, no chunk of code is immutable or absolute, the machines of the Processing pool are ever-changing things that collectively sample the space of all possible (generative) computation. Just as it refers directly to the space of potential of its own (local) system, generative multiplicity alludes to the unthinkable space-of-spaces that contains that system - a space the system gradually traverses with every change in its code.
This, for me, explains the aesthetic and cultural charge that multiplicity carries. It's a gesture towards an abstract, unthinkable figure; an aesthetics of the virtual, in the Bergson / Deleuze sense of the word. What's more this particular form of virtuality, or possibility - the one accessable through code and computation - is at the core of digital culture and our contemporary situation. Generative multiplicity is, quite literally, a visualisation of that figure.
