Given my rantings about digital materiality and transduction, fabrication is a fairly obvious topic of interest. I posted earlier about an experiment with laser-cut generative forms and Ponoko - more recently I've been playing with 3d-printing via Shapeways, as well as trying out data-driven (or "transduced") forms. This post covers technical documentation as well as some more abstract reflections on this project - creating a wearable data-object, based on 365 days of local (Canberra) weather data.
Shapeways has good documentation on how to generate models using 3d-modelling software. Here I'll focus more on creating models using code-based approaches, and Processing specifically. The first challenge is simply building a 3d mesh. I began with this code from Marius Watz, which introduces a useful process: first, we create a set of 3d points which define the form; then we draw those points using beginShape() and vertex().
The radial form of the Weather Bracelet model shows how this works. The form consists of a single house-shaped slice, where the shape of each slice is based on temperature data from a single day. The width is static, the height of the peak is mapped to the daily maximum, and the height of the shoulder (or "eave") is mapped to the daily minimum. To create the radial form, we simply make one slice per day of data, rotating each slice around a central point. As the diagram below shows, this gets us a ring of slices, but not a 3d-printable form. As in Watz's sketch, I store each of the vertices in the mesh an array - in this case I use an array of PVectors, since each PVector conveniently stores x,y and z coordinates. The array has 365 rows (one per day, for each slice) and 5 columns (one for each point in the slice). To make a 3d surface, we just work our way through the array, using beginShape(QUADS) to draw rectangular faces between the corresponding points on each of the slices.
To save the geometry, I used Guillame laBelle's wonderful SuperCad library to write an .obj file. I then opened this in MeshLab, another excellent open source tool for mesh cleaning and analysis. Because of the way we draw the mesh, it contains lots of duplicate vertex information; in MeshLab we can easily remove duplicate vertices and cut the file size by 50%. MeshLab is also great for showing things like problems with normals - faces that are oriented the wrong way. When generating a mesh with Processing, the order in which vertices are drawn determines which way the face is ... er, facing... according to the right hand rule. Curl the fingers of your right hand, and stick up your thumb: if you order the vertices in the direction that your fingers are curling, the face normal will follow the direction of your thumb. Although Processing has a normal() function that is supposed to set the face normal, it doesn't seem to work with exported geometry. Anyhow, the right hand rule works, though it is guaranteed to make you look like a fool as you contort your arm to debug your mesh-building code.
The next step in this process was integrating rainfall into the form. I experimented with presenting rainfall day-by-day, but the results were difficult to read; I eventually decided to use negative spaces - holes - to present rainfall aggregated into weeks. Because Shapeways charges by printed volume, this had the added attraction of making the model cheaper to print! The process here was to first generate the holes in Processing as cylindrical forms. Unlike the base mesh, each data point (cylinder) is a separate, simple form: this meant I could take a simpler approach to drawing the geometry. I wrote a function that would just generate a single cylinder, then using rotate() and scale() transformations made instances of that cylinder at the appropriate spots. Because I wanted the volume of each cylinder to map to rainfall, the radius of each cylinder is proportional to the square root of the aggregated weekly rainfall. As you can see in the grab below, the base mesh and the cylinders are drawn separately, but overlayed; they were also saved out as separate .obj files. The final step in the process was to bring both cleaned-up .obj files into Blender (more open source goodness) and run a Boolean operation to literally subtract the cylinders from the mesh. This took a while - Blender was completely unresponsive for a good few minutes - but worked flawlessly.
Finally, after checking the dimensions, exporting an STL file from MeshLab, and uploading to Shapeways, the waiting; then, the printed form. I ordered two prints, one in Shapeways' White, Strong and Flexible material, and the other in Transparent Detail. You can clearly see the difference between the materials in these photos. The very small holes tested the printing process in both materials; in the SWF print the smallest holes are completely closed; in the TD material they are open, but sometimes gummed up with residue from the printing process (which comes out readily enough). Overall I think the TD print is much more successful - I like the detail and the translucency of the material, as well as the cross-hatched "grain" that the printing process generates.
So, a year of weather data, on your wrist - as a proof of concept the object works, but as a wearable and as a data-form it needs some refinement. As a bracelet it's just functional - the sizing is about right, but the sharp corners of the profile are scratchy against the skin. As a data-form, it could do with some simple reference points to make the data more readable - I'm thinking of small tick-marks on the inner edge to indicate months, and perhaps some embossed text indicating the year and location. More post-processing work in Blender, I think.
Another line of development is to do versions with other datasets - and hey, if you'd like one for your city, get in touch. But that also raises some tricky questions of scaling and comparability. The data scaling in this form has been adjusted for this dataset; with another year's data, the same scaling might break the form - rain holes might eat into the temperature peaks, or overlap each other, for example. A single one-size-fits-all scaling would allow comparisons between datasets, but might make for less satisfying individual objects - and, finding that scaling requires more research.
What has been most enjoyable with this project, though, is the immediate reaction the object evokes in people. The significance and scale of the data it embodies, and its scale, seem to give it a sense of value - even preciousness - that has nothing to do with the cost of its production or the human effort involved. The bracelet makes weather data tangible, but also invites an intimate, tactile familiarity. People interpret the form with their fingers, recalling as they do the wet Spring, or that cold snap after the extreme heat of February; it mediates between memory and experience, and between public and private - weather data becomes a sort of shared platform on which the personal is overlayed. The form also shows how the generalising infrastructures of computing and fabrication can be brought back to a highly specific, localised point. This for me is the most exciting aspect of digital fabrication and "mass customisation" - not more choice or user-driven design (which are all fine, but essentially more of the same, in terms of the consumer economy) - but the potential for objects that are intensely and specifically local.
Wednesday, October 07, 2009
Weather Bracelet - 3D Printed Data-Jewelry
Sunday, August 23, 2009
Tiny Sketching
As a kind of test pattern to fill the current break in transmission, here are my contributions to Tiny Sketch, an OpenProcessing / Rhizome competition (open until mid September) for Processing sketches under 200 characters.
In Bit Sunset I just load the pixels[] array, pick a random block of pixels, and add a large number to their value. This process throws up some surprising results as the colour values gradually increase, then start pushing into the alpha bits of the ARGB integer; eventually, as it fills the alpha bits, it settles into a pallette of pinks and greens that are gradually smashed into pixel-dust.
Albers Clock was an attempt to slow the pace of TinySketch even further; it visualises the current time in the form of an Albers square, with three colours, one each for hour, minute and second. I also like that it creates an image that is synchronous (within timezones, at least), unlike the asynchronous, individualised runtimes of most sketches.
There are dozens of amazing sketches in this collection - it's a fascinating microcosm (in every sense) of the current Processing / generative / code art scene. Given the tight constraints it's not surprising to see some demoscene virtuousity in the code - like Martin Schneider's Sandbox, a physical simulation painting app in 200 characters. There is also some classic software art conceptualism and reflexivity - like Jerome St Clair's Joy Division cover and Kyle MacDonald's Except. Great to see projects like this - and OpenProcessing itself - reviving applet culture in an open source, web2.0-flavoured way.
Friday, May 15, 2009
Landscape, Slow Data and Self-Revelation
This text was an invited contribution to Kerb 17: Is Landscape Architecture Dead? This looks like a rich volume with a sharp critical edge, and a swathe of interesting material spanning architecture, urbanism, art and landscape. Unfortunately my contribution was edited fairly severely; so here's the unabridged version. Redundancy warning for regular readers: there's a slight rehash of Watching the Sky in here; but afterwards there's fresh material on landscape / data projects by Driessens and Verstappen and Usman Haque.
Data is, we imagine, an immaterial thing; or at least ethereal, made of light and electricity, processed at superhuman speed, transmitted in real time. The everyday world we move in seems dense and slow by comparison. The landscape is slower again; thick, heavy and persistent. At the moment however those two domains, the fast lightness of data and the heavy slowness of the landscape, are urgently linked. We are faced with the prospect of momentous change in the landscape that is somehow both slow and fast; too slow for our real-time culture to grasp, and too fast for the living systems of the landscape to adapt to. This paper presents a handful of works that dwell in that disjunction, between landscape and data; not solving it at all, but at least forming links, complicating assmptions, and recasting the relationship between two terms that seem to neatly encapsulate our future.
In Watching the Sky a camera looks out my office window, at the sky and the landscape. A banal view over a university campus to a bushy ridge in Belconnen. The camera takes an image every three minutes; four hundred and eighty images in twenty four hours. Tethered to a computer, the camera records for weeks at a time; the computer accumulates thousands of images. I think of the images as data, traces of change in the world outside the office window. I visualise, or re-visualise, this image data in the simplest possible way; an automated process "cuts" a narrow vertical slit from the same location in each image, and compiles all these slits together (this is a digital imitation of an analog photographic technique known as "slit-scan"). In the rectangular visualisations the slices are tiled from left to right. In the radial visualisations slices are gradually rotated so that a twenty-four-hour period spans one complete revolution (the "seam" is at midnight).
In the resulting images the patterns of change within and between days are immediately visible. As I imagined, day and night, cloud and sky are obvious. The brief, delicate colour shifts of dawn and dusk were more surprising. Below the horizon, though, patterns appeared that complicated the work's nominal focus on the sky. It became clear that some of the richest and most revealing data here came from the landscape. In one of the earliest sketches I found small but distinct variations in the horizon line over the course of a day, and recurring on successive days. I eventually realised that this was caused by the afternoon breeze, shifting foliage by a few pixels within the frame. In other words, subtle changes in the material field of the landscape carried through to the image data. Moreover in many ways the landscape visualises its own internal structure: the trees blowing in the breeze are partly instruments, revealing material changes around them (the breeze); but also data, traceable as pixels. In many images the passage of a shadow across the ground appears as a recurring pattern, an enfolded or multiplexed representation of another set of material interactions; the landscape measures and reveals itself, but not as an object, image or view. It is a connective, dynamic, material system; what is revealed are the specific interactions of that system with itself. The image data acts as a kind of core sample, drilling through multiple spatial and material systems, but each is connected outwards, beyond the frame. The wind in the trees doesn't belong to this image, but like the angle of the sun revealed in the shadow, is an index of a wider system.
It also became clear that the landscape is densely packed with human, social data which is equally apparent in image data. In the rectangular visualisations presented here stripes of colour are visible towards the bottom of the frame. These are caused by cars, parked illegally under the trees; they form another ad-hoc graph that reflects cultural, institutional calendars and cycles, though again they are intermingled with other scales and structures.
Landscape is also cast as a self-revealing instrument in Driessens and Verstappen's Tschumi Tulips project. This landscape installation occupied the Tschumi Pavilion, in Hereplein, Groningen, during the northern Spring in 2008. The pavilion is a rectilinear glass container, rising at an angle from the surrounding park. In this installation the artists filled the base of this box with soil and planted over ten thousand white tulips. A matching array of tulips was planted outside, extending the line of the pavilion. Like scientists, the artists set up two identical subjects, but vary their environment: ten thousand tulips inside, ten thousand outside. A webcam reveals how these variations in environment are slowly materialised in the life of the tulips. The tulips inside grow, bloom and then, wonderfully, decay more rapidly than their twins outside. As in Watching the Sky, long time spans are compressed into human-scale time and space; and here too digital imaging plays a pragmatic role in that revelation. Deployed in rectangular masses we can easily read the flowers as abstract, sculptural materials; organic pattern and variation enframed and aestheticised. But at the same time the work has a kind of deadpan resonance, a rendition of life, and death, inside a greenhouse.
The Huey-Dewey-Louie Climate Clock, by Usman Haque and Robert Davis, addresses the long timescales of environmental change head-on in a proposal that further develops this articulation of slow data and landscape. The clock is a multi-layered system of autonomous machines and material processes. The "Huey" agent slowly builds "accretion mounds" using material extracted from the atmosphere and formed into accumulating conical stacks over the course of a year; like tree rings or geological strata these embed environmental materials directly into a designed representation. The "Dewey" element is a circular array of one hundred transparent containers, in which air and biomass samples are preserved year by year. Like Driessens and Verstappen's Tulips, Haque and Davis propose a biological instrument of one hundred genetically identical daffodils, which are sown and harvested each year, then entombed in the plinths - again a simple grid, a layer of invariance is imposed that allows the landscape to essentially represent itself, materially. Finally Louie, an autonomous solar-powered robot, gathers soil samples and compresses them into cubes, one per day. The surface of each cube is imprinted with some current data point - chosen by daily popular vote; perhaps oil price, or rainfall. So here fast, real-time, socially selected data comes to rest directly on the slow, material medium of the soil.
At one stage, not long ago, it may have seemed that we were leaving the landscape behind, or drafting it in only in as a support or substrate for the flickering patterns of real-time culture. Even now, that seems possible: the monthly figure for new housing construction, a bellwether for economic growth, is imposed on the landscape by earthmovers and roadbuilders, underscored by raw mounds of earth. The works presented here suggest an alternative role, perhaps an alternative future for the landscape; as slow data and slow instrument, a complex material system that can be subtly designed into self-revelation.
Sunday, May 03, 2009
Transduction, Transmateriality, and Expanded Computing
In common usage a transducer is a device that converts one kind of energy to another. Wikipedia lists a fantastic variety of transducers, mapping out links between thermal, electrical, magnetic, electrochemical, kinetic, optical and acoustic energy. In this form transducers are everywhere: a light bulb transduces electrical energy into visible light (and some heat). A loudspeaker transduces fluctuations in voltage into physical vibrations that we perceive as sound.
In analog media, transduction is overt (put the needle on the record...). But digital media are riddled with it too. Inputs and output devices all contain transducers: the keyboard transduces motion into voltage; the screen transforms voltage into light; the hard drive mediates between voltage and electromagnetic fields. A printer takes in patterns of voltage and emits patterns of ink on a page. Strictly transduction only refers to transformations between different energy types; here I want to extend it to talk about all the propagating matter and energy within something like a computer, as well as those between that system and the rest of the world. From this transmaterial perspective a computer is a cluster of linked mechanisms and substrates; a machine for shifting patterns through time and space.
If this sounds unfamiliar, it's only by historical accident. Mechanical computers, where these patterns are physically perceptible, predate electrical (let alone digital) ones, by centuries (above: a replica of Konrad Zuse's Z1, a mechanical computer from 1936. Image by rreis). Materially, our current computers are more or less black box systems. Their transductions come as a sort of preconfigured bundle or network, a set of familiar relations constructed again by mixtures of hard- and software, protocols, standards: generalising frameworks. I press a key, a letter appears; this is all I need to know. Click "OK". No user-serviceable parts inside.
Except that currently, across the media arts and a whole slew of other fields, the computer is undergoing a rich and productive decomposition. It's composting, to borrow a Sterlingism. This goes under all kind of different names: hardware hacking, device art, homebrew electronics, physical computing. Such practices mount a direct assault on the computer as a material black box, literally and figuratively cracking it open, hooking it up to new inputs and outputs, extending and expanding its connections with the environment. Microcontrollers like the Arduino present us with nothing but a row of bare I/O pins. Finally we can tackle the question of what should go in, and what should come out: of transduction. A whole generation of artists, designers, nerds and tinkerers are taking up soldering irons and doing just that. Below: the Spoke-o-dometer from Rory Hyde and Scott Mitchell's Open Source Urbanism project.
One side-effect of this decomposition of computing is that the ontological status of the digital starts to break down with it. As Kirschenbaum shows brilliantly, the digital is just the analog operating within certain tolerances or threshholds. Thomas Traxler's The Idea of a Tree (below) is a solar-powered system that fabricates objects from epoxy, dye and string, by turning a spindle. Solar energy generates electrical energy, which drives the motor, which draws the string through the dye and onto the spindle: a chain of analog transductions produce an object that manifests specific changes in its local environment. The work is a beautiful demonstration that variability doesn't have to be worked up with generative code: if the system is open to it, it's already there in the flux of the material field.
This is not to dismiss computing, only to recast it: an incredibly dynamic, pliable set of techniques for manipulating the material environment. Paradoxically the very generalities of computing - the abstractions and protocols that insulate it from local, material conditions - make it a powerful tool for transduction, that is, the propagation of specificities. Usman Haque's Pachube is a generalised infrastructure, a set of protocols and standards that rest in turn on wider standards like XML, and which assume a whole stack of functional layers: IP, HTTP, and so on. All in order to propagate material patterns and flows from here to there: this is an architecture of transduction whose utopian aim is to "patch the planet" into a translocal ecology of linked environments.
Digital fabrication is part of the same shift: an expansion and extension of the computer's range of material transductions. Digital pattern, to lasercutter instructions, to physical form. Fabbing shows how material matters. It's unsurprising that a piece of laser-cut ply is aesthetically different to a luminous pattern of pixels; more interesting is the way computation reaches out into the substrate's material properties, and the range of potential applications and domains it opens up. Fabbing has often presented itself with a narrative of materialisation, making the virtual real, translating bits into atoms - Generator.x 2.0 was subtitled "Beyond the Screen." Not so: because of course, the "virtual" never was, and the screen is material too. Fabbing does get us beyond the screen, but only because its processes and materials have different properties, different specificities, and they hook us up to new contexts, as well as new sensations. (Below: Andreas Nicolas Fischer & Benjamin Maus: Reflection - from 5 Days Off: Frozen)
Transduction suggests a way to link practices like physical computing, fabrication, networked environments, and many more. Data visualisation - in the broadest sense, from poetic to fuctionalist - is about creating customised transductions, sourcing new inputs and/or manifesting new outputs (even if they don't reach "beyond the screen"). We could add tangible interfaces, augmented reality, and locative systems. What does all this amount to? In 1970 Gene Youngblood observed a similar moment as the dominant cultural form diversified into a networked, participatory, interdisciplinary field of practices. He called it expanded cinema. So perhaps we can call this expanded computing: digital media and computation as material flows, turned outwards, transducing anything to anything else.
Wednesday, April 15, 2009
Master of Digital Design / Grow Your Own Logotype
Over the past year or so I've been working on a major new offering here at UC. So, I'm delighted to finally launch the new Master of Digital Design online. This course will offer something quite unique in the Australian context: a trans-disciplinary coursework Masters focused on digital practice for designers and creative practitioners of all sorts. The key practical approaches are generative techniques, data visualisation and design, and physical computing; and we'll be using these to address three core themes or questions: the urban, the public, and the sustainable.
As readers of this blog will know, these themes and approaches are right in line with my own research and creative interests; so frankly, I'm thrilled to be leading this course. Teaching with me will be a crew of talented designers, artists and researchers including Stephen Barrass, Sam Hinton and Geoff Hinchcliffe. Finally, we'll be drawing on the wisdom and experience of an international advisory panel whose work exemplifies what we mean by digital design - a practice that engages deeply, and critically, with digital processes, digital materials, and digital contexts: Karsten Schmidt, Rory Hyde, Nervous System, Anthony Burke and foAM.
The course launch has also provided a great excuse (er, opportunity) to play with some ideas around generative branding and marketing. I've been tinkering with this logotype for ages; it uses the same basic algorithm as Limits to Growth but artificially constrains the growth to a letterform (in the guise of a hidden bitmap image). Lately I've extended the logotype into a little generative marketing gadget; a Processing applet that lets you grow endless variations, and receive the results as a PDF file, attached to an email. The aim is to provide a little taste of the power - and pleasure - of generative design.
Behind the scenes this project was yet another demonstration of the brilliance of Processing and its community. The key technical challenge was the upload-and-email functionality. Seltar's "save to web" hack provided the template; upload image data over HTTP, and have a PHP script catch and save the file. From there it was relatively straightforward to have PHP generate the email, with the help of the Pear MailMime package. The final step was uploading a PDF, rather than a bitmap. This seemed impossible, because the built-in PDF library needs to write a local file, which means the extra annoyance of a signed applet. I posted a query on the Processing forums and within 24 hours PhiLho saved me with a solution that extends the PDF class to allow access to the PDF data as a Byte array, without first saving the file. Amazing: thank you! Add the super-useful ControlP5 for the UI sliders and buttons, and the whole thing is built on, in and with free, open-source software. Again, a demonstration of why digital design is such an exciting field of practice right now.
Sunday, March 15, 2009
Watching the Street (Navigator) / citySCENE
Vague Terrain 13: citySCENE has just launched. As editor Greg J. Smith writes:
This issue of Vague Terrain is founded on two notions - that the city is a stage set for intervention and an engine for representation.The collection expands out from this premise in multiple directions: carto-mashups, projection-bombing, sound walks, psychogeographic imaging and ubicomp experiments. Early highlights for me included Crisis Fronts' Cognitive Maps and Database Urbanisms, which presents some impressive work on data visualisation and generative models as urban mapping strategies (below: Case Study: Los Angeles). Overall, on a first look, this collection is incredibly rich. It shows that a creative, wired-up, critical urbanism is not just a wisftul aspiration of the technorati, but a real practice.
Having said all that, it's a privelege to be a part of this collection. My contribution is Watching the Street (Navigator), a browsable visualisation of a single day of images from the Watching the Street dataset. It tests out the hunch that these time-lapse slit-scans can be used to read real patterns in the urban environment - that they are (or can be) more than just suggestive abstractions. It uses a simple interface to display both a single source frame, and a correlated slit-scan visualisation, with image-space and time-space sharing an axis, a bit like a slide rule. Greg Smith called it an "urban viewfinder", which sums the intention up nicely.
Playing with the navigator for a while seems to confirm that hunch. The composites reveal temporal patterns in the environment, but not the spatial context that allows us to identify their causes; the source frames show that spatial context, but not the change over time. Reading the two against each other involves chains and cycles of discovery, analysis and inference. These might be open-ended (spatiotemporal browsing) or more directed. What time do the sandwich-boards go out? How long does the delivery truck stay?
Building the navigator presented some interesting technical challenges: mainly, how to make a web-friendly interface to 1440 source frames (240 x 320) and 480 slit-scan composites (720 x 320). That adds up to about 75Mb of jpegs. Processing 1.0 came to the rescue, with its new built-in dynamic image loader. requestImage() pulls in an image from a given URL, on cue, without bringing the whole applet to a grinding halt; it provides some basic feedback on the state of that image - whether it's loading, loaded, or un-loadable. I also blundered into two other useful lessons: how to use the applet "base" parameter, and how to manage Java's local cache, which kept throwing up earlier versions of the applet during testing.
Having made a lean, mean, browser-friendly version, I'm now thinking of adapting the navigator into a full-screen, offline app, with the whole eight-day dataset, and perhaps some tools for annotation and intra-day comparison. Best of all would be a long term installation; a sort of urban space-time observatory, watching the street but also opening it up to ongoing interpretation. If you'd like it running in your foyer, let me know.
Friday, January 16, 2009
JCSMR Curls
This post is (belated) documentation of a project I worked on in 2007-8, creating an audio-responsive generative system for a permanent installation for the Jackie Chan Science Centre (yes, that Jackie Chan) at the John Curtin School of Medical Research, on the ANU campus. Along with some Processing-related nitty gritty, you'll find some broader reflections on generative systems and the design process. For less process and more product, skip straight to the generative applets (and turn on your sound input).
In mid 2007 my colleague Stephen Barrass and I were approached by Thylacine, a Canberra company specialising in urban art, industrial and exhibition design. Caolan Mitchell and Alexandra Gillespie were designing a new permanent exhibition, the first stage of the new Jackie Chan Science Centre, housed in a new building - a razor-sharp piece of contemporary architecture (below) by Melbourne firm Lyons. Instead of just bolting a display case and a few plaques to the wall, Mitchell and Gillespie (wonderfully) proposed a design that hinged on a dynamic generative motif - a system that would ebb and flow with its own life cycles, and echo the spiral / helix DNA structures central to the School's work, and already embedded in the building's architecture.
My initial sketches (below) took the spiral motif fairly literally, drawing vertical helices and varying their width with a combination of mouse movement and a simple sin function - the results reminded me of the beautiful spiral egg cases of the Port Jackson Shark. At that stage we were talking about the possibility of projecting back onto the facade of the building, which has big vertical glass panels; this structure informed the vertical format. I made a quick video mockup of the form on the facade - which was incredibly easy, thanks to the robust, adaptable, extendable goodness of Processing (a recurring theme in the process to come).
These sketches meet the simplest criteria of the brief (spiral forms) but do nothing about the more interesting (and difficult) ones: cycles of birth, growth and death, and dynamics over multiple time scales. Over the next couple of months I developed two or three different approaches to this goal.
The phyllotaxis model blogged earlier was one attempt. Spurred on by the hardcore a-life skills of Jon McCormack and co. at CEMA, I built a system in which phyllotactic spirals self-organised spontaneously. Because in Jon's words, anyone can draw a spiral, what you really want is a system out of which spirals emerge! The model worked, but I had trouble figuring out how phyllotactic spiral forms might meaningfully die or reproduce. Also, by that stage I had two other systems that seemed more promising.
From the early stages I wanted to make the system respond to environmental audio. The installation would be in a public foyer with plenty of pedestrian traffic, so audio promised a way to tap in to the building's rhythms of activity at long time scales, as well as convey an instantaneous sense of live interaction. In the two most developed sketches audio plays a key role in the life cycle of the system.
One sketch moved into 2d, and started with a pre-existing model for growth, by way of the Eden growth algorithm (this system would later be adapted again into Limits to Growth). I had already been playing with an "off-lattice" Eden-like system where circular cells could grow at any angle to their parent (rather than the square grid of the original Eden model). This system also made it easy to vary the radius of those cells individually. The next step was to couple live audio to the system; following a physical metaphor, frequency is mapped to cell size, so that larger cells responded to low frequency bands, and smaller cells to high frequencies. Incoming sound adds to the cell's energy parameter; this energy gradually decays over time in the absence of sound. Cell reproduction, logically enough, is conditional on energy.
The result is that cells which are best "tuned" for the current audio spectrum will accumulate more energy, and so are more likely to reproduce, spawning a neighbour whose size (and thus tuning) is similar to, but not the same as, their own; so over time the system generates a range of different cell sizes, but only the well-tuned survive. The rest die, which in the best artificial life tradition, means they just go away - no mess, no fuss. In the image below cells are rendered with stroke thickness mapped to energy level. The curves and branches pop out of rules sprinkled lightly with random(), resulting in a loose take on the spiral motif, which is probably the weak point in this sketch. I still think it has potential - nightclub videowall, anyone? Try the live applet over here (adjust your audio input levels to control the growth / death balance).
The third model takes this approach to energy and reproduction - about the simplest possible a-life simulation - and folds it back into the helical structures of the first sketches. In this world an individual is a 3d helix, built from simple line segments. Again each individual is tuned to a frequency band, which supplies energy for growth; but here "growth" means adding segments to the helix, extending its length. Individuals can "reproduce", given enough energy, but here reproducing means spawning a whole new helix, with a slightly mutated frequency band. All the helixes grow from the same origin point - they form a colony, something like a clump of grass.
This sketch went through many variants and iterations over the next month or so; in retrospect the process of working to a brief, within a design team, pushed this system further than I ever would have taken it myself. At the same time I was testing the system against my own critical position; I've argued earlier that the generative model matters, not just for its generativity but the entities and relations it involves.
From that perspective this system was full of holes. Death was arbitrary: just a timer measuring a fixed life-span. "Growth" was a misnomer: the number of segments was simply a rolling average of the energy in the curl's frequency band, so the curls were really no more than slow-motion level meters. Taking the organic / metabolic analogy more seriously, I worked out a better solution. An organism needs a certain amount of energy just to function; and the bigger the organism, the more energy it needs. If it gets more than it needs, then it can grow; if it gets less than it needs, for long enough, it will die. So this is a simple metabolic logic that can link growth, energy and death. Translated into the world of the curls: for each time step, every curl has an energy threshhold, which is proportional to its size (in line segments); if the spectral energy in its band is far enough over that threshhold, it adds a segment - like adding a new cell to its body; if the energy is under that threshhold, it doesn't grow; and if it remains in stasis for too long, it dies. Funnily enough, the behaviour that results is only subtly different to the simple windowed average. Does the model really matter, in that case? It does for me at least; if and how it matters for others is another question.
Next, the curls developed a more complex life-cycle - credit to Alex Gillespie for urging me in this direction. In line with the grass analogy, curls grow a "seed" at their tip when they are in stasis; when they die, that seed is released into the world. Like real seeds, these can lie dormant indefinitely before being revived - here, by a burst of energy in their specific frequency band. After several iterations, the seed form settled on a circle that gradually grows spikes, all the while being blown back "down" the world (against the direction of growth) by audio energy (below). As well as adding graphic variety, seeds change the system's overall dynamics. Unlike spawned curls, seeds are genetically identical to their "parent" - attributes such as frequency band are passed on unaltered. Because each individual can make only one seed, that seed is a way for the curl to go dormant in lean times; if it gets another burst of energy, it can be reborn. The curls demo applet demonstrates this best (again, adjust your audio input and make some noise).
A few technical notes. One big lesson here was the power of transform-based geometry. Each curl is a sequence of line segments whose length relates to frequency band (lower tuned curls have longer segments); each segment is tilted (rotateZ), then translated along the x axis to the correct spot. A sine function is used to modulate the radius of each curl along its length; frequency band factors in here too; this radius is expressed as a y axis translation. Then the segment is rotated around the x axis, to give depth. I iterate this a few hundred times to get one curl, and repeat this process up to twenty times to draw the whole world - each curl has its own parameters for tilt, x rotation increment, and frequency band.
In the live applet audio energy ripples up the curls, from base to tip. This was added to reinforce the liveness of the system and add some rapid, moment-by-moment change. It was implemented very simply. I used a (Java) ArrayList to create a stack of audio level values; at each time step, the current audio level value is added at the head of the list, and the ArrayList politely shuffles all the other values along. So each segment's length is a combination of three values; the base segment length, a function to taper the curl towards the tip, and the buffered audio level.
The graphics are all drawn with OpenGL - following flight404 I dabbled with GL blend modes, specifically additive blending, to get that luminous quality. The other key visual device here is the smearing caused by redrawing with a translucent rect(); instead of erasing the previous frame completely this fades it before overlaying the new frame. It's an easy trick that I've used before. But as Tom Carden explains, in OpenGL it leaves traces of previous frames. I discovered this firsthand when Alex and Caolan asked whether we could lose the "ghosts." I was baffled: on my dim old Powerbook screen, I simply hadn't seen them. Eventually, juggling alpha values I could reduce the "ghosts" to almost black (1) against the completely black (0) background - but no lower. Finally I just set the initial background to (1) instead of (0), and the ghosts were gone.
The adaptability of Processing came through again when it came to realising the installation. The final spec was a single long custom-made display case, with three small, inset LCD panels. These screens would run slide shows expanding on the exhibition content, but also feature the generative graphics when idle; the case itself would also integrate the curls as a graphic motif. For the case graphics, I sent Thylacine an applet that output a PDF snapshot on a key press; they could generate the graphics as required, then import the files directly into their layout.
The screens posed some extra challenges. The initial idea was to have the screens switch between a Powerpoint slideshow, and the curls applet; but making this happen without window frames and other visual clutter was impossible. In the end it was easier to build a simple slide player into the applet: it reads in images from an external folder, allowing JCSMR to author and update the slideshow content independently.
So to wrap up the Processing rave: it provided a single integrated development and delivery tool for a project spanning print, screen, audio, interaction, animation and even content management. Being able to burrow straight through to Java is powerful. Development was seamlessly cross-platform; the whole thing was developed on a Mac, and now runs happily on a single Windows PC with three (modest) OpenGL video cards. The installation has run daily for over six months, without a hitch (touch wood).
Some installation shots below, though it's hard to photograph, being a glass fronted cabinet in a bright foyer - reflection city. I'll add some better shots when I can get them. If you're in Canberra, drop in to the JCSMR - worth it for the building alone - and see it in person.
And very finally, photographic proof of the Jackie Chan connection - image from The Age.
Friday, December 19, 2008
Fabricated Growth Forms (Processing to Ponoko)
Like many others playing with generative techniques, I'm fascinated by the potential of digital fabrication. Getting beyond the screen and into the world of objects is a significant move for a field that has, until the last few years, reveled in its own immateriality. There's a lot to think about in this material turn, but that's for another post. Here, a quick report on my first experiment with generative fabrication.
I don't have a laser cutter handy at my workplace (though as William Turkel writes there are lots of good reasons why I should) so I decided to check out Ponoko; I wanted to see what was involved in generating, uploading and fabbing a small project. I started with the Processing sketch from Limits to Growth, and tweaked it to turn out much smaller forms (a few hundred nodes, rather than tens of thousands). I used the built-in PDF export, then opened the PDFs in Illustrator. (Illustrator is the only commercial/proprietary software step in this process, so I'd be interested to hear of any alternatives). The forms are drawn as linked line segments of varying stroke widths. Ponoko needs an EPS with only the outside edge of this form, so I used Illustrator to merge it into a composite path, then set the stroke colour and width as instructed (0,0,255 and 0.001mm).
The upload to Ponoko took a few tries - I was getting some strange errors as their system failed to "see" the cutting paths on the template - but after some swift and cheerful technical assistance it all worked. Pricing was also trial and error; the first design I uploaded was more complex than these, and of course these branching forms pack a long cutting path into a small surface area. I simplified the design, packed four forms onto a sheet, and opted for 4mm ply rather than acrylic. Final cost including (expensive) shipping to Australia was about $A60 (currently around $US40). Not what I'd call cheap, but not prohibitive either. There are intricate discussions of the economics of the business - shipping, exchange rates, local vs global, etc - on the Ponoko forums.
Eighteen days later, they arrived. Novelty counts for a lot here, but still, I'm totally charmed by these objects. A few surprises, but all good: they are smaller and finer than I imagined, and they smell very slightly of charred wood (excellent!). The cut edges are dark with a nice smooth, burnished surface, and the ply surface is clean. The scale and intricacy of the things seems to entice people to touch and handle them. I find them far more satisfying than the (much more detailed) laser prints I made with the same system.
Immediately it's clear how the fabbing process, and the materials, can reach back up through the production chain and influence the design and the generative system. One flaw in the design is a product of how I'm drawing the shapes: there are small rounded "shoulders" at the joints between line segments, caused by the overlap between one rounded line cap and the next segment - this is obvious in the physical forms. Better to draw the segments as tapered rectangles, and avoid the shoulders. Also, the branching topology is structurally risky; how to introduce more joins without breaking the generative model? This interplay, between computational process, manufacturing process, material and form, seems really promising. Ponoko seems to be an excellent, affordable way to try this out, and the built-in fab-on-demand shopfront is great, if you want to sell your wares. But it's still, ironically, working with a mass-production paradigm of one design, n copies. With hooks for a more dynamic, generative front end, it could get really interesting: designers like the wonderful Nervous System are doing this already. More documentation of the growth forms over on Flickr.
Thursday, November 27, 2008
Watching the Street
The recent Dorkbot show seemed to go off nicely - it was great to be part of such a strong show of local work (some documentation). I showed some prints from Limits to Growth, as well as a more experimental process piece, Watching the Street - a (sub)urban remake of Watching the Sky.
Credit to Nathan McGinness for the suggestion: use the same time-lapse / slit-scan technique to image change in an urban environment. Technically, the setup was fairly straightforward. Instead of a digital stills camera I used a webcam (in portrait orientation), and wrote a simple Processing script to save stills at one-minute intervals, while extracting and compiling one-pixel slices into 24-hour composites. The webcam was installed in a window box on the gallery street front, with a view across the road, under a street tree, to one of Manuka's low-rise shopping arcades (above). I also attached a printer to the installed rig, so that a new composite could be produced and pinned to the wall each day. So here, some of the resulting images, and a bit of commentary.
The image-gathering process got off to a rocky start. After a few hours, the webcam came unstuck from the side of the window-box, and lay forlornly on its side for the next 48 hours (here's what that looks like). I gaffed it back in place just before the opening, and restarted the capture in time to catch some gallery-goers loitering around out the front.
These two are the Frday the 7th and Saturday the 8th of November, the first two full day composites. Those striped rectangular chunks around mid-frame are cars, parked in the 30 minute loading zone accross the road. Some stay for a few minutes, a couple for what looks like an hour. Of course on the Saturday, the loading zone doesn't operate, and there's a single car parked in it from mid-morning to mid-afternoon. The single-pixel vertical shards give an indication of passing car and pedestrian traffic.
A quiet, sunny Sunday the 9th; the form hinted at on the 8th, reveals itself as the shadow of the big plane tree, creeping across the footpath. Then the following Friday the 14th. It's all happening; lots of car and pedestrian traffic, changes in sunlight, looks like an afternoon breeze in the foliage as well. The dominant, bluish horizontal stripe in all these images is the neon sign on the shopping centre - which runs all night. The orange rectangle that extends into the evening is the interior light of a shop - which you'll notice switches off at slightly different times each night.
So you'll notice that as in Watching the Sky, I'm persisting in reading these as visualisations of the environment, as well as digital images in themselves. I'm struck by how this simple, indiscriminate process reveals both expected and unexpected patterns, and continues to provoke new questions. This despite, or I would argue because of, its openness to multiple material / temporal systems. In an interesting bit of synchronicity, I was teaching in the UTS Street as Platform masterclass with Dan Hill (more on that soon) while this piece was running. Could a simple visualisation process like this function "informationally", as it were; to help answer real questions about a very specific slice of urban environment, in near-real time? More interesting for me, could it function in that way without prescribing the question in advance - that is, could it support an open-ended process of exploration and interpretation? I'm planning to build an interactive version of this piece, to try out these ideas. In these static visualisations there's a huge amount of data missing: I set the slice point more-or-less arbitrarily, so there are 479 other potentially interesting slices to browse. It would be nice to be able to change the slice point dynamically, as well as navigating through the source images. I notice that Processing 1.0 (yay!) now supports threaded loading of images: could come in handy. Meanwhile, the full set of composite images are up on Flickr.
Monday, October 27, 2008
Dorkbot CBR at Manuka CCAS
