The "Synchresis" DVD

The Australian Network for Art and Technology (ANAT) has produced a DVD entitled "Synchresis", distributed with the Summer 2007 edition of ANAT’s magazine Filter. The DVD contains ten recent pieces from Australian audiovisual artists, including my piece Triangular Vibrations. The associated issue of Filter includes an essay "Monsters and Maps" by Mitchell Whitelaw, who curated the DVD, and articles by three of the artists represented on the DVD, namely Robin Fox, Jean Poole and Botborg.

"Synchresis" was not a word I knew; Mitchell's essay informs me that it was coined by the French theorist Michel Chion, and defined by him to mean "the spontaneous and irresistible weld produced between a particular phenomenon and visual phenomenon when they occur at the same time". The word only makes sense in a technological culture, as before the invention of the telephone, the phonograph, and ultimately the movie soundtrack, there was no way of separating a sound from its physical cause. Now we can associate any sound with any image, producing hybrid experiences that have no counterpart in reality, the "monsters" of Mitchell's title.

Here I wish to make a few comments on the relationships between the sounds and the images in the works on the DVD. Inevitably these comments recapitulate Mitchell's to some extent.

Most of the pieces lie at one of two extremes of a possible spectrum: either the sounds and images are very tightly coupled or they are apparently totally disconnected. From this point of view Peter Newman's Rosebud is the most conventional piece: crackly sounds to go with (slowly revealed) fiery imagery, a background of synthesiser sounds for ambience; so a loose coupling based partly on convention. I should say that it is only in the relation of sound and image that this work is conventional; in other ways it is extraordinary.

In three of the "tightly coupled" pieces the connection between sound and image is literally mechanical, via hardware. Robin Fox’s piece Immaculate Infection is created by feeding carefully crafted audio signals into an oscilloscope; an ordinary diagnostic technique is elevated to an artform. The other two "mechanical" pieces involve perversions or unintended uses of equipment: Andrew Gadow’s Technè-Auxons uses video signals from a Fairlight CVI (Computer Video Instrument) as audio; Botborg feed video signals into audio inputs and vice versa, creating complex and unstable results.

In the two remaining pieces with tightly coupled sound and image the connection is more abstract. Here we encounter the "maps" of Mitchell’s title. There has been much discussion about processes creating images from sounds (and the other way round); an elaborate example is here. At the simplest level any such process requires the specification of a map (function, correspondence, mapping) that takes some aspect of sound as input and produces a colour or other visual component as output. Maps need not be from one perceivable object (sound or image) to another; the fields of sonification and visualisation take abstract data in many forms (stockmarket prices, brainwave signals, rainfall) and map the data to either sounds and images. The choice of map is crucial. In 2004 I made a sonification of brainwave data for a concert of sonifications, and much of the creative work went into developing the mappings.

My piece on the "Synchresis" DVD has an underlying abstract process which generates both the sound and the image. So two maps are required, one from the process to the sound and the other from the process to the image. My process was a mathematical model representing an idealised drum, and my mappings followed the physics closely, though not slavishly. The other work that depends on mappings is Julian Oliver's and Steven Pickles's Fijuu2, a piece of software allowing a user to build up electronic music tracks in layers using a video game controller. Each sound has a different visual counterpart that reflects changes in the sound by becoming more or less elaborate. It appears that each sound is controlled by a few numbers and has an associated pair of mappings, one from the numbers to the actual sound we hear, and another from the same numbers to the visual associated with the sound. The maps appear more arbitrary than mine in Triangular Vibrations. The DVD track for Fijuu2 is a demo of the software.

It is hard to comment on the four pieces on the DVD with no close relationship between sound and image. The connections are made in the viewer’s brain, and the pieces need to be experienced rather than described.

The style or mood of the various pieces was quite unrelated to the degree of coupling of sound and image. Several pieces had what I think of as an epileptic aesthetic, very flickery and jerky, including the pieces by Robin Fox and Ian Andrews, and especially Peter Newman's, which is extraordinarily intense; I found it physically difficult to watch. These three pieces span the spectrum from tight coupling to no apparent coupling. Again, I would call my own piece slow but intense; Wade Marynowski's piece is also slow, but laid back; it appears to be a record of a live performance, and the slow tempo may have been dictated by the demands of real-time video processing. [Edit: I was wrong about this. Wade's piece is basically a recording of a live performance, but Wade tells me that he could have made it frenetic; the relaxed pace is an artistic decision.] My piece has tight coupling; Wade's has very loose coupling. The decision of the creator about the degree of audio-visual coupling is a meta-aesthetic decision, with indirect effect on the surface sensory qualities of the artwork.

What the pieces on the DVD do have in common is that they are audiovisual: they take seriously the relationship between sound and image. As Mitchell points out, the DVD explores territory nearer to sound and music than to video art. I found the DVD and Mitchell’s essay illuminating both in bringing together the actual works and in providing a framework to understand them and their context.