From October 2019 to March 2020, I have been a contributing artist to Testing Grounds Studio program, and based my practice at Southbank, in Melbourne Arts Precinct.

Here, I focused my research on drawing and sounds in virtual reality.

Producing 37 walk through, VR drawings that describe the heard sounds happening around me. Using Tilt Brush software to develop these drawn responses whilst listening closely to the city soundscape, and identifying and inventing a drawn vocabulary of ‘sound marks’ that audiences will be invited to interact with in a virtual environment.

During the first month at Testing Grounds, I concentrated the research on to listening to the sounds of the ‘place’, ‘space’ and ‘site’ itself. Making audio recordings, using binaural H1 and Zoom H3-VR devices. Identifying the different types of sounds occurring in the area, and testing approaches for drawing the heard sounds. 

This initially took the form of using coloured pencils and gauche onto paper journals and then rapidly changed to drawing directly in virtual reality, using Tilt Brush Software.

Through this daily exercise of listening and drawing I began sketching a type of ‘sound-map’ of Melbourne city and developing a key of marks and colours to represent the sounds and auditory events occurring there.

This process of journaling and drawing sounds onto paper was relatively short lived due to physical limitations such as paper/page size, and the unsatisfactory nature of drawing 360 sound events onto a 2Dimensional surface.

Taking up virtual reality as a medium to explore drawing sound, enabled the sounds to be drawn, felt out, thought about, and responded to as they are experienced in 3Dimensional space, and time. Allowing for the development of a mapping process that positions ‘sound events’ as drawn ‘sound marks’ in 360 space, corresponding to the sound source’s orientation, and direction it is traveling from.

What I discovered very quickly, is the city of Melbourne is packed full noise and numerous immersive and rich soundscapes, and that these noises and sounds are specific to a location, time, and space.

For those of you who aren’t familiar with Testing Grounds site, is located in Melbourne Southbank Arts Precinct and is a temporary infrastructure that supports experimental art practice.

The site, originally a vacant lot, is fitted with an infrastructure, shipping container ‘studios/performance spaces’ and a garden, and is surrounded by main roads, tall modern architecture, building sites, and concrete infrastructure. Materials in the immediate vicinity (concrete, steel, bitchumen, and glass) act as reflective surfaces, resonating and bouncing the sound back off the city’s architecture and funnelling the amplified sounds through the corrugated steel studio walls, almost like a speaker system.

The dominant sounds on site are traffic noise with smatterings of ‘sound events’ such as, car horns, nail guns; airbrakes; helicopters; voices; with the occasional bird. 

Including: minor/crow/finch/cockatoo/lorikeet/blackbird calls.

Studio Method:

The VR drawings are composed whilst listening to the sounds of the city as a ‘lived space’ and are intended as a direct, and intuitive response to the heard sound around me.

The direction the sound source is heard from, and its orientation to the site and studio determines the position of the drawn ‘sound-mark’ in virtual space of the studio and to some extent, the overall composition of the VR drawing itself. As I if I imagined the position of the sound on a page.

Below is an early example of drawing in response to heard sounds in VR

Considerations:

Geography, map making and sound as an orientational device.

Sound and Architecture – impact and acoustic qualities of space and place listening from. This steel walled studio/shipping container resonates with noise and is like listening from inside a speaker bin.

Sound and Location – the studio, positioned on the corner of Fanning Road and City Road, is surrounded by tall architecture and infrastructure comprised of materials including glass, steel, tiles, and concrete which reflects the soundwaves via a process of reverberation and the built environment acts as a funnel changing the sounds direction as it moves through auditory space.

This month I chose to invite as many artists, curators, friends, VCA colleagues and visiting academics to come visit the studio and share the work in progress.  This has started some interesting conversations and is an informal way of testing out, how people engage with the VR drawings and move through a virtual environment.

Thanks to all who’ve dropped by to experience the works including: Tilly from Science Gallery; Ioannis and Jess VCA researchers, Supervisors Roger Alsop and Helen Gaynor and the Film and Television students from Melbourne Uni, friend Sharon and visiting academic Dr Jackquelene Drinkall from Tasmania University. 

This second month I’ve continued experimenting with listening to the sounds in the studio and drawing a response to these heard sounds directly in VR, using Tilt Brush software. 

Project ‘Exercises in Listening’ :

Making audio recordings onsite at Testing Grounds and around the city, and documenting the auditory soundscape of Melbourne.

Experimenting with different types of field recording equipment such as: Zoom H1 with Roland Headphone Mics, iPhone Voice Memos App, and Zoom VR recorder. 

Recording locations include:

Flinders Station Clock Tower – Remembrance Day 1 min silence 11, Nov 2019,

Private Amenities, Degreaves Street Campbell Arcade 28 Nov 2019,

Testing Grounds Ladies Cup Day 8 Nov 2019,

RMIT Fashion event TG 5 Dec 2019.

In conjunction, to a daily exercise of drawing the everyday sounds heard at the studio, using virtual reality in the HTC Vive headset, I have been making audio recordings of the immediate soundscape.

Testing different field recording set ups at various locations, events, times of day and weather conditions etc. Building up a picture of the local auditory space and the sound events that happen there and responding to the sound in 360 drawn lines in virtual environment. Developing a type of vocabulary of hand-drawn ‘sound-marks’, responding to heard events happening around Testing Grounds, and applying this invented vocabulary in the drawn virtual environment.

Example of drawing responding to event sounds : –

Study 058 Testing Grounds on 8 Dec 2019 360 VR drawing responding to sound on site

1. Studio Notes November 2019
2. Compositional considerations:- Orientation and direction of the sound source and its relationship to TG site. Identifying different types of ‘sound events’ and assessing its ‘felt’ location in the auditory space. Physical position of myself, as the listener and draftsperson responding to sound, and the relationship of the marks in the drawing in 360 virtual space.
3. Developing a key of drawn ‘sound-marks’ with aim of building a ‘sound map’ of Testing Grounds site, using the auditory data and site responsive information gathered.
4. Noticing the act of listening and the auditory senses of hearing sound, happens not just through the ears but through the body as well. Using a process of Bone Conduction – reference the research Installation Imperial War Museum WW1 Armistice by Coda to Coda 2018, reinterpreting a graphic recording made on 18 Nov 1918 at “The End of the War” using ‘a ‘recording’ made on film of sound pressure impulses picked up by ‘sound ranging’ equipment stationed along the allied front’ and a process of listening to the installation using audiences elbows to hear the last ‘few’ exploding bombs on front. Ref: Coda to Coda, 2018 ‘IWM – WW1 Armistice Interpretation (Sound Installation)’ Nov 6 2018.
5. Initial challenges of VR drawings of sound – map and correspond to the direction of heard sound in movement and attempts to locate the ‘sound event’ in 360 virtual space and relationship to myself the listener a. e.g. Crow above head orientated NW
6. Drawings – defined as a series of hand drawn sound events describing a soundscape over time.
7. Critical question – How to treat time in drawing ? Drawing is a fixed event which takes time; versus Sound a unique, mark of time and in constant motion.

This series of investigative drawings has allowed me to formalise a key and colour code to represent the “sound events” and a logical process for positioning the drawn “sound marks” in the virtual environment.

The chosen colour palette is site specific and based on the corresponding colours found in the immediate surroundings and architecture on site at Testing Grounds.

Experiment Notes:

VR drawing in response to sound – Initial drawings are intuitive responses to sounds identified in the city soundscape in VR. “Sound mark” a shape and formal structure in line this is influenced by embodied, learn’t and visual knowledge.

Consider imagined responses, shadows, references to the ‘seen/real’ world in a ‘virtual’ space, the felt perception in virtual environmental, emotional mood as making the work.

Direction and placement of mark in virtual composition – is determined by length and orientation/direction of the sound. Consider the length, frequency, pitch, volume, reflective nature of the sound source and its position in relation to me, the listener and maker. Bone conductivity, where I perceived I felt the sound in my body etc.

Key of Sounds:

Helicopter – Warm Light Blues

Lorries and Buses (idling) – Dark Grey – Black Dark Blues

Motorbike – Madder 

Horn – Scarlet (Warm Red)

Birds – Yellow/Green with black (blackbirds/Minor/sparrows)

Crow/Cockatoos – Black Yellow

Music – Opalescent Pink/Blue (Tints)

Gate/Traffic Squeaks – Hot pink (Straight lines)

People – Purple

This daily exercise of listening and drawing in VR is building into a personalised sound vocabulary.  What I hope moving forward is to develop this into a series of hand drawn, 3Dimensional virtual drawings of a soundscape that audiences can walk through, touch, and listen to. 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

This month, I’ve discovered whilst listening regularly to the space at TG, is that I am listening with my body and not just hearing with my ears.  Feeling the soundwave at the height and direction of the sound source, through a process of bone conduction and sensing the reverberation, echo and orientation of the auditory stimuli at different points in the body and corresponding spaces in the studio. Therefore, expanding my personal understanding of auditory perception and the act of listening.  

I’d like to consider the work ‘Coda to Coda: A new world armistice in sound wave 2018’ at the Imperial War Museum London, UK as it references the physicality of sound and processes for sensing and listening using bone conduction to bridge the gap and transmit the sounds. This sound exhibit is an interpretation of an archival image from a ‘sound ranging unit’ recording taken on 11 November 1918. Ref: Coda to Coda Making a New World Armistice Soundwave 8 Nov 2018 https://codatocoda.com/blog/making-a-new-world-armistice-soundwave/

This sound installation reconstructs the sounds of WW1 gunfire, interpreting an image from a “sound ranging unit” recording onto film. Reconstructing the image into 1 min imagined soundscape at the end of world war one armistice. The Archive footage dated 11 Nov 1918 and is part of Imperial War Museum collection, London UK

The soundscape is densely populated with traffic noise and peppered with high frequency/sharp sounds such as: accelerating motorbikes, care horn, occasional crow or cockatoo, and raised voice, music from Community Precinct coffee and later in the evening blackbirds with a smattering of minor birds and sparrows throughout the day.

A repeated daily exercise of drawing and listening to sounds in the one space.  And making audio recordings and VR drawing of the sounds heard directly in the studio using a Vive Headset and Tilt Brush.

December 2019

At this point, I’ve produced 37 walk through VR drawings in response to sounds with corresponding field recordings from around Testing Grounds Site and Melbourne CBD.

Supported by 50 ‘Art Objects’ perspectives/view points from inside the VR headset. These works document the development of the drawn responses to the soundscape around me. Using the formal process of drawing to invent a vocabulary of sound in colour, line, and shape. Reflecting on the drawn findings and identifying relevant component forms, structures, and compositions that I can use subsequent drawings. Transposing this personal ontology of ‘sound marks’ into subsequent drawings. Thereby devising a graphic score and hand-drawn vocabulary of heard sounds.

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

Studio process: Recording immediate heard soundscape at studio using ‘Voice to Text’ – google docs app on phone. Aiming to record initial responses and stream of consciousness for further analyse; as a record of approach; and research findings in this Blog.

Field recording experiments and corresponding location/sound event with H1 Zoom with binaural headphone mics and or Zoom H3-VR audio recorder.

I’m reading a book by Cretien Van Campen “The Hidden Sense. Synesthesia in art and Science.” 2007. This book explores the experience of Synethesia from scientific and artistic perspectives. And establishes connections to visual and auditory perceptions of colour and sound, through the process of Synesthesia. Describing “these as uncommon sensory experiences when two or more senses cooperate in perception.”

February 2020

From January to February this year, I’ve been making drawings in virtual reality that respond to the sounds heard at Testing Grounds studios. Undertaking a daily exercise of listening to the immediate soundscape and drawing a response using Tilt Brush software. Resulting in the production of 37 3Dimensional hand-drawn studies that describe the local acoustic ecology.

This process of inquiry has required me to develop a drawn language of ‘sound events’ and invent an accompanying vocabulary of ‘sound marks’ using colour, line and form that represent the types of sounds I am listening to e.g. red line in the shape of a horseshoe = car horns; and thick blue/black lines = lorries and trucks on city road. Using this vocabulary of ‘sound marks’ to draw a type of graphic notation and recording of sounds in 360 space.

This is an important point in my research as I have identified a process for describing sounds heard using a drawn language which I will use as a key for plotting sound maps in future work.

I was lucky to get to the book launch earlier this month and hear some of the contributors speak about this fantastic book I’m reading at the moment called ‘The Project is the Relationship. Working with Communities’ edited by Jade Lillie, with Kate Larsen, Cara Kirkwood and Jax Jacki Brown A. It is certainly easy to read, with straightforward and professional considerations and questions to hold in your mind when working with people in the community. Practical and relevant to anyone working with others in a creative capacity.

March 2020

This month at Testing Grounds studios I have been working closely with Warren Armstrong on the design for a ‘Sound Field Sketch Pad’ or SFSP, a bespoke software tool that visualises 360 sound in VR. Together we have developed a process for using a Vive Tracker, VR object that allows you to bring ‘real’ objects into a ‘virtual’ world, and thanks to a cunning hack from Warren the tracker now allows participants to see a drawing without the use of a headset.

The ‘Sound Field Sketch Pad’ is similar to a spectrograph, in that it uses colour to visualise frequencies in an ambisonic field recording, in VR. It is designed to allow me to see, locate and eventually plot sound events in 360 space in a virtual environment. The SFSP displays the changing frequencies over time as different colours, registering from black (no registered sound), through to a sliding scale of red (low sound); to orange; to yellow; to green (moderate sound); to blue/purple (high sound).

The initial prototype is now working and testing of the software has begun in VR with options for navigating the ‘seen’ soundscape by pausing, rewinding, and fast- forwarding the sound recording in 360 space. The next phase in the design process is to devise a system for sketching these sounds in VR and providing options for me to plot and locate the sound events as they occur in time. With the aim to be able to sketch out and export these drawings of sound and develop them into a map that traces the sound experience in the coming weeks to months.

Another key focus this month was developing interactions with a Vive Tracker. Working closely with virtual developer Warren Armstrong to devise a process for viewing the VR drawing using a Vive tracker and projection system and thereby circumnavigating the VR headset.

With the aim of collaborating with Ioannis Sidiropoulos, body improvisation and performance artist to develop a performance that explores VR drawing virtual technology and the bodies movement through a virtual environment.

Thanks to a cunning hack from Warren, a participant holding the virtual tracker can now see the VR drawing in space without using a VR headset, through the tracking device and projections alone. Freeing up the participant to walk through the VR drawing and use the tracker as a ‘torch’ and discover the unseen drawings in the dark, like an explorer in a cave.

Began initial research with Ioannis using the bodies movement, VR tracker and projection system to explore virtual drawings. Consulting with Centre for Dramaturge and Curation as we develop a performance, with the hope of a public outcome in the near future. Initial plans for a performance at ‘The Great Rehearsal’, Testing Grounds in 26 April was postponed due to COVID-19.

I am re-reading a charming book called Birds Art Life Death. A Field Guide to the Small and Significant by Kyo Maclear.

References:
Harriet Hawkins For Creative Geographies Geography, Visual Arts and the Making of Worlds

Memory Edited Ian Farr Documents of Contemporary Art

Sonic Agency Sound and Emergent forms of Resistance Brandon LaBelle

Marco Fusinato Lets destroy Work – Graphic score

Making a New World: Armistice Soundwave Coda to Coda installation Imperial War Museum (Hearing through Bone conductivity) https://codatocoda.com/blog/making-a-new-world-armistice-soundwave/

Listening In. “What Does Colour Sound Like” https://youtu.be/2xDnxkzQtdI, YouTube 20 August 2019

Wassily Kandinski – Colour to Sound ** notes

Brandon La Belle book Sonic Agency and The Floating Citizen 2018 project dancers improvising and responding to heard sounds in 3 different sites across the city of Barcelona. https://www.brandonlabelle.net/floating_citizen.html

Zotero/ Biblography correct reference