Illustration 2: The main hall of the Gallery building at Herrenhäuser Gärten, Hannover, looking west.

Illustration 1: The flow chart of Chroma XII used by the composer.

Illustration 3: Post-production at the IEM Cube in Graz, on a 24-speaker hemisphere of Tannoy 1200s. The console on the left controls the CUBEMixer. The MIDI controllers are attached to the machine on the right, which runs the Ardour session. The grand piano was extremely handy while working with the score. The two red buttons on the tables are emergency-mute switches for the speaker system.

Illustration 5: Setting up Chroma in the SPIRAL at CeReNeM, University of Huddersfield.

Illustration 4: ICSA playback setup, 40 Neumann KH-120 plus four KH 810.

Field Report II - A contemporary music recording in Higher-order Ambisonics

Capturing Chroma XII by Rebecca Saunders

Jörn NETTINGSMEIER Freelance audio engineer Lortzingstr. 11 Essen, Germany, 45128 nettings@stackingdwarves.net

Abstract

In 2010, the author had the privilege to capture a performance of Rebecca Saunders' intricately spatial composition Chroma XII in fully periphonic third-order Ambisonics. The production grew to considerable complexity and provides an excellent showcase for a large-scale Ambisonic production using free software. This paper discusses the artistic motivation (or even necessity) of using a with-height recording method for the work at hand.

After a short description of the composition, its instrumentation and the performance space, the microphone and mixing techniques are being discussed in detail, including hardware and software toolchains, post-production workflow, and lessons learned from subsequent replays on various systems.

This paper is a follow-up to a workshop paper presented at LAC 2010 in Utrecht [1 Jörn Nettingsmeier, 2010: “Field Report: A pop production in Ambisonics”, in: Proceedings of the Linux Audio Conference 2010, Hogeschool voor de Kunsten, Utrecht
http://stackingdwarves.net/public_stuff/linux_audio/lac2010/Field%20Report-A_Pop_production_in_Ambisonics.pdf ].

Keywords

Ambisonics, live production, with-height surround, free software.

Introduction

In previous higher-order Ambisonic productions, the author had gained some experience in blending first-order Soundfield recordings with higher-order spot microphones in a pop context [1], and in creating entirely artificial renderings with discrete microphones only, in a church acoustic [2 Jörn Nettingsmeier, 2009: “Shake, Rattle, and Roll: An attempt to create a "spatially correct" Ambisonic mixdown of a multi­miked organ concert ”, in: Proceedings of the Linux Audio Conference 2009, Casa della Musica, Parma
http://stackingdwarves.net/public_stuff/linux_audio/ambisonic_organ_remix/joern_nettingsmeier-shake_rattle_and_roll.pdf ]. In order to hone these methods some more, and to test their limits, the author had been putting out feelers for a spatially demanding production of instrumental music.

To be suitable as a periphonic recording showcase, a piece should do away with the traditional concept of a frontal stage, with no excuse for anything but a fully isotropic recording system. It should be interesting and powerful even when recorded documentary-style, that is, naturalistic and without much interpretation during the recording process. It should include elevated sources, and it should take place in an acoustically interesting space, to tax the diffuse-field reproduction capabilities of the recording system. Lastly, it should allow the timbral quality to be judged against well-known references, which rules out electro-acoustic music in favour of instrumental or vocal works.

In the absence of any budget whatsoever, it was clear that the project would have to be piggy-backed on a live concert production.

The composition

Enter Rebecca Saunders, a highly esteemed British composer of instrumental contemporary music with a keen sense of sound texture and space. Chroma is one of her most successful works. Originally conceived in 2003, it has been in high demand ever since, with seventeen successful stagings in various surroundings as of March 2011, despite the considerable resources it requires.

For lack of a better category, Chroma is spatial chamber (or turbine hall, or castle, or museum, or philharmonic, or, in our case, baroque-gallery-with-anterooms and French garden) music. Its sounds originate from two pianos, two percussion sets, two violins, a cello, two double basses, an electric guitar, two trumpets, two clarinets, an (electric) organ, a large number of wind-up music boxes and a portable record player. In the composer's words:

Chroma explores three different key issues: the architecture of the space, the density of the collage in the given acoustic, and the nearness or distance to the different music being performed. So firstly, it’s about entering into a dialogue with an architectural space, exploring and emphasising the particular characteristics of the space. [3 from an interview on the website of Huddersfield Contemporary Music Festival 2010,
http://www.hcmf.co.uk/Pulling-threads-of-sound-Rebecca-Saunders-interviewed ]

For each staging, Chroma is adapted to its new environment by the composer, and may grow a new musical module or two. The audience is invited to move freely around the place during the performance. Conditions permitting, the piece is performed twice in close succession, so that the listeners can explore two different paths through the soundscape. [4 previous three paragraphs taken verbatim from:
Jörn Nettingsmeier, 2010: “Reconstructing Chroma XII by Rebecca Saunders – a recording experiment”, program note for a concert replay at dafx 2010 in Graz, Austria.
http://dafx10.iem.at/concert/rebecca-saunders-and-joern-nettingsmeier.html ]

The musical language of Chroma consists of gestures, not melody or harmony in the traditional sense, and employs advanced non-standard playing techniques. Explosive, almost violent eruptions alternate with very soft and fragile but highly expressive sections.

The work is organized in modules, played by soloists, duos, or trios. These modules are scored individually, and within each module the voices are synchronized to a common time or rubato feel. Parts of Chroma can be (and are) performed individually as chamber pieces.

Each module is played in a specific place inside the larger venue, determined in advance or during rehearsals. The players of a module need not be close together - in the Hannover staging, some groups were spread out over 10 metres or more.

Sometimes, a single soloist or group will play exclusively, but usually, different modules are overlapping. There is an implied global tempo and a carefully laid out flow chart, but usually no precise rhythmic interlocking between modules1 The players all carry stopwatches, which are started together a few minutes before the performace. . Hence, the overall result is not strictly determined in time except when the composer desires a specific unison effect or close interaction. In this case, special cues will be agreed on and rehearsed as required.

The overall dramatic structure, while precisely adhered to once it has been determined, is not set in stone - it exists only in the composer's working chart, which is adapted to each new location, iteratively, during rehearsals (see illustration 1).

When a module is finished, the musicians will quietly pick up their instruments and proceed to the next location. Together with the tip-toeing audience, this imparts a constant extra-musical sense of motion to the scene.

In addition to the instrumentalists, Chroma incorporates some objets trouvés which are added to the sound collage at predetermined times: a large collection of wind-up music boxes spread out on the floor and later on music stands, and one or more portable record players with folk music recordings. In our case, the local bird life added a rather aleatoric but strikingly beautiful layer of its own.

The performers

Based in Köln, Germany, the Ensemble musikFabrik [5 http://musikfabrik.eu ] is an internationally acclaimed specialist ensemble for contemporary music, with an extensive track record of Saunders performances and a history of close collaboration with the composer. One trumpet and one double bass module in Chroma were written specifically for members of the Ensemble musikFabrik.

The ensemble is self-governed - all repertoire decisions are made by the musicians.

The familiarity with and dedication to the material was very beneficial to the recording. Musicians, stage management and composer worked very efficiently, and there was much room for experimentation despite the time pressure.

The venue

Chroma XII2 The Roman numeral indicates the 12th staging and uniquely identifies the Hannover event. Since the premiere at Tate Modern in London, the basic score has constantly been changed, augmented and adapted to each new location. had been commissioned by Kunstfestspiele Herrenhausen to take place in the baroque gallery building of the Herrenhäuser Gärten in Hannover and the adjacent garden. The gallery's main hall is 67.45 m by 11.40 m, with a height of 8.0 m. Most of the modules in the first half happened here. Some musicians were seated in two adjoining hallways, and up on the main hall's two balconies. Additionally, two clarinets and a trumpet were placed on a lighting structure near the center of the main hall, directly above one another.

The building is a baroque masterpiece with floor-to-ceiling fresco paintings and one of the largest stucco ceilings in Europe.

At some point, the windows and door to the garden were to be opened, and musicians would gradually move outside and resume playing in the outdoors. The sound difference between the highly reverberant hall and the semi-anechoic garden was quite intriguing, particularly at the open windows where sounds from the inside and outside were allowed to mix. The piece concluded in the garden, gradually diminishing al niente, which in this case means it ducked under the natural ambience of crickets, birds and faraway chatter, and is probably still there.

The garden paths are covered in gravel, which produced clearly audible crunching footsteps in the recording. We feared this might be a problem, even though it was justified by the composer's directive that the audience move around. In the end the sound turned out to be so charming that it even received a gentle emphasis in post-production.

Recording approach

Periphonic reproduction fulfils three important roles in Chroma. It helps to separate in space those musical gestures which overlap in time, it provides a more complete reproduction of the room acoustics and its interaction with the music, and it has to substitute the freedom of the live audience to move around.

The author spent two days at the rehearsal studios of the musikfabrik in Köln, to listen in on as many module rehearsals as possible and to get acquainted with the musical language. A score was available for study, but while it is easy to look intellectual when sight-reading contemporary music, little insight can be gained unless one is intimately familiar with the notation, instruments and playing techniques at hand.

A preliminary microphone and cabling concept was developed after a visit to the location. The historic gallery building added its own share of complications: there is no way to suspend microphones anywhere, so stands it had to be.

In the absence of a practical higher-order microphone3 At the time of the recording, the commercially available Eigenmike was not yet usable as a B-format microphone. All available spherical arrays suffered from coloration, noise, and bandwidth issues. , we relied on a previously tested hybrid approach [1]: traditional soundfield microphones at the desired listening positions were combined with spot mikes for each individual instrument or group of instruments. These were panned in higher order for the purpose of augmenting and sharpening the image. In order to convey different perspectives and some sense of motion, we used two alternating main microphone positions.

At the rehearsals, we found that the most magic moment was the listener's passage from the hall into the garden - one of the many aspects of the work which could not be anticipated from score analysis. In the absence of an elaborate dolly system, the second main microphone was carefully picked up and carried outside. We made sure the cable run was neat and tried to decouple the microphone from handling noise as much as possible, but some remaining rumble had to be filtered aggressively. The operator's footsteps however were left in, because they would have been part of the live experience as well.

To increase the sense of motion during the passage, the active spot microphones were dynamically panned along.

During setup, it became clear that our gear was visually too intrusive to remain during the evening concert, so we switched from “as neat as possible” to “as easy to tear down as possible” and had to make do with a recording of the two run-throughs of the general rehearsal. It also became clear that we would not be able to spot every single instrument in the garden. In the end, the localisation performance of the first-order main mike alone was perfectly adequate, partly because the outdoor part was more sparse, and partly because there are no confusing reflections in the semi-anechoic free field. The two omnis next to the second grand piano in the garden did not contribute much spatial information, but they filled in the bottom end nicely (particularly because the main had to be rolled off due to wind noise).

After the general rehearsal, we recorded clicks at each instrument position into all microphones, to ensure proper time-alignment in post-production.

The hardware

Our original plan had allotted 22 channels, including two first-order main microphones.

Since it was clear that considerable distances had to be covered, we decided on a MADI ring infrastructure, to avoid the hum problems and interference of a large analog system powered by a strung-out, old, and possibly dodgy power grid. The interior MADI line alone was over 250 m long, with several remote-controlled microphone preamps distributed evenly throughout the venue, so as to keep the analog lines short.

A research team of Technicolor Hannover were bringing their Eigenmike, which was to be recorded on a separate system, synchronized to the main MADI clock.

In the “control room” (a broom cabinet), we used a splitter to distribute the stream to two independent Linux machines equipped with MADI cards, each runninng redundant disk arrays for storage.

Our modest miking plans were quickly steamrollered by the composer's restless creativity: after countless position changes and additions, we ended up with 40 channels altogether, at which point we were running 7 preamps. When the last pair of microphones was connected (and gaffer-taped to a tree because we had depleted our hardware stash), our spare cable box was down to three short XLRs, and we prayed for no more sparks of genius.

The software

For recording and post-production, we relied exclusively on free software based on Linux systems and the JACK audio server [6 Paul Davis, Stephane Letz, et al.: the JACK audio connection kit, http://jackaudio.org ].

On the main recording machine, an Ardour2 instance [7 Paul Davis et al.:Ardour digital audio workstation, http://ardour.org ] took care of writing the data and provided simple two-channel monitoring. We documented the microphone setup with Ardour's track comments feature and used track markers to take notes during the recording.

Ardour was configured to use 60 seconds of recording buffer per track, which increases data loss in the case of a crash or power outage (a situation that would have ruined the take anyways), but helped reduce the disk load, particularly as the session kept growing.

One of the tetrahedral microphones was matrixed and equalised using TetraProc [8 Fons Adriaensen: TetraProc, http://kokkinizita.linuxaudio.org ], the other produced B-format directly. Both were converted to UHJ with jconvolver [9 Fons Adriaensen: jconvolver convolution engine, http://kokkinizita.linuxaudio.org ] and monitored in stereo.

The fallback recording machine ran jack_capture [10 Kjetil S. Matheussen: jack_capture, http://archive.notam02.no/arkiv/src/?C=N;O=A ]. This software writes a multichannel interleaved file, which uses the disk bandwidth very efficiently. An additional load-in step is required before the audio data can be edited and mixed, but for a backup system, the issue only arises if the primary machine fails.

The preamps were remote-controlled with the custom midiremote software [11 Florian Faber: midiremote remote control application for RME Micstasy and ADI-8, available from the author on request ], which turned out to be invaluable, as the farthest preamp was easily a hundred-metre walk from the control room.

Post-production

Microphone bleed

In hybrid HOA recording, directional and very close spot microphones are desirable to minimize crosstalk from other instruments. In this particular case, the problem was even more pronounced because of the huge size of the venue: for sufficiently far-away spots, the time delay of the bleed signals exceeded the echo threshold of human hearing, and the resulting artificial echoes were very audible.

Sometimes, echoes could be attacked with heavy automation, muting or dipping unused microphones whenever possible. But sometimes, the problematic spot carried important content of its own and could not be turned off. There is no perfect solution, and a compromise between score fidelity and echo suppression must be found. In this recording, we were lucky to have many natural room echoes already, so the remaining artificial ones did not stand out too much.

Intensive spot miking can result in small and narrow-sounding sources which do not combine well, particularly if the spots are used at high levels. It gets worse the higher the order of the panners and reproduction systen is - in this sense, too much resolution can be harmful.

To keep the mix from falling apart, each spot must be very carefully time-aligned with the main microphone to ensure proper blending, and it is advisable to use stereo pairs of spots whenever the sound source is large or consists of more than one instrument.

These stereo pairs should either be coincidental (X/Y or M/S) or very widely spaced, if at all possible.

Whenever strongly correlated but non-coincident signals are being used in an Ambisonic system, one must be aware that they are combined internally (i.e. with mathematical precision), which results in very pronounced comb-filtering artefacts. This sounds just as bad as a conventional A/B stereo pair panned towards the center.

While crosstalk can be low enough at higher orders to yield usable results for small A/B or ORTF-like configurations if they are panned far enough apart, the price is a strong dependency on playback order and decoder design (with severe coloration and imaging artefacts if the playback order is too low). Hence, it could be argued that the resulting signal is not proper B-format any more.

Very widely spaced pairs are sufficiently decorrelated at mid and high frequencies to be usable, but at low frequencies, combing can still occur. The main reason to choose wide pairs is that they work well with omnis, which are well regarded for their extended LF response. Since this advantage will be lost in an Ambisonic context due to LF combing, there is really no point in using spaced omnis at all.

Furthermore, omnis will confuse Ambisonic localisation rather than improve it, by adding artificial run-time cues which should not exist in an Ambisonic system.

Coincident pairs on the other hand are inherently free of comb-filtering problems, and give you maximum control over the source width in post-production without unwanted side effects.

Despite their disadvantages, a few omnidirectional microphones did creep into the Chroma recording, partly for their superiour LF response, and partly for the mundane reason that we had run out of cardioids.

For each gran cassa, we used boundary microphones on the floor below. The corresponding overhead microphones of the percussion sets were high-pass filtered to mitigate LF combing.

The grand piano was covered with two cardioids spaced about 1.5 m apart, one above the tuning pegs (to accommodate non-standard playing techniques), the other at the rear end. Likewise, the upper microphone had its bass rolled off slightly.

Two small A/B setups were tried: one on a double bass duo, and the other in the garden. The bass pair was severely affected by the issues mentioned above, but a usable signal could be salvaged by switching off one of the microphones. The garden pair worked quite well, but only after it was delayed enough for the main microphone to dominate the directional perception. In the end, it was just used for some LF fill-up and a dash of “spaciousness”, not for proper localisation.

It should be obvious that omnis are a lot more prone to pick up unwanted echoes. We only got away with them because either the covered instrument was extremely loud compared to everything else (the gran cassa), the particular module was effectively a solo (the double bass), or because some directional confusion was found to be tolerable (in the garden).

Post-production stages

Since full 3D mixing rooms are rare and in high demand, the workflow was split so that each stage could be completed with minimal equipment. Individual tracks were cleaned up (and equalised if necessary) on a standard stereo monitoring system. The time spent on the quite tedious cleanup runs was used to get familiar with the score, and to take notes about necessary mixing interventions.

Gradually, ideas for a “spatial interpretation” of the music began to form, to create a path through the music similar to what a moving listener would experience.

In the next step, the Ardour session was extended to third-order by adding a 16-channel master bus and the appropriate panning plugins [12 Fons Adriaensen, AMB plugins, http://kokkinizita.linuxaudio.org, third-order extension contributed by the author ], with group busses inserted as required. A basic panning scene was created blind, using a floor plan drawn by the composer and some photographs taken during the recording. All tracks were then time-aligned according to the clicks recorded earlier.

Now the production was ready to move to the author's hexagonal system, which is capable of second-order horizontal surround. With the sources roughly in place, a preliminary balance was created, and microphones were automated away when not needed.

When most of the basic homework was done, the session was taken to the IEM Cube in Graz, Austria, for a final ten-day mixing run. The Cube offers a hemispherical 24-channel monitoring system of co-axial Tannoy 1200s, in 12-8-4 configuration, using a prototype decoder by Franz Zotter et al. [13 Franz Zotter, Hannes Pomberger, and Markus Noisternig, 2010: “Ambisonic Decoding with and without Mode-Matching: A Case Study Using the Hemisphere”, in: Proceedings of the 2nd International Symposium on Ambisonics and Spherical Acoustics, Paris.
http://ambisonics10.ircam.fr/drupal/files/proceedings/keynotes/K3.pdf ]

Slowly, the final interpretation was taking shape, and once the acoustic perspectives had been decided, all spot mikes were carefully aligned with the image of the corresponding main microphone. Finding the right balance was the most labour-intensive task. The absence of a global score did not help, as the mess in illustration 3 indicates.

In order to avoid mistakes with the automation, three MIDI controllers with 8 motor faders each were connected to Ardour to provide visual feedback, and for easy access to solo and mute functions. Panners were mapped to a set of rotary controllers with LED indicators for a quick overview of source positions.

Both run-throughs of the general rehearsal were mixed, but only the second one was selected for public playback, because it includes a number of last-minute changes made by the composer during intermission.

Playback

So far, the production has been played back on four periphonic and six horizontal-only systems, and it has stood up pretty well.

Some excerpts were performed during the dafx10 conference at the MUMUTH in Graz. The array was an elongated hemisphere of 29 Kling & Freitag CA 1001 arranged in layers of 12, 10, 6 and 1, driven by a max-rE decoder designed by Thomas Musil and Peter Plessas. The rather long reverberation time disturbed the effect of the outdoor part of the recording, but otherwise the rendering was quite convincing.

During the 2010 Huddersfield Festival, where Rebecca Saunders was artist-in-residence, the author had the chance to hold a workshop at CeReNeM and to demonstrate the mix to the composer in a private session, using the SPIRAL facility [14 Paul Mac, 2009: “A Multi-channel Experiment at Huddersfield University”, Audio Media Magazine, June 2009.
http://www.genelec.com/news/132/358/A-Multi-Channel-Experiment-at-Huddersfield-University/ ]. The system had to be converted to Ambisonics rendering for the occasion, using a dual-band third-order decoder designed by Fons Adriaensen. It consists of 25 Genelec 8240A in three rings of eight plus a zenith speaker, and four 7270B subwoofers. With a reverb time of only 0.3s down to 150 Hz, the room was the most effective in transporting the free-field acoustics of the garden. It was surprisingly free of phasing artefacts, probably due to its ring structure.

The most recent replay took place at ICSA 2011 [15 International Conference on Spatial Audio 2011, Detmold, Germany. http://icsa2011.org ], on a rig of 40 Neumann KH120 monitors and four KH810 subwoofers designed by the author, again using a third-order decoder by Fons Adriaensen. This rig was vastly over-specified for third order, and even though the decoder used only a subset of speakers, the first trial run sounded phasy and irritating. It became clear that the native Soundfield signals in Chroma had to be kept separate from the HOA spots and needed a dedicated first-order decoder that used even fewer speakers. With this treatment, the result was as good or better than the previous renderings. The lesson is clear: mixed order content must not be lumped together into a single higher-order bus.

All these replays have sounded different, which is to be expected given the extreme variance in room acoustics, layout and speaker design. On more than one occasion, the author found himself longing for a mastering step and clearly defined “industry best practices” for listening room acoustics. But even if the portability of Ambisonic mixes is not perfect, a fully satisfactory rendition could be achieved in each case, with only minor adjustments.

Conclusion and future work

Chroma is a perfect example for the artistically and aesthetically justified application of periphonic audio. The composer has been extremely happy with the listening experience and expressed her surprise at the precision and sound quality with which the live experience was reconstructed. She suggested some minor changes in balance where the mix strayed from her personal “favourite path” through the work, but was otherwise fully convinced of the technology and the execution. Workshop participants throughout were likewise convinced of the feasibility of the method, and found the music adequately represented.

To address the mixed-order problem, sharpening techniques such as the HARPEX decoder [16 Svein Berge and Natasha Barrett, 2010: “High angular resolution planewave expansion”, in: Proceedings of the 2nd International Symposium on Ambisonics and Spherical Acoustics, Paris. ] could be evaluated to “upmix” the soundfield signals to third order.

Direct comparison of the hybrid HOA recordings with the IOSONO renderer and material created with the SPAT toolkit from IRCAM at the recent ICSA conference revealed other limitations. The virtual sources of Chroma never seemed to move more than a few meters away from the speakers, whereas the IOSONO system had no trouble vanishing the walls of the listening room altogether and suggesting a room that was vastly larger. The IRCAM SPAT workshop demonstrated the necessity of going for higher orders: it drove the rig at its maximal resolution, which resulted in greatly reduced phasing, and emphasized the need for a flexible room synthesis algorithm in the HOA recording tool chest.

Further study is necessary to determine the role of the floor reflection picked up by the spot mike, which might dominate distance perception and might account for the observed lack of depth (see Rumsey [17 Francis Rumsey, 2001: “Spatial Audio”, Focal Press, p.35 ]). It could be eliminated by absorbers on the floor, or by designing a suitable inverse filter.

After clearing out all the equipment, we enjoyed a gorgeous live rendition of Chroma, free from technical anxieties, as part of the festival crowd. And when the moon rose over the double bass player while the nightingales were singing and the summer scents of the garden drifted by, it became all too obvious that even the most sophisticated recording techniques can be ridiculously inadequate.

I like to point out that we did get the nightingales, though.

Acknowledgements

Chroma XII was composed and directed by Rebecca Saunders, commissioned by Kunst­festspiele Herrenhausen 2010 and performed by musikFabrik Landesensemble NRW e.V.

Live recording by Jörn Nettingsmeier and Florian Faber, mixed by Jörn Nettingsmeier.

Thanks to:

Florian Faber, without whom the project would have been dead in the water

Stephan Flock and Claudio Becker-Foss for generous hardware support

Rebecca Saunders and Mark J. Baden for their friendly cooperation despite the tight rehearsal schedule

Lukas Hellermann, Michael Bölter, Martin Schmitz and all the musicians of musikFabrik for their curiousity and open-mindedness

Lilly Marie Weber at Kunstfestspiele Herren­hausen for the permission to record

Florian Trötschel and crew at Galerie Herren­hausen for competent and friendly local tech support

Technicolor Hannover for helping hands, interesting research and useful gear

Winfried Ritsch, Matthias Franz and Thomas Musil at IEM for the invitation to work there, and for their patience and support

Peter Plessas for a very well-tuned concert P.A. system at dafx10

Pierre-Alexandre Tremblay and Prof. Michael Clarke at Huddersfield University for the invitation to work at the SPIRAL

Fons Adriaensen for a huge body of excellent free Ambisonic software, lovingly hand-crafted decoding matrices, and for answers to countless questions.

Recorded and mixed with free software, thanks to all authors and project contributors.

Financial support by musikFabrik, Technicolor Hannover, and IEM Graz.