Object-based immersive audio recording techniques

Advances in audio formats have been remarkable over time, from modest narrowband mono to complex high-definition multichannel systems. Nowadays, the evolution has led us to immersive audio, which goes beyond conventional stereo and immerses us in a multidimensional environment, from personal headphones to multichannel systems in cinemas or other large venues.

Immersive audio encompasses a wide variety of surround formats, from the traditional 5.0/5.1 to more complex ones like 7.1 and 9.1. Additionally, there are formats today that include height information, based on both channels and objects, which provide us with a completely immersive experience.

The basic and straightforward setup for channel-based 5.x (5.0/5.1/5.2) surround sound is the application of five microphones in a spaced array. There are different ways to select and position the microphones; this depends on many factors such as the acoustic qualities of the recording room (e.g., a concert hall/jazz club/church), the arrangement of the sound sources present, the directivity of the microphones used, or perhaps simply preference. Configurations can vary from mathematically calculated and psychoacoustically verified to more "emotional" ones.

A simple way to conceptually understand these techniques is to think of the coverage of a 360° circle around the listening position. We should consider every two neighboring microphones as a stereo pair, each pair covering a specific segment of the circle. Another way to see it, for example, would be to consider that the front microphones provide the main sound stage and the rear microphones provide a surround sound feel.

Nowadays, different techniques are used to record immersive audio. This article describes the most commonly used microphone setups by engineers around the world. It is important to define the listening configuration before choosing any recording setup. In broadcast and music, the starting configuration is ITU-775. 

Coincident arrays vs spaced arrays

An array is a physical arrangement of multiple microphones. The array can consist of several microphones mounted on a single stand or perhaps on multiple stands or mounts. In some models, the microphones are integrated into a single unit (such as the DPA 5100 surround microphone).

In a coincident array, the microphones are mounted very close to each other. In principle, all microphones of this type of array receive sound simultaneously.

In the coincident technique, localization is based solely on level differences between the signals. This technique can create adequate localization accuracy but little envelopment and will have a small "sweet spot" (on two axes). However, the advantage of a coincident array is that it is compact, portable, and mono-compatible. It is easy to mix the channels into a single channel without causing coloration and other artifacts.

A spaced array creates a three-dimensional surround sound feel by providing an adequate amount of decorrelation between the signals (the localization cues are based on time-of-arrival differences). By positioning the microphones (distance and angle) in the sound field, spaced arrays continue to provide adequate localization accuracy.

These techniques generally provide a wide and pleasant sweet spot and give listeners the sensation of a broad and enveloping soundstage. The downside is their size and, in some situations, the setup time. Additionally, it is not advisable to mix all channels into a mono signal, but rather to use an adapted mix of them.

5.X

The basic and straightforward setup for multichannel 5.x (5.0/5.1/5.2) surround sound is the use of five microphones in a spaced array. There are different ways to select and position the microphones.

It depends on many factors, such as the acoustic qualities of the recording room, the arrangement of the sound sources present, the directivity of the microphones used, and/or the engineer's preference. Configurations can vary from mathematically calculated and psychoacoustically verified to more experimental and practical ones.

Here are some common configurations that can serve as inspiration and will be a great starting point for a good recording.

Omnidirectional surround array

Composed of five omnidirectional microphones arranged in a spaced array, they provide a good tonal balance. The low-frequency content is reproduced very convincingly. This setup also provides excellent envelopment. When played back, the listener is surrounded by sound. The downside of this setup can be the lack of channel isolation.

The three front microphones are arranged in a Decca tree formation. The positions are chosen according to the optimal recording angle of the sound source.

The position of the rear microphones is chosen independently of the surrounding sound field. Normally, the rear microphones should not be placed too far from the front microphones. If the distance is too great, the delay may become audible. Additionally, some directivity may be preferable for surround pickup. This can be achieved using acoustic pressure equalizers (APEs), which ensure directivity in the higher frequencies but maintain the advantages of omnidirectional microphones for a good low-frequency response.

A good starting point for measurements for this configuration could be: Distance between the outer front microphones from 60 to 120 cm. The wider the source, the smaller the separation between the microphones should be. The center microphone is approximately 15 to 45 cm in front of the left/right pair.

The two rear microphones are placed 2 to 5 meters behind the front triplet. The distance between the rear microphones should be 2 to 3 meters. As mentioned, APEs can be used to prevent front transients from being reproduced by the rear channels.

The Scottish engineer Michael Williams has conducted numerous studies on MMAD or multichannel microphone array design.

Cardioid surround array

The array of five cardioid (directional) microphones has the advantage of greater channel separation compared to the omnidirectional-based array. To provide the correct coverage in the array, the microphones can be placed closer to each other, creating a smaller array. Of course, this can be taken to the extreme by arranging the microphones in a coincident configuration.

This would be a typical 5-channel cardioid setup, providing the same coverage in all segments of the circle.

Wide cardioid surround array

The “Wide Cardioid Surround Array (WCSA)”, introduced by Mikkel Nymand, provides equal tonal qualities, a high degree of envelopment, and good low-frequency properties.

To achieve the desired sound character, the five signals must be decorrelated. This means that the microphones must be placed at a sufficiently large distance, but on the other hand, the signals must not be too different (distant) from each other. If this occurs, the resulting sound will not be coherent.

Often, omnidirectional microphones are preferred in spaced arrays. This is due to their natural sound color and their ability to blend direct signals with the room's timbre. Wide cardioids (also known as sub-cardioids) have a slightly more directional quality, providing greater ambient control, frontal image, and localization accuracy.

The surround array created by Geoff Martin and Jason Corey uses one omnidirectional and one cardioid microphone to create wide cardioid characteristics.

Focusing on avoiding channel interference, microphone pairs are spaced 60 cm (left-center), 60 cm (right-center), 60 cm (front-rear), and 30 cm (rear left-rear right). The rear microphones used are upward-facing cardioids to capture height information.

DPA Microphones has adapted this array to use five identical wide cardioid microphones (adjusted within a very tight tolerance of ±1 dB in frequency response and sensitivity). The choice of five identical microphones instead of a specific microphone type maintains natural mixing and leads to more authentic and uniform reproduction of all channels.

After intensive listening sessions and numerous practical tests in different recording applications (symphonic music, modern jazz, PA/Live, pop concerts, and ambient recording), it has been found that this configuration tends to work best with greater separation, especially for the rear channels. This array creates intense, dynamic, and enveloping sound characteristics.

For large spaces with large formations of musicians (or large distances from the array to the source), this array can be expanded with two additional left/right omnidirectional outriggers to benefit from the low-frequency pickup of these microphones. When these microphones are mixed with the left and right channels of the array at an appropriate level, they provide a coherent, precise, and rich surround sound image.

Soundfield/Ambisonics

In the early 1970s, British engineers Peter Felget and Michael Gerzon invented the sound field principle later known as Ambisonics (Today known as first order ambisonics). The format is based on a coincident microphone array. The goal is to facilitate arbitrary orientation of microphones in any direction, left/right, front/back, up/down. Basically, the sound field principle works like an M/S, through the sum and subtraction of available signals. There are two configurations associated with Ambisonics: Format A and Format B.

Format A is the physical arrangement of four cardioid microphone capsules and their output: FU (front upper), RU (rear upper), LD (left lower), and RD (right lower). The angles between the capsules are consistent with a tetrahedron.

Format B is a converted version of Format A, resulting in a virtual format formed by three orthogonal-oriented bidirectional capsules: X (front/back), Y (side), Z (up/down), and one omnidirectional (W).

By adding and subtracting individual signals, it can be converted into a directional microphone. For example, an omnidirectional (W) and a bidirectional (X) create a cardioid pointing in the X direction.

DPA Microphones manufactured microphones for this format, but they no longer do.

Optimized cardioid triangle (OCT)

OCT is an array designed exclusively for the three front channels. The system offers great separation between the center-left and center-right. Careful consideration should be given to additional configuration for surround channels.

A cardioid microphone is used for the center channel placed just 8 cm in front of two hypercardioids for the left and right channels, pointing outward. The separation between the left and right microphones is key to the desired recording angle. Designers recommend distances between 40 cm and 90 cm, resulting in recording angles from 160° to 90°.

One or more omnidirectional microphones can be added to the system to compensate for the lack of low frequency from the other capsules.

Double MS

This is a compact and adjustable coincident surround sound configuration for immersive sound. In this setup, two cardioid microphones and one bidirectional microphone are used. Alternatively, the configuration can be created from four cardioid microphones. The principle of Double MS technique is one forward-facing cardioid and one rear-facing cardioid providing central information, sharing a bidirectional microphone for lateral pickup. In this configuration, signals need to be processed to create the final format and attention must be paid as there is a risk of frequency and phase response of sound reproduction being different from different positions.

This results in the following channels:

• - Center = Front Cardioid
• - Left = Front Cardioid + S
• - Right = Front Cardioid - S
• - Left Surround = Rear Cardioid + S
• - Right Surround = Rear Cardioid - S

Each signal mix is adjusted for correct spatial distribution, respecting the frontal image. Typically, the left/right amplitude is greater compared to standard MS.

Double MS technique can be achieved using four identical cardioid microphones 4011A or 4011C evenly paired and angled in the horizontal plane at 0°, 90°, 180°, and 270° respectively. Membranes should be placed one above the other to achieve the best temporal alignment in the horizontal plane.

In practical recording with a mixer, simply pan "left cardioid" to the left and "right cardioid" to the right and phase invert. The "dirty" way to do this is by using a Y-sum cable and phase-inverting the XLR connector for the right cardioid.

Fukada tree

Fukada Tree is a Decca Tree array, but with five cardioid microphones and two additional omnidirectional microphones as stabilizers to mix between front and rear channels. This configuration was designed by Akira Fukada in 1997.

The choice of cardioid microphones improves channel separation, and rear cardioids facing backwards also minimize direct front sound to the rear speakers.

Omnidirectional microphones are often preferred in Decca Tree setups for music recordings due to their natural color and full frequency bandwidth. The two omnidirectional extensions serve this important purpose in the Fukada array.

Since Akira Fukada introduced this design, he has made a series of positioning modifications to improve frontal localization, but his microphone choice remains DPA for its sense of transparency.

Hamasaki Square

Hamasaki Square consists of four bidirectional microphones arranged in a square.

It is designed to capture the ambient/diffuse part of surround sound recording. It is a square of four microphones with a spacing of 1.8 to 2 m between bidirectional microphones. These microphones point their sensitive directions in phase towards the sides and their nulls towards direct sound.

Compared to other ambient recording systems, this system is the least sensitive regarding distance between the main array and the ambient array.

This configuration has been developed by Japanese sound engineer Kimio Hamasaki.

Immersive audio with height

Configurations developed for traditional surround recordings (like 5.1) have proven to work very well. However, adding height to these recordings generates a very interesting effect during listening, as it adds a new dimension to the experience.

The challenge, however, is how to add upward-directed sound images without altering the perceived location of horizontally positioned sound sources, which means minimizing vertical crosstalk between channels. This requires consideration of vertical time and level differences. Also, the separation of vertical microphones necessary for decorrelation must be taken into account. Lastly, how to avoid comb filtering in subsequent downmixing to fewer channels?

When height information is added properly, it enhances the perceived envelopment created by sound. Moreover, this practice has shown improvement in perceived accuracy when localizing sound sources, even in the horizontal plane.

Dr. Hyunkook Lee from the University of Huddersfield (UK) and his research group have provided substantial theoretical and practical information on perceived sound imaging.

An important finding is that the precedence effect (the effect where the first arriving sound determines the direction) does not function in the vertical plane. Hence, it is crucial to focus on level differences. When the same sound is reproduced from both the lower and upper speakers, it was observed that higher frequencies and transient signals push the localization towards the upper speaker.

These findings have led to the microphone configuration shown below. It consists of eight cardioid microphones and two supercardioids.

The microphone orientation is such that frontal sound enters more attenuated in the upper layer of microphones. Overall, any microphone in the upper layer should receive as little sound as possible from primary horizontal sources and sources below the horizontal plane.

IRT Cross

IRT Cross is designed for capturing ambience. This configuration consists of four cardioid microphones.

Specifically, it is designed to capture the ambient/diffuse part of a surround sound recording. It is a square of four microphones with 20-25 cm between the cardioid microphones, pointing towards left, right, left surround, and right surround at an appropriate level compared to a frontal array.

IRT Cross is typically placed a couple of meters behind the main array. However, it should not be placed too far away as timing issues (like echo) in the reproduced signal can occur. Optimal placement of IRT Cross aims for a balance between capturing enough ambience and avoiding echo.

Object-based audio

For years, surround sound has been channel-based. For example, one channel for mono, two channels for stereo, six channels for 5.1, or 24 channels for NHK 22.2.

Conventions regarding speaker placement for each format have been the backbone of sound design. Panning between channels aided by delay or level adjustments has been the tool for placing sources in the sound scene. The final product will be contained within a fixed number of channels; although the program material will be originally recorded on a large number of audio tracks, the result fits into a specific number of channels, one for mono, two for stereo, etc.

Object-based audio (OBA) is different. A "sound object" can be recorded on one or multiple tracks. Alongside the audio are metadata indicating where to place the object in the sound scene.

An object could be a voice recorded in mono. If producers want the voice to come from the right part of the stage, the voice recording metadata contains coordinates for this sound.

In principle, an object can also originate from an ambisonic recording or any other format. Therefore, an OBA audiovisual program is built from a chain of objects, like voice recordings, music, ambient sounds, special sound effects, etc. Each object contains metadata about where in the sound scene it should be played back.

OBA Why?

The general idea is to allow more freedom to the listener, especially in broadcasting. Now, emphasizing a single object is possible. If a listener with hearing impairments wants to raise the level of dialogue, it's a possibility if you record the dialogue as an object. You can also change the language of the dialogue if you assign different language objects.

From TV productions like Formula 1 races, we know that special cameras on each car can be selected if the viewer wants to follow a specific car. The sound of that specific car is an object tied to the picture. Specific musical instruments of an orchestra can be considered objects. Alternatively, the sound of a concert, recorded at different listening positions, can be objects.

Another argument for OBA is that almost any playback format is valid. Downmixes of the main mix are optimized depending on the number of channels and their available playback positions (as long as the number of channels is at least two). Binaural playback is also allowed.

Microphones?

The basic idea is that the sound engineer can use whatever type of microphone they want. There are no necessarily specific microphone requirements, microphone configurations, or microphone brands. The special requirement depends on the production team, which sets the metadata and, of course, the final formats.

Recommended microphones and accessories according to capture techniques:

Omnidirectional-based surround array

• - 4006A Omnidirectional Microphone.
• - 4006C Compact Omnidirectional Microphone.
• - 5006A Surround Kit of five 4006A paired, clips, and windscreens in a Peli™ case.
• - S5 Surround/Decca Tree Structure.

Cardioid surround array

• - 4011A Cardioid Microphone.
• - 4011C Compact Cardioid Microphone.
• - S5 Surround/Decca Tree Structure.

Wide cardioid surround array (WCSA)

• - 4015A Wide Cardioid Microphone.
• - 4015C Compact Wide Cardioid Microphone.
• - 5015A Surround Kit of five paired 4015A, clips, and windscreens in a Peli™ case.
• - 4006A Omnidirectional Microphone.
• - 4006C Compact Omnidirectional Microphone.
• - 3506A Kit of two 4006A, clips, and windscreens in a Peli™ case.
• - S5 Surround/Decca Tree Structure.

Optimized cardioid triangle (OCT)

• - 4011A Cardioid Microphone.
• - 4011C Compact Cardioid Microphone.
• - 4018A Supercardioid Microphone.
• - S5 Surround/Decca Tree Structure.

Double MS

DPA does not offer any bidirectional microphone. The Schoeps MK8 with CMC6 preamplifier is suggested. However, if you wish to try this setup with DPA microphones, we suggest replacing each bidirectional microphone with two cardioid microphones:

• - ST4011A Stereo Pair of 4011A cardioids.
• - SB0400 Stereo Microphone Mount.
• - UA0836 Stereo Microphone Mount.
• - DUA0019 Stereo Microphone Spacing Tool, 19 mm.

Fukada tree

• - 4011A Cardioid Microphone.
• - 4011C Compact Cardioid Microphone.
• - 4006A Omnidirectional Microphone.
• - 3506A Kit of two 4006A, clips, and windscreens in a Peli™ case.
• - S5 Surround/Decca Tree Structure.
• - ST4011A Stereo Pair of 4011A cardioids.
• - SB0400 Stereo Microphone Mount.

Hamasaki Square

DPA does not offer any bidirectional microphone. The Schoeps MK8 with CMC6 preamplifier is suggested. However, if you wish to try this setup with DPA microphones, we suggest replacing each bidirectional microphone with two cardioid microphones:

• - ST4011A Stereo Pair of 4011A cardioids.
• - S5 Surround/Decca Tree Structure.

Immersive audio with height

• - 8 x 4011A Cardioid Microphone.
• - 2 x 4018 Supercardioid Microphone.

IRT Cross

• - 4011A Cardioid Microphone.
• - 4011C Compact Cardioid Microphone.
• - ST4011A Stereo Pair of 4011A cardioids.
• - MMC4011 Cardioid Capsule.
• - MMP ER/ES Modular Active Cable.
• - SB0400 Modular Stereo Mount.
• - UA0837 Stereo Mount.

DPA 5100 surround microphone

The DPA 5100 Surround Microphone is a perfect plug-and-play solution for numerous applications. It features three front-facing cardioid directional capsules arranged in a coincident manner. The rear channels are captured by a spaced pair of omnidirectional microphones. The unit also provides an LFE output. All channels are calibrated with unity gain. The LFE is reduced by 10 dB according to standard. The 5100 is highly regarded in cinematography for second unit work.