Audio description for 360-degree content

A 360-degree display supports multiple assets such as characters, props, space and design etc that benefit from additional description. Nonetheless, the delivery of content is still linear and the time available for adding audio description is limited. So, the question is how to make the audio description effective enough that the viewer is immersed in the content which is one of the aims of the 360-degree format.

Possibilities:

Script in first person

This is where the audio description is scripted in first person and the main character of the story becomes the describer. The style of writing and delivery is also similar to how the character sounds and speaks.

Example:

“I reversed away from him and he watched me go. I learned to drive with my knees while I played my guitar. There were four of us in the car now, Nat on guitar, Chuck on melodica, Big Sam on dashboard drums.”

Script in second person

This is where the describer is sitting next to the viewer or standing over their shoulder describing the scene the scene. Style of writing and delivery is casual, informal and friendly.

Example:

“She reverses the car. He watches her go. Later she steers with her knees, playing a guitar. There are three friends with her – a girl on guitar, a boy on melodica, while a bulky lad drums on the dashboard.”

Standard audio description

This is the standard audio description style where describer objectively sets out what is happening in the scene, the characters etc.

Example:

“She puts the car into reverse and pulls away from him. He watches her go. Later she steers the car with her knees as she plays her guitar. Three friends are with her, playing guitar, melodica and one drumming on the dashboard.”

One of the trickiest aspects of describing 360-degree content is predicting what the viewer will do next – for example, look left where one can see the snow-capped peaks of the Swiss Alps or right, towards a meadow filled with daisies.

So, what happens in these situations where the viewer has complete control over what they see and when they see it?

We considered further controls which allowed people to activate descriptions when they changed direction.

In this mode, there are three audio tracks:

1. Original soundtrack of the content
2. Audio description track
3. Additional description track which is the extended description.

The original soundtrack remains untouched.

The audio description track describes the five key elements of the scene – who, what, why, where and when in order of priority for understanding the storyline as in standard audio description.

The additional audio track or the extended description is used to set the scene and describe elements such as props, characters and their costumes, lighting etc or event background knowledge which would help people with sight loss visualise the scene. The rationale here being greater familiarity with the scene is linked to greater immersion.

This track can be triggered by the viewer when they hear a "beep" that is used to signal the presence of this additional track.

Audio introduction 

In addition to the above, an audio introduction could be useful. The viewer listens to the audio introduction before watching the content which sets the scene. It refers to details that the describer cannot include in the content because of the lack of time. These are quite common in the theatre world and are used to introduce to the viewers to the characters, their physical description, their costumes, the set design etc.

During our testing, we’ve discussed various factors that contribute to an immersive experience including how many voices would be considered too many in an immersive environment, use of language, intonation, delivery style and the importance of directionality and placement of the audio description in a 360-degree environment. However, it does seem that the 360-degree content opens up a whole host of alternatives on how audio description can be presented to the viewer and often a customised approach in which a combination of the scripting style and sound design would be needed to achieve the ideal experience.

Also, something to consider is the use of HMDs for people with significant sight loss who are unlikely to engage with content visually. Are they necessary? Perhaps, another device for head tracking may be more appropriate but further work is needed to investigate the viability of these alternatives.

Improving our understanding of how any form of access features i.e., the audio description could be adapted for 360-degree and VR requires working with content and different formats, whether it’s sports, music, documentaries, or short-form fiction. So, we request content producers to please help us out and share some samples with us.

Please drop us a line at [email protected] if you’d like to discuss this further.