Opinion Pieces

Opinion Pieces

Sound for Moving Pictures

Dr. Neil Hillman is a man with a ready smile; perhaps surprisingly given the kind of year 2020 turned out to be for the film and television industry (with postponed and cancelled projects across the board), but he is upbeat about the future: ‘First and foremost, I’m very happy to still be actively involved with sound, almost 40 years on from when I first entered a university radio studio as a teenager and realised that I’d found my calling; and it was in audio. Secondly, I look at it as I was gifted time this year to finish the book without quite so many distractions in the studio… Not so much that the glass is half-full or half empty, just thankful that there is something actually in the glass!’


Those decades have certainly spread across some interesting technological developments in audio; and Hillman’s involvement in production sound - as a drama and documentary location sound recordist on many household name television series’, as well as British independent feature films - and his work as a Supervising Sound Editor, a Dialogue Editor and a Re-recording Mixer through his audio post-production company The Audio Suite, has ensured that he remains current and abreast of kit, trends and best working practices.


But it is his PhD in sound design that has spurred our conversation; or more specifically, it is his forthcoming book, Sound for Moving Pictures - The Four Sound Areas. It is the result of thirty-plus years of professional practice alongside nine years of self-funded, part-time doctoral research and part-time teaching in higher education, whilst continuing full time as an audio professional. Published by Routledge this spring, the book would seem to be perfectly timed to surf what seems to be a new wave in cinema sound; and in particular, a renewed interest in the delivery of emotive sound design by people outside of the sound department.


‘My PhD research took a much closer look than I initially planned at how we humans react to emotional stimuli. My initial driver was curiosity. As a mixer and an editor I wanted to know if there were consistent triggers that would evoke intended emotions in an audience: at its simplest, “edit like this and people will feel sad, mix like this and people will feel happy…” That kind of thing. But of course, human emotions are an incredibly deep and interesting field of study in their own right; so it wasn’t possible to answer that question without first embarking on some detective work to better understand the nature of human emotions, and ideally, how those emotions relate to the way we experience, and are affected by, moving picture media.’


Given the heavy-lifting that needed to be done in researching the degree, Sound for Moving Pictures still manages to remain readable, successfully providing a bridge between the needs of an academic and a practitioner. By exploring the necessarily complex concepts in an accessible way, the reader is taken on a guided tour of Hillman’s solo journey to his PhD. But that does not mean that the book is light on assembling a serious and comprehensive overview of our current understanding regarding emotions.


‘My training is as an engineer, not a scientist; but I became absorbed by the work being done on human emotions by some fascinating people, big names in the scientific world such as Antonio Damasio and Lisa Feldman Barrett, Paul Ekman, Karen Collins and Vilayanur Ramachandran. It was so interesting to see that in parallel to their work, what we do as sound designers, editors and mixers by instinct, figuratively and literally in the dark, these people were cataloguing and defining. So then I looked for convergence between the science and the deep thinking film sound theorists such as Michel Chion, David Sonnenschein, William Whittington and Mark Ward; and I began to properly appreciate how much and how far Walter Murch has brought the art of picture and sound editing to the notice of the academic world. Practitioners who want to be taken seriously by academics owe a great debt to Walter’s evangelizing.’


So how does all of this translate into writing a book that has an ambition to become an indispensable filmmaking treatise?


‘I felt it would be helpful to shine a light on some very relevant science for practitioners, and in doing so I wanted Sound for Moving Pictures to be a book on sound that would be as useful and meaningful to a film director or picture editor, as it was to a sound editor or a Re-recording Mixer; becoming a platform that would become the common ground, or the creative meeting point, for both sound professionals and non-sound professionals to construct, deconstruct and communicate together the emotional aspects of soundtrack design, in ways and using terms that both parties could understand. The sound design framework that came out of my research, and which is described in the book as the Four Sound Areas, provides a practical, working template for precisely this. By breaking down the aural content of a scene into its Four Sound Areas constituent parts, what I have called a soundtrack’s Narrative, Abstract, Temporal or Spatial elements, a full soundtrack can be built in such a way as to link the emotional intention of the director, the picture editor or the sound editor to the types of sounds used by the sound editor in the track lay; and the emphasis placed on certain aural components in the mix could be plotted and steered by the Re-recording Mixer. For the benefit of academics and students, I wanted the Four Sound Areas to be a way that they gained a clearer understanding of how the magic of a beautifully balanced mix occurred. Because for the Re-recording Mixer it’s a visceral experience: you know it’s right when the hairs stand up on the back of your neck! It’s totally instinctive, therefore it’s hard to explain how or why that is.’


Hillman’s Narrative Sound area is concerned with sounds that are used to communicate meaning or insight. Dialogue and commentary are important examples of this area in the sound mix, but it may also include certain diegetic music, as well as symbolic and signalling sounds such as the ringtone of a telephone, ambulance sirens and so on, as their meaning is clearly defined, almost like a language.


The Abstract sound area is concerned with sounds that are less codified in their meaning, such as atmospheres, backgrounds, room tones and synchronous and non-synchronous sound effects; as well as abstract diegetic music, where the music is being used as an emotive, atmospheric device rather than for signalling something specific.


The Temporal Sound area is concerned with the temporal evolution of the soundtrack, through rhythm, pace and punctuation, such as the non-diegetic music score or a specific sound design element with a strong rhythm, and it is characterised and contrasted by the difference between high rhythm, fast pace, high structure and, conversely, slow pace, loose structure and low rhythm.


The Spatial Sound area is concerned with both the positioning of sounds within the soundfield of film and television programmes, and the space placed specifically around the presented sound: from the convention of on-screen, front Centre-focussed monophonic dialogue, the front Left-Right stereophonic imaging of events to the left or right of the projected image, through to enveloping atmospheres placed in the surround channels, or specific out-of-vision sound effects, spot effects or dialogue, behind or above the audience.


Music neatly takes its place in whichever sound area is the score’s primary function. Utilising the book’s unique ‘Mix Wheel’, a visual aid of twenty emotional terms (based on the Geneva Emotion Wheel), and its companion ‘Mix Disc’, another visual aid with four coloured quadrants, it is possible to identify and communicate where in the emotional spectrum a particular scene is intended to sit. By a Re-recording Mixer then considering the balance of which sounds of the four sound areas should be prominent in the mix - or in the case of a sound editor building a soundtrack, what types of sounds need to be present - a soundtrack strategy can be established. Using the framework also means that the sound department can become involved much earlier in the post-production process, working with the picture department and the director on the first cut; instead of the usual ‘spotting session’, much later in the process.


The book offers practical examples of how the Four Sound Areas can be seen at work, by using the framework to reveal the way that the soundtracks are constructed on films like Ingmar Bergman’s Winter Light, a monophonic, art-house film from 1963; Francis Ford Coppola’s pioneering surround-sound epic, Apocalypse Now; as well as Steven Spielberg’s Minority Report and Lars von Trier’s Dogville.


The author also offers insight into his own work as a Sound Designer and Re-recording Mixer using the Four Sound Areas, with analysis of three of his own feature films, and detailed notes on him sound designing and mixing for sports Outside Broadcasts.


Whether you’re an experienced film practitioner or a student at the beginning of your career, an academic looking for a deeper understanding of sound design or a teaching professional designing a Master’s degree module, or even a lecturer or student involved in the wider topic of film studies, there is a wealth of theoretical, practical and applicable knowledge contained within this educational, informative and entertaining book.


Other Opinion Pieces