Peter Jackson’s Beatles Documentary Used Artificial Intelligence To Perform Restoration Magic

In August of 2020, Disney announced that the theatrical release of The Beatles: Get Back, Peter Jackson’s highly-anticipated documentary, would be delayed from September of 2020 until August of 2021. The entertainment juggernaut hoped that the pandemic would be under control by then, allowing for a successful run in theaters. In 2020, I wrote the following: “It seems like a lifetime has passed since we could crowd into a movie theater without worrying about the public health crisis that could result from such brazen behavior. Will we be ready for The Beatles: Get Back when it hits theaters a year from now?” We were not, in fact, ready. The documentary was ultimately released, not as a feature-length film in theaters, but as a 3-part docu-series on the Disney+ streaming service at the end of November, 2021. The series was celebrated by fans and critics alike, though some felt that the runtime of nearly 8 hours was a bit much. Personally, I couldn’t get enough. Thanks to the wisdom and influence of my Baby-boomer parents, I’ve been a huge Beatles fan since before I started kindergarten. For me, watching them in such an intimate way, as they created musical masterpieces in a matter of days, was nothing short of mind-blowing. Having read a fair bit of the pre-launch press surrounding the documentary, I knew generally what to expect. Using over 55 hours of raw footage originally shot for Michael Lindsay-Hogg’s 1970 film entitled Let It Be, along with over 140 hours of unreleased audio recordings, Peter Jackson lovingly stitched together what he hoped would be a more complete narrative of the 21 days of rehearsals and recording sessions that would eventually result in the Beatles’ legendary rooftop concert in 1969, and the group’s final studio album in 1970. While Michael Lindsay-Hogg’s film depicted the high-running tensions of a band on the verge of collapse, Peter Jackson’s more fleshed-out version of events provides context and counterbalance. Yes, there were disagreements and snarky comments. There were moments of frustration and awkwardness. George Harrison famously quit the band (only to be convinced to return by his fellow Beatles). But there was also musical magic being made, and there was clear evidence of the affection, camaraderie, and mutual respect that allowed the group to ascend to such great heights in the first place. John and Paul goofed off with an almost relentless, adolescent glee. George showed genuine interest and support as he helped Ringo write “Octopus’s Garden.” Paul playfully joked — in a moment of uncanny premonition — that 50 years in the future, people would say that the Beatles broke up “because Yoko sat on an amp.” Amazingly, most of this footage sat locked away in a vault for over 50 years before Peter Jackson was given the green light.

I am really happy that Peter has delved into our archives to make a film that shows the truth about The Beatles recording together. The friendship and love between us comes over and reminds me of what a crazily beautiful time we had.

— Paul McCartney

As a Beatles fan, I was utterly rapt from the first frame. (And as a wannabe songwriter, I was positively infuriated to watch Paul write the song “Get Back,” almost as if by accident, in a matter of minutes. George wrote “I Me Mine” while watching TV!) As I mentioned earlier, I generally knew what to expect in terms of the content of the documentary, but I was not at all prepared for the stunning picture quality of the decades-old footage, nor for the you-are-there realism of the sound. How can I describe it? It was as if the Beatles were right there. Still in their 20s, looking real and alive and accessible, as if the footage had been shot yesterday. Like most Audioholics readers, I appreciate great image quality and accurate sound in any situation, but I was frankly stunned by the degree to which the immaculately-restored footage seemed to reduce the distance between me and the moments in history that I was being given the privilege to witness. Whenever I’ve seen footage of the Beatles — even from other recent projects, such as Ron Howard’s excellent 2016 documentary following the group’s early years — I’ve always had the sense that I was seeing something archival — a glimpse, from afar, of events that took place long ago. Peter Jackson has described the look and feel of aged, archival footage as being “like a barrier between us and the actual people that were being filmed.” Watching The Beatles: Get Back doesn’t feel anything like that. It’s more like living inside a transcendent episode of Doctor Who. You step inside the Tardis, and emerge into the vibrant, colorful, and effervescent world of London in 1969. As Peter Jackson himself put it, “You never once feel this footage is 52 years old.” My girlfriend, Hannah, was instantly obsessed with George Harrison’s voguish wardrobe, and perhaps a little too taken with Paul McCartney’s flawless complexion, rosy cheeks, and shiny hair. The experience was all the more remarkable for viewers who had previously seen the original 1970 documentary, or even just clips of the rooftop concert. Originally intended for a TV special rather than a feature film, the footage was shot on 16mm film and then blown-up to 35mm. It was grainy, with muted colors and a rough-around-the-edges quality, even before it sat in a vault for half a century. So, how was this miraculous transformation achieved?

In 1969 that film had a quite chunky, grainy, desaturated look to it. One of the purposes was to try to restore it sort of making it look as natural as possible. Suddenly the colors were just unbelievable. People say, ‘So how did you do all those colors?’ And I’m saying, we didn’t do the colors, they were there.

— Peter Jackson

The Beatles: Get Back was not Peter Jackson’s first restoration rodeo. In fact, it was Jackson’s previous restoration work that led to his involvement in the Beatles project. In 2018, his World War I documentary They Shall Not Grow Old garnered attention first and foremost for its pioneering restoration of 100-year-old footage. Using cutting-edge technology and purpose-built software, Jackson’s team at Park Road Post Production (a subsidiary of his production company WingNut Films) colorized and restored footage taken on the front lines of the Great War, making it look shockingly crisp and modern. The film caught the attention of Jeff Jones, CEO of Apple Corps Limited — the multimedia corporation that has handled The Beatles’ business affairs since the Fab Four started the company in 1968. Jones, along with Apple’s director of production Jonathan Clyde, reached out to Jackson and asked if he would be interested in working his magic on some Beatles footage. But at first, it wasn’t a documentary that they had in mind. Jackson revealed in an interview with The Guardian that Apple Corps was exploring the possibility of creating an interactive museum experience, perhaps inspired by similar exhibitions celebrating The Rolling Stones and David Bowie. The folks at Apple were drawn to Jackson not only because of his impressive restoration work, but also because of his interest in virtual and augmented reality. Those plans have been shelved, according to Jackson, but his relationship with Apple Corps led to discussions about the fabled footage from the Let It Be project. It has been public knowledge since the year 2000 that Apple Corps still has reels of footage from Michael Lindsay-Hogg’s contentious film (which has been out of circulation for years). When Jackson was finally given access, he realized that it had the potential to reshape history.

I just can’t believe it exists. But then I can’t believe any of it – that the Beatles let Michael shoot all that footage, that it sat in a vault all this time. What other band in the 60s or 70s allowed themselves to be shot in such an intimate way? There isn’t another. And then I got to edit it. The whole thing boggles my mind.

— Peter Jackson

The restoration work reportedly took a team of 14 people at Park Road Post Production almost four years to complete. According to Jackson, his “department of computer effects guys” is the best in the world. Jackson himself admits to being woefully untechnical, but he described the process of restoring old film thusly:

You take this film, and you figure out what’s wrong with it… and try and correct it then. You can’t actually just do it with off-the-shelf software, so one of the things we do have there is a department of code writers who write computer code in software. So largely what we do in the films we make… is we would take off-the-shelf software that’s available, and you often write sort of a plugin. If there’s a particular thing that you want to do that the software isn’t capable of doing, then you have your own department of software writers to actually author… plugins that you can plug into the commercial software.

Jackson further discussed the process in a conversation with Kara Swisher, host of the podcast “Recode Decode,” from the Vox Media Podcast Network.

The quality is totally dependent on the amount of manpower or labor that you put into the job. I do love the idea that we’re using all of this (modern) technology, but we’re using it to go back in time. I don’t sit in bed at night and think about the code and the software and the pixels. I don’t think like that. All I think about is story and character, and I think what the technology is, is just a step, it’s a technique that you have available to tell a story. So I don’t get obsessed with the technology.

Jackson may have been a bit vague on the technical details, but the results of the visual restoration speak for themselves. Jason Perlow, Senior Technology Editor of the business technology news website ZDNet, called Park Road Post’s restoration work on The Beatles: Get Back “an actual watershed moment for technological advancement.” But of course, the picture is only half the story for any film, and for a Beatles documentary, sound plays an especially important role. The music was overseen by music supervisor Giles Martin, son of the legendary Beatles producer George Martin, and himself a regular producer of Beatles projects since 2006. But before Giles could have a say in how the music for the film should be mixed, Park Road Post once again had some very tricky obstacles to overcome. Luckily, the team was up to the challenge. (Park Road Post handles sound for Jackson’s feature films, such as The Lord of the Rings: The Return of the King and King Kong, both of which won Academy Awards for sound mixing.) The most daunting problem was presented by the Beatles’ rehearsals at Twickenham Studios, which took place during the first week of January, 1969. The only audio recordings from these sessions were the unmixed, monaural recordings captured by the camera crew’s relatively unsophisticated audio gear. Later, when the Beatles had relocated to the newly-built recording studio at Apple’s Savile Row headquarters, the group recorded everything to eight-track, using an array of studio mics, with the help of recording engineer Glyn Johns and producer George Martin. But at Twickenham, John, Paul, George, and Ringo were captured on a single track — their voices and instruments haphazardly squashed together. According to Jackson, the sound was “all over the place.” The vocals were drowned out by the guitars, and Ringo’s drums were either too loud or nearly inaudible, depending on the location of the microphone.

Even their speaking voices were sometimes hard to parse. Never passing up an opportunity to be cheeky, John and George would toy with Lindsay-Hogg by purposefully drowning out their conversations. They’d turn up their amplifiers and strum tunelessly on their guitars, not playing music, but just making a sort of sonic camouflage to cover up their speech. Jackson said that you could see the Beatles having a private chat on film, but all you’d hear was loud guitar. At one point in the documentary, George cracks a coy smile and facetiously says to Lindsay-Hogg, “Oh, you’re recording our conversation?” In 1969, this tactic was successful, and the conversations couldn’t be overheard by the camera crew. But thanks to some innovative work in artificial intelligence and machine learning, Jackson’s team was able to shine new light on these indiscernible chats, and make sense of the “all over the place” audio from the group’s rehearsals.

To me the sound restoration is the most exciting thing. We made some huge breakthroughs in audio.

-Peter Jackson

Jackson’s team utilized a technology called “demixing,” which, according to the director, is “an idea that’s been around for a while,” but has never been used with this level of success. The clever people at Park Road Post built custom software using an AI-based machine learning system that allowed them to teach the computer “what a guitar sounds like, what a bass sounds like, what a voice sounds like. In fact we taught the computer what John sounds like and what Paul sounds like.” Given an adequate education, the software was then able to analyze the mono tracks from Twickenham and split them into multiple tracks, thus allowing the music to be remixed cleanly, as if the voices and instruments had all been recorded to separate tracks.  

We can take a mono track of them in Twickenham performing and we can say, ‘Just give us the vocal track’. And the machine learning will render a vocal track only. So, you’ll see Ringo in the background playing the drums, but you don’t hear the drums. Not one bit of drum noise – just the vocals, clean. So the big breakthrough for us was actually not (restoring) the pictures, even though that’s what you obviously look at, it was the sound. The way that we managed to split off the mono recordings in the moldable tracks.

— Peter Jackson

This breakthrough technique also allowed Jackson to strip the guitars away from the private conversations that the Beatles had under cacophonous cover. Jackson admits that exposing these conversations was perhaps “a little bit naughty,” but insists that the resulting scenes are “some key parts of our movie.” Paul, Ringo, Yoko Ono, Olivia Harrison, and Dhani Harrison were all supportive of the project, which will be remembered for reshaping our understanding of the interpersonal relationships and creative processes that resulted in the Beatles’ final studio album, and the group’s final live performance on the rooftop of the Apple Building in London’s Savile Row. Years from now, the film may not be remembered for its ground-breaking restoration techniques, but I think it’s fair to say that these technological advancements played a big part in making The Beatles: Get Back such a magical experience for Beatles fans everywhere.

To learn more, check out this short clip about the restoration process:

Restoring History | The Beatles: Get Back | Disney+