The Many Faces of Pride: An Automated Adventure in 4K Video Recording
Installations have a way of bringing out the best in a team. Through combined efforts from production, creative, development and onsite techs, our 4K testimonial booth at Toronto’s Pride celebration proved no different.
The Many Faces of Pride
Every year, Toronto’s Pride celebration attracts people from all walks of life. The LGBTQ+ community is as diverse as any and each individual has such a unique story about how they got to where they are today. With this in mind, Netflix and Sid Lee wanted to encourage people in attendance to share their experiences. By providing a comfortable and private interview environment, the hope was to inspire honest, candid responses to fifteen Pride-themed questions. The crowds of people would prove to be a major challenge, with upwards of 1.6 million people walking the streets during the parade and participating in the surrounding activities. The experience also needed to fit onto the packed intersection of Church and Maitland (at the centre of the festivities) and shelter participants from the sound and heat of the ongoing party outside.
The Challenge
Netflix and Sid Lee asked us to create an experience that could capture candid personal interviews from an unmonitored (on the inside) user-driven interview booth. This would be an exercise in automating the documentary filmmaking process, which would typically be carefully composed and monitored on site by filmmakers. The hardware has many options, buttons, and hidden menus, which would overwhelm the average person. Most people would be in over their head if they were asked to calibrate the system to a professional standard, let alone turn it on. The system would need to be adaptive enough to minimize manual adjustments during filming, while being structured enough to guarantee a certain level of quality.
The booth would shoot interviews in 4K from two different angles and archive full-quality 4K footage for professional editing during and after the event. There would also be a live preview of the video being captured to help guide the participant during recording sessions. The ultimate challenge would be to create a bridge between an interface anyone could understand and the professional video equipment. Although most of us can shoot a selfie from a camera phone, not many of us (in comparison) know how to adjust shutter speeds or aperture on a professional camera.
Finding the Right Gear
Not only would the system need to be automated, it would also have to fit into a footprint small enough to be installed on the street amongst other booths during Pride. The booth assembled for the experience would need to be a maximum of 80 square feet (8 x 10ft) and would need to reserve as much internal space as possible (to ensure the cameras could get shots wide enough to capture hand gestures and body language). These restrictions would prove to be challenging for many reasons.
In order to get a quality shot at this distance in a darker environment, we would need the perfect lens. To capture the participant sitting in the dim interview room, the lens would have to step down to a low enough aperture, while having a thick enough depth of field. This would ensure that the participant stayed in focus, no matter how they situated themselves in the interview chair. Technically speaking, these two things would work against each other — the more light the lens lets into the camera, the smaller the depth of field becomes.
Maximizing the space in front of the camera would mean minimizing the space behind it, but compact 4K recording hardware is an oxymoron, isn’t it?
Utilizing the Studio Micro and Hyperdeck Mini product lines from Blackmagic, we were able to find cameras and disk recorders small enough to bring a 4K hardware stack, typically reserved for studios, out onto the street.
Blackmagic gear is also fully controllable through external commands over their open control protocol via TCP and Serial. Traditionally, these interfaces are used in a live production workflow between cameras, live switchers, and various pieces of recording and networking equipment in a studio environment. Scrapping the bulk of a rack and network switch, the project opted for a simple wireless router. Surprisingly, the router ended up being the largest piece of equipment in the kit, other than the PC.
The Hyperdeck Mini had the added convenience of FTP file transfers. Since we were working with limited space, having the ability to dump recordings to a backup disk without entering the booth was advantageous.
Bringing It All Together
The greatest challenge was to design a user-controlled system that triggered the video production gear to record footage and receive video feeds back in real-time. In an attempt to minimize the decisions a participant had to make, the system locked most settings and options to optimal presets. This allowed participants to start, stop, and trigger playback of their latest recording.
The feed from each camera was recorded at 4K via the Hyperdeck Mini and then passed to a capture card. Once captured, it was streamed over RTSP (Real Time Streaming Protocol) to the booth iPad at a downsampled 1080p for live previews and reviewing captured footage after recording.
In order to successfully place control of the system in the hands of the participant, it was presented on an interface every participant was comfortable with — an iPad. Our team created two web applications using React.js: one for inside the booth (referred to as the “booth tablet”) and one for outside the booth (referred to as the “brand ambassador tablet”), which was handled by brand ambassadors
Booth Tablet
The booth tablet acted as an interactive teleprompter and was responsible for guiding the participant through the experience. By dynamically selecting new interview questions for the participant, it emulated the role of an interviewer in a documentary filmmaking process. The booth tablet communicated with the rest of the system over a Websocket and fired off messages as items were tapped on-screen. These signals let the recording hardware know when it was time to pause and play audio, start and stop the recording, switch from a live feed to playback mode, and some other rudimentary commands necessary for identifying the participant who was active in the booth. Each participant got a one-time preview of their recording before advancing to the next section. When the participant completed the experience and hit the submit button, it triggered a notification on the brand ambassador tablet to signal when it was time for the next person in line.
Brand Ambassador Tablet
The brand ambassador tablet was responsible for registering participants for the experience (using a digital form and waiver system), managing the digital queue, notifying the brand ambassadors when it was time for the next person to start the experience, and if there was a fault in the system.
The PC was the backbone of the system. It ran TouchDesigner and a Node.js server that had many connections of its own and communicated over the local network.
TouchDesigner
On the PC, TouchDesigner processed the signals coming from the booth tablet and sent TCP commands to the Hyperdeck Minis. TouchDesigner also downsampled and streamed the 4K footage over RTSP and ran various audio/video cues inside the booth. TouchDesigner was rock-solid for integrating a lot of different hardware and communications protocols into a single system — it was the central hub for all hardware and video connections. It relayed Websocket commands from the tablets to the recording hardware via TCP. It was also responsible for ingesting the live feeds from the recording hardware’s SDI loop out and streaming it back to the booth tablet. A user interface in TouchDesigner provided feedback on the remaining storage space of the high-performance SD cards in the Hyperdeck Minis and provided a handful of buttons to quickly reset the system in case of an error.
Our project resulted in the development of a reusable TouchDesigner component for interacting with the Blackmagic protocol in the future. Starting and stopping recordings was just the tip of the iceberg. The component can also make rough edits, drop clips onto a virtual timeline, insert timecode, and log metadata, amongst many more things. Theoretically, it could be expanded to full camera and lens control with the newly released SDI Arduino shield and the SDI embedded camera control protocol. This means that, in the future, there could be a director of photography working remotely making adjustments in real-time over the internet.
Websockets
A Websocket connection was created by the server and acted as a manager by relaying messages across the system while grabbing vital information, such as the digital queue when necessary. The server sent out heartbeat signals regularly to ensure all the system’s components were still connected. It also fired off warnings to the brand ambassadors and technicians if any part of the system dropped out.
RTSP
The server connected to the RTSP feed streaming from Camera 1 (head-on) via TouchDesigner, converted it on the fly with FFMPEG, and streamed it to the booth tablet over a Websocket (on a different port than the communications). The booth tablet consumed the stream using a Javascript library designed for Websocket streaming and playback.
MongoDB
A MongoDB instance ran on the PC in order to save users’ waivers and create the digital queue. The server captured users’ contact information and digital signatures via an API call and stored it in Mongo. Their digital signatures were stored directly on the server as an image and were marked with a unique id (generated during the API call) so they could be matched up afterwards.
The Result
By the end of the activation, we had recorded over 140 responses from members and allies of the LGBTQ+ community. This compilation delivered 9 hours of 4K footage to Netflix and Sid Lee to edit and distribute on any platform they choose.
Thinkingbox is really excited to work on projects, like this one, that have a positive social impact. We’re proud of how we were able to create an innovative way of capturing important narratives and presenting diverse perspectives. We hope this style of activation will be used to document the stories of marginalized communities for years to come.
