This is Shawn: The Experience

Canada’s Shawn Mendes, made famous by his covers on Vine, is now an international star who writes and performs his own songs. With this massive trajectory, it seems like there’s nothing holding him back (we had to).

The Ask

This is Shawn, presented by Verizon, was created to allow fans to get a first-hand look into his creative process and experience it themselves. Our task? Work with Momentum Worldwide to build a fully immersive and interactive installation to give participants an experience that exceeded their expectations. We also designed and built a microsite to drive traffic to the event This Is Shawn.

What We Did

The exhibit included four key rooms, which organized and tracked the user journey of each participant. At any given time, there could be 5–10 fans in each space.

Registration

We built the registration webpage, as well as QR and scanning functionality. All participating fans registered with a BA and received a unique QR code (sent to their phone using Twilio) before making their way to the next room.

Inspiration Room

When the user scanned their BA at the Inspiration Room, a video text of a personal welcome message from Shawn was sent to their phone via Twilio. The Inspiration Room was designed to look like the Malibu seaside, which is where Shawn wrote “In My Blood”. It included a floor-to-ceiling video wall, sand, plants, rocks and a reflective mirror wall.

Production Room

This room was designed and built to be a replica of Shawn’s studio. It featured instruments, a mixing board and two recording booths. Fans scanned their QR codes and were given the opportunity to record their own rendition of “If I Can’t Have You”. Participants wore headphones and followed the lyrics displayed on the screens. They were able to do a practice take before the real deal, which was captured on video.

Backstage and Stage Room

The final room gave fans the chance to be part of a backstage experience, as well as feel what it’s like to perform in front of thousands of people. The stage was decorated with a baby grand piano and a beautiful towering rose, which was designed to look like the actual stage Shawn performs on. Participants were surrounded by a 360 degree projection of a packed stadium, full of people screaming and singing along to “Life of the Party”. The Stage Room was set up with three cameras, which were synced to the projection to capture photos at key moments.

As fans exited the Stage Room, they checked in with one final BA and scanned their QR code. This triggered a summary email to be sent to them, which contained their Production Room video and photos from the Stage Room.

How We Did It

Microsite

The microsite, built on React, included a custom Google Maps integration and a service for adding the event to your favourite Calendar app. It held users’ mobile experience passes with obfuscated URLs, as these needed to be publicly accessible for each user to receive via text message. It also contained users’ video content so they could view it or email it to themselves as an attachment in order to download it on iOS devices.

Registration Website

This was built on React and included a digital signature integration (using Canvas) to save users’ signed waiver forms. The form was submitted to an AWS Lambda function that generated a new custom QR code, saved the signed waiver to AWS S3 and stored user information in Firebase. After this, participants received a text message with a link to their unique mobile experience pass, which they scanned at each station to identify themselves. Fields included name, email, phone number, carrier, and social handles.

QR Scanners

The QR code scanners, built on React, were part of the brand ambassador app. Tablets were placed at each outpost, where a brand ambassador would scan a user’s QR code to send them exclusive content or queue them up for one of the two major experiential pieces. Once a group of users were scanned into one of these spots, the brand ambassador started the experience with the tablet and scanned in the next set of users.

Production Room

Lyrics

A karaoke-style video was created for users to follow along with the lyrics and record a portion of one of Shawn’s hit songs. Using WebSockets, signals were passed from TouchDesigner to a mobile phone in the room to keep the lyrics in sync on the phone, mimicking one of his recording sessions. Between recording sessions, the lyrics video was replaced with a live 4k video feed so participants could preview their positions on camera before the system began recording. Audio was streamed wirelessly and on individual frequencies into headsets for each booth.

Video

After their practice run, users did a live take. Using TouchDesigner, two Black Magic Studio Micro cameras recorded the sessions from two angles (via SDI) running into a capture card on the PC regulating the experience. Videos were encoded using cross-platform audio and video codecs, so the videos could be opened on any mobile or desktop device. After each participant completed their session, a pre-made bumper was attached to the end of the file. These files were saved directly to Amazon S3 using a virtually mounted network drive.

Audio

Using the mics found in Mendes’ studio, audio was streamed into the PC through a high quality external sound card and mixed onto the live video stream as it was being recorded. The mic’s audio was resampled at the correct sample rate to match the audio coming from the karaoke video. This allowed us to capture and mix the two audio sources with zero latency.

Camera Integration (Projection)

Several DSLR cameras were triggered via USB3 during the projection portion of the experience. Cues were set from timers, which controlled scripts to set custom locations for the media to be received on S3 and link newly captured media to a user’s current session. They also programmatically set exposure, shutter speed, iso and other camera settings dynamically with realtime readings of what was happening in the room.

Video (Projection)

The projection was powered by a Watchout system. Our TouchDesigner patch received WebSocket signals (from a React app the BA was using) to check people into a portion of the experience. Once triggered to start, Touch would relay this signal to Watchout via UDP and an intermediary AV bridge called Companion.

Summary Email

Upon exiting the experience, users scanned their QR code one final time. This allowed them to receive their user-generated content via a summary email. The email included their video from the recording booth, three photos from the stage experience and a unique link to send the video as an email attachment (to download it on all mobile devices).

The Result

The activation generated quite a bit of buzz and we saw lineups, full of Shawn Mendes super fans, that wrapped around the corner of the venue.

As proud Canadians, we were honoured to be part of this project and have the opportunity to highlight a fellow Canuck. Bringing a creative idea to life is always fun, especially when we’re able to give fans a chance to experience something out of the ordinary.