The Evolution of Apps — Figr Budtender
The evolution of apps has made combining AR and reality a feasible task. Kit Kong, our Interactive Developer, walks us through how this came into play while we developed the Figr app.
Apps have come a long way. They started as flat, 2D programs with rudimentary graphics and have evolved into fully fleshed out, 3D environments that rival graphics on console game systems. With this growth trajectory, apps have reached beyond their own 3D world and encroached upon the real world, which is now Augmented Reality (AR).
Through AR, 3D apps are now merging with the real world, which includes the massive improvement we’ve seen in camera optics on devices. Not only are 3D elements capable of living in the same space as a camera, but they’re also capable of recognizing the real world for what it is with object and plane recognition. 3D objects can now be poked, prodded, placed and moved in relation to the real world.
The Ask
With this idea of combined realities in mind, we were tasked to create an AR app with Wunderman Thompson for Figr (a company that creates hand-crafted cannabis products). Figr wanted an educational app to showcase their products and to highlight the steps they take during production. After much deliberation, we opted to build our app in Unity3D with the Vuforia package. Unity3D with Vuforia provided all of the foundations for our needs including cross-platform compatibility between iOS and Android, as well as AR capabilities and image targeting.
The Work
To give users a more personal experience, we decided to create an AR app that they can explore in their own environment. The AR portion of the app was sectioned off into two main sections, the Product View and the Grow Room, with each demonstrating a different aspect of AR.
The Product View
The first section of the Product View is a comprehensive introduction to Figr products, which provides a 3D digitization of the plant buds that users can look at and learn about. The experience is triggered with a scan of a Figr product label or a predetermined selection, in case the user doesn’t own a Figr product. The plant bud model they see is superimposed over a live stream of the camera feed, so it looks like it exists in the real world. Information, like where the plant was sourced, the chemical composition, flavours and aromas, is overlaid over the bud. In addition, we were able to provide users an approximate distance from the source of the plant thanks to geo location tagging.
The Grow Room
The Grow Room is where users are able to interact more with the plants grown at Figr. At this stage, they’re given a choice of plants to investigate and are instructed to place the plant into their real world environment (with real world plane tracking). This essentially anchors the plant into a position in the real world that’s recognized by the camera and software. The user is then able to move around and get closer or further from the plant by physically moving, as if they’re viewing a real plant. Brief information panels are presented on top of the plant and linked to specific parts to highlight where the different qualities of the products are derived from.
This marriage between 2D overlays and 3D assets overlaid and superimposed on a live camera feed presents the user with a mixed reality experience that further enhances the believability. In addition to the AR experiences, the app also gives users a chance to familiarize themselves with the company in the About Figr section, as well as register for an email newsletter.
Innovation Never Stops
All in all, the process of creating an AR app has drastically changed thanks to the numerous readily available tools. Unity with Vuforia is one combination that provides developers with many additional capabilities on top of the existing Unity game engine. More platform specific tools, like Facebook’s Spark AR and Snapchat’s Lens Studio, gives developers more specialized functionalities such as face tracking and background segmentation with more restrictions like file formats and file sizes.
Whichever tool developers choose to use, AR capabilities are constantly improving as we speak. I’m excited to see what the future holds with the continuation of boundary-pushing innovative ideas.
Want to learn more? Download and check out the Figr Budtender app for Apple and Google Play.
