PC Virtual Reality, Augmented Reality through an IOS and Android app
December 18, 2019
Unity/ C#/ Shaders/ Lighting/ Adobe Suite/ Figma/ Git Hub
The interactions consisted in the form of points of interest. The user could move around the waka to these different points and pick up, inspect, use or play with the different items around the waka. We knew the experience was going to have a lot of younger people coming through so we wanted to make these simple but also informative to really put in in the space of the waka
For the virtual reality experience the client wanted it to be available for both PC and mobile VR (using the gyroscope on the mobile phone to allow the user to look around their space). This meant we needed to design it with scalability in mind making sure it could perform on and look good on both platforms
To optimize the virtual reality experience a creative solution was employed to reduce texture memory usage. By packing all textures needed for the waka onto only 3 trim sheets, a balance was struck between quality and resource efficiency. This approach was made possible by collaborating with waka experts to accurately depict various weathering effects on the waka, which were incorporated into the material trim sheets. The simple composition of the waka from raw materials made it an ideal candidate for this technique.
In order to enhance the virtual reality experience the stars were featured as a key element that appeared on the horizon just before dawn. This time of day was selected as the starting point for the experience and it progressed through until after the sun had risen. The timing allowed for optimal use of baked lighting with a mid-point just before dawn, ensuring the lighting appeared natural. To further refine the scene, a small global tint was applied in the shader to adjust the final color of all objects to match the slight shift in light.
To accurately position the stars in the virtual reality and augmented reality experience, we utilized an app called Stellarium and developed a custom plugin to export a cube map. This cube map was then imported into Unity and integrated into the sky, ensuring the stars were in the correct position and the sky rotated in a realistic manner.
For the augmented reality component of the project, content was displayed on the interior of a truck. This required tracking of large image targets, sometimes as big as 2x2 meters. To overcome this challenge, we needed to find a way to combine multiple image targets into one, providing users with the ability to view content from a distance and also up close without losing tracking. To accomplish this, we extended the AR plugin being used and added the capability to break down a large image target into smaller chunks. This allowed for increased detail and tracking points in each chunk while still preserving a large tracking space.
Below is an example of single image chunks being used. Each one of these chunks is about 2x2 meters and because it is feed in as one image it lacks detail and tracking points
And below we can see an AR wall scale example. This tracker is made up of multiple images allowing the user to move in closer but will also act as one unit if more than one is in view
Here is a video showing how the content can be viewed from far away but also has the ability for the user to go up close without loosing tracking of the scene.
As we were building this for a truck that we could not get access to until a week before it went live we decide to build a virtual replica of the truck in order to allow us to visualize the space and test content. This proved to be extremely useful and we were able to go inside of it in VR to better understand the scale and prototype out interactions
