Tutorial: Image Recognition - How to use markers to showcase a product in AR
Viro’s mission is to enable AR/VR everywhere by building the tools developers need to create amazing apps. The more we understand the real world, the better our AR experiences will become. With image recognition, developers can find and recognize the position of 2D images in the real-world and respond to them in their apps. Image recognition is a big step toward unlocking new AR experience for users such as interactive museum exhibits, dynamic movie posters, visual navigation overlays and 3D product showcases.
This tutorial will show you how to use a logo as a real world target to showcase a product in AR. We will use the cover of the Tesla owners manual as our marker and once recognized we will make a 3D model of Tesla Model X appear above the marker.
Download or clone the files and assets located here -> ARCarDemo.
Setup
If this is your first time as a Viro developer, use the quick start guides below to get setup in minutes. The Viro platform is free with no limits on distribution:
You will also need either an iOS ARKit enabled device or an Android ARCore enabled device.
(Note: This tutorial is for ViroReact (React Native) developers. If you are interested in Java development, check out ViroCore and the code sample located here -> ARTesla)
Open ARCarDemo.js to view the complete scene for the Tesla demo. We will breakdown this file and show you how the Tesla demo is created. (If this is your first exposure to AR, consider reviewing this tutorial, How to build an interactive AR app, to understand some of the basics of AR app development.)
Recognizing Images
In order to use Image Recognition, you need to:
1) Specify the target image and its properties
2) Add a ViroARImageMarker component with the created target
With Viro, we specify our image target usingViroARTrackingTargets which contains the information required for AR tracking components to work properly. Image targets have the following properties:
Source — file path of the image we want to detect in the real world. In this demo, we are using the Model X Owner’s Manual cover as our target.
Orientation — the orientation of the source image.
PhysicalWidth — The width of the image in the real world in meters. This value is important for better tracking and estimating depth.
The ViroARImageMarker component allows developers to add components which will be hidden until the target image is found, at which point, the components will be rendered relative to the found target. This allows the developer to not only know that the target image has been found, but to enable them to provide a visual response relative to the target’s position/rotation in the real world.
In ARCarDemo.js, we start with ourViroARScene (line 40) and then we add aViroARImageMarker (line 44) within that scene. We define our image target (lines 219–225) and name our target = “logo” in line 220. Below is a simplified version of the code showing ViroARImageMarker and ViroARTrackingTargets:
Adding 3D Objects to the marker
Now that we have specified the image we want to recognize, we want to define what occurs once the image is found. To make the car appear, we add our Tesla model within our ViroARImageMarker component (lines 82–108). The car model is the Viro3DObject while the ViroSpotlight and ViroQuad create a shadow for the car to add realism to the scene. Any components contained within ViroARImageMarker will appear in your scene once the image is recognized.
For a better user experience, we add an animation to our Viro3DObject (line 90) so the Tesla does not just “pop” into view. This animation is triggered by onAnchorFound (line 44) which executes when the target image is found in the scene, animating the car model (lines 114–118). We use a scale animation called scaleCar (lines 232–233) to animate the Tesla outward from the marker. In this animation, we are scaling the Tesla from its original scale = {[0,0,0]} (line 83) to scale = {[0.09, 0.09, 0,09]} over 500ms using a “bounce” easing. You can read more about these properties in our developer guide on Animations.
Highlights of other demo features
At this point, you should understand how to set up image recognition using the Tesla demo as an example. The key is to specify the image you want to recognize and then define what action occurs once the image is found. Some other features in the demo are quickly highlighted below:
- Sphere Color Switcher — use spheres as controls (lines 45–80) to swap textures on your 3D model (lines 166–196).
- Billboarding a node — use
transformBehaviorsto billboard any component to always face the camera (line 45) - PBR and Image Based Lighting (IBL)— Physically-based rendering, or PBR, is a collection of rendering techniques that produces more realistic lighting results for your scenes by incorporating an advanced model of real-world lights and materials. IBL refers to the techniques used to physically simulate ambient light. These techniques makes the Tesla appear more realistic and natural within your scene(line 42).
Next Steps
Extend the Tesla AR Marker Demo and build your own AR application. Here are some things you might want to try:
- Add snow or fireworks to your scene. Check out our Particle Emitters Sample Code and read our Particle Effects Guide for info on how to accomplish this.
- Check out our Black Panther Movie Poster Demo to see another example of Image Recognition -> ViroReact Github | ViroCore Github
