Programming
Article

How to Build an AR Android App with Vuforia and Unity

By Srinivas Rao

Android AR apps with Vuforia and Unity

  • How to Build an AR Android App with Vuforia and Unity

Virtual and Augmented Reality has been getting a lot of attention lately. Lots of really cool Head Mounted Devices (HMDs) like Oculus, Hololens, HTC Vive, FOVE, and Magic Leap have been coming up. The simplest and cheapest of all these HMDs is Google Cardboard, and it already has more than a million users worldwide.

Google Cardboard

Patrick has covered Building Google Cardboard VR Apps in Unity in his recent post. In the following two tutorials, we will be developing a basic Android Augmented Reality application for Google Cardboard using Unity3D. We will be using the Augmented Reality SDK named “Vuforia” to make this app.

The following software and the corresponding versions will be used in the following two posts:

  • Vuforia 5
  • Unity 3D 5.x
  • Cardboard SDK version 0.5.x (needed for next tutorial)
  • Android Studio and SDK tools (these need to be installed and set up for Unity builds to work. More information about this can be found in the “Integrating with Unity” section.)

Brief Overview

There are several approaches to making Augmented Reality apps. We will be using a marker based approach for making our AR app for mobile devices. In the next tutorial, we will be adding cardboard support.

Marker/Image Target – this is an image which is recognized and tracked by a mobile device’s camera. This acts as an origin for augmentation. Once recognized, content can be displayed with respect to it.

This tutorial will broadly have three parts:

  1. Getting started with Vuforia
  2. Integrating with Unity
  3. Demo (Fun part)

Getting started with Vuforia

To get started, we will need to go to Vuforia’s Developer Portal and create an account.

Once logged in, we can proceed to the “Develop” page. To use Vuforia with Unity, we will need to do two things:

  1. Create a license key for our application

    A license key is a unique ID which is required to create an app in Unity which uses Vuforia. To create a license key, we need to head over to “Develop >> License Manager” and select “Add License Key”. We will be prompted with a page like this one:

    Add License Key

    Once we fill in the details, we can select “Next” and then “Confirm”. This will direct us back to the “License Manager” page where the application’s name will appear.

    License Key Created

    Once we select “ARDemo”, we will get the license key. This key will be used later in Unity.

    License Key code

  2. Create a Database and an Image Target

    The next step is to set up an image as a target. We head over to “Develop >> Target Manager” and select “Add Database” and fill in the details:

    Create Database

    Next, we will need to add images to the newly made database. So, we select the database and click on “Add Target”. Vuforia supports various kinds of targets like single image, cylindrical, cuboidal, 3D image etc. For this tutorial, we will be using a single image target and the stones_hires image. Any arbitrary value can be selected for width, say 50.

    Add Target

    After a short upload time, the target will be added and we will have a “Download Dataset” option.

    An important thing to note here is the “Rating”. A good rating means that it can be used for tracking, whereas a bad rating means that the image does not have enough feature points (interesting points in the image like edges, corners etc which are used for tracking in image processing).

    Rating

    We can also have a look at the feature points of our image by selecting “stones_hires >> Show features”

    Show features

Let’s download the dataset by selecting “Download Dataset >> Unity editor”.

Download Dataset

This will generate a Unity package containing trackable information about this trackable database.

Integrating with Unity

Unity is a powerful and widely used game engine. Recently, it has also been emerging as the game engine for HMDs. The tutorial assumes very basic knowledge of Unity. You can find some more interesting tutorials on Unity here.

Let’s start by creating a new Unity3D project by the name of “ARDemo”.

Project Created

Next, we will have to import the Vuforia Unity packages. This can be done by going to “Assets >> Import Package >> Custom Package”. We will have two packages, one corresponding to our dataset (Image_Targets.unitypackage) and the other corresponding to Vuforia 5 SDK for Unity (vuforia-unity-5-0-5.unitypackage) which can be found here.

Packages Imported

Time to add some 3D models to our project. Let’s proceed to “Window >> Asset Store”. Here we can find a Samurai model by the name of “Red Samurai”, for free. We will have to log in with our Unity Account to download and import this model.

Red Samurai

Once imported, our Assets folder will look like this:

Samurai Imported

Next, we will have to add an ARCamera to our scene. “ARCamera” is an Augmented Reality camera prefab from Vuforia and is very similar to a regular camera. We can find this under “Assets >> Vuforia >> Prefabs” and drag it into our scene at X=0, Y=0, Z=0. Let’s also delete the Main Camera from the scene (it can be found under the Hierarchy Panel, “Right Click >> Delete”) as we already have an ARCamera.

For our ARCamera to run, we will have to add the App License Key. In the Inspector panel, we will have to paste the unique key we had previously received from Vuforia’s License Manager.

AR Camera Added

At this point in time if we press the Play button in the Unity Editor we should be able to see our WebCam feed in the Game View.

Next, let’s add the ImageTarget to our scene. It can be found under “Assets >> Vuforia >> Prefabs”. Let’s add it to our scene at X=0, Y=-50, Z=0. Now, in the Inspector panel for the Image Target we will have to make some changes. Under the “Image Target Behaviour (script)” we will have to change “Data Set” to “Image_Targets” and “Image Target” to “stones_hires”. Once done, the stone_hires image will appear in our Scene View.

Image Target Inspector Panel

Next, let’s drag our Samurai onto the Scene. It can be found under “Assets >> prefab >> samuzai”. For the samurai to appear over ImageTarget, it needs to be made a child of ImageTarget (can be done by just dragging the samuzai prefab in the Hierarchy panel). Then, whenever the ImageTarget is detected by a mobile device’s camera, all the children of the target will also appear together. The scale of the samuzai needs to be changed to X=0.5, Y=0.5, Z=0.5.

Samurai Added

Now we need to activate the Image_Targets Dataset. This can be done by selecting “ARCamera”, and in the Inspector panel, under “Dataset Load Behaviour (Script)”, we need to select “Load Data Set Image_Targets” and then “Activate”.

Dataset Load Behaviour

At this point, if we select “Run” and point a print out of the image towards our webcam, we should see the samurai on it and doing “Attack animation” which is its default animation.

Samurai animation

We are almost done. Let’s save the scene with the name “ARDemo”.

The last step is to build the project for Android. We need to go to “File >> Build Settings”. We need to add the current scene by selecting “Add Current”. Then, we need to select a platform (Android). Here, we will have multiple options:

  • Google Android Project: This will allow us to export the current Unity project to Android Studio so it can be edited and used to add more elements.
  • Development Build: Enabling this will enable Profiler functionality and also make the Autoconnect Profiler and Script Debugging options available.

We will be making a simple build as follows:

Build Image

Before pressing the Build button, we need to make some more changes in the “Player Settings” options in the Inspector panel. Here, the company’s name needs to be changed; e.g. to “SitePoint”. Next, we need to change the Bundle Identifier under “Other Settings” say, to “com.SitePoint.ARDemo”.

Image Company's Name

Image Bundle Identifier

Now, we can proceed to “Build and Run”. Other changes like Minimum API Level etc. can be made as per additional requirements. We will need to connect an Android mobile device via USB and enable USB debugging.

We will also need to have an Android developer environment set up before we can test our Unity games on the device. This involves downloading and installing the Android SDK with the different Android platforms, and adding our physical device to our system (this is done a bit differently depending on whether we are developing on Windows or Mac).

Since, this is a complex process, a basic outline of the tasks that must be completed before we can run code on our Android device or in the Android emulator has been provided in the Unity docs. However, a more detailed guide can be found in the first part of Patrick’s tutorial on VR.

The Final Result (Demo)

Finally, the fun and magical part!

Once the app builds and runs on Android we can point the mobile device’s camera to the marker and watch the animation. This is how the animation looks in my Nexus 5.

Output gif

The AR app will work even with a black and white image of the stones because the feature points will still remain (they usually depend on other factors like gradient, etc. instead of color).

Conclusion

Lots of really cool stuff can be done with Unity and Vuforia. In this tutorial, we have made a simple AR app with animation without writing a script! In the followup tutorial, we will be adding cardboard support and some more interesting Vuforia features to our app.

If you get stuck at any point or want to view the source code, you can find it on Github. You will just have to enter your Vuforia App License Key to build the app.

If want to download or test the app you can find the apk here.

If you make something cool out of this tutorial, or have any queries/feedback feel free to leave a note in the comments below!

  • http://www.imobdevtech.com/ Dev Patel

    Really informative post. I like it.

    • Srinivas Rao

      Thank You!

  • http://SalaryNet30.com Theresa Gay

    In today’s life everyone knows that money is very important, so I want to make everyone’s life easier and provide you financial freedom by telling you about a project that is paying me $10 k or more every month by doing simple tasks that anyone with basic computer skills can do and you need only good internet connection

    ~~check`my` ~~http website~~~listed~~~on~~~my~~~~{PrIvate}`~~~~~`page` ~~~~~~~!~

    **%*%*%^*%^*%^*%^*%^*%^*%^*%^&*%^*%^*%^&*

  • Aleksander Koko

    Have worked with this sdk some weeks ago. It is really really nice. Looking forward for other people to use this. Nice tut :)

  • http://www.mobileapptelligence.com/ kailash mobileapptelligence

    Android App Development Company – With android OS commanding a massive lead in usage, global businesses need professional Android app development company as partner to draft & execute mobile strategy. At MobileAPPtelligence, we deliver the apps, which keep our clients ahead of the curve, Mobileapptelligence [dot] com, an award winning Android app development company, is delivering best in class Android apps to global clients.

  • manoj

    sir iam not able to download the sdk for unity pls help me asap it just shows License Agreement

    In order to continue downloading, you need to agree to the terms below.

    Print Decline I Agree when i click agree a file of 0 bytes as sdk gets downloaded
    thanks for your tutorial easy to follow and understand

  • manoj

    thanks for your reply i have downloaded through mozilla it was not working on chrome dont know btw thanks

  • afiq

    hi , i have a problem, i cant view the project via webcam in unity, it came out black screen whenever i click play. one more thing. does vuforia 5.0.6 support unity 5 windows(64 bits). thank you in advance

  • Ricardo Nogueira

    Hello, i’d like to ask how you got the model to animate. What settings do you have in your prefab? Although i loaded an .fbx to unity, i can’t get it to animate in the AR scene.

  • anon

    Hello, firstly, thanks for writing such an excellent post!

    I have the same problem as afiq. I’ve followed the steps you suggest but it shows the background colour still, not the webcam. What should I do next?

    I’ve looked at a lot of threads on Unity but can’t find anything. Do I need to install flash or something like it?

    • Srinivas Rao

      Can you try building the scene for PC made by the steps I suggested above? After building it try running the executable file on a different PC having a webcam. This will help in figuring out whether the problem is your PC specific.

  • anon

    If you’re getting the black screen you need unity3d version 5.1.0 32 bit

    That got it sorted for me

  • asri edi

    hi , i have a problem. MissingMethodException: Method not found: ‘UnityEngine.Behaviour.get_isActiveAndEnabled’.
    Vuforia.VuforiaAbstractBehaviour.UpdateStatePrivate (Boolean forceUpdate, Boolean reapplyOldState)
    Vuforia.VuforiaAbstractBehaviour.Update ()
    thank you.

  • deepzk sahdev

    hi i need to develop an app in which furniture can be rotated for 3d view.please help me with the technology set i shall be using and role of each

  • afiq

    hi, i got another problem on exporting the apps. after i build and run, it says “unfortunately,(app name) has stop working. can you help me please..

    Thank you in advance.

  • deepzk sahdev

    Thanks Srinivas, but in my usecase I want to showcase a real furniture.what steps do you suggest in that case.

    • Srinivas Rao

      @deepzksahdev:disqus Can you explain your problem in a more detail? My understanding of the problem is that you have 3D models of real furniture and would like to augment them say in a room using AR markers. Is this correct?

  • Jovis Joseph Aloor

    sir you told that there willbe a follow up tutorial for this ..is it already live,then i couldnt find it..please do help

    • Srinivas Rao

      The tutorial is almost complete. It will take couple of days to get it peer reviewed and posted online. I am really sorry for the delay caused in making the follow up article. I was busy with other works. I would appreciate if you could wait for couple of more days.

  • Prodromos Lilitsis

    I am looking for a similar tutorial with Cloud Dataset using Vuforia 5 for Unity, any ideas please?

  • magda saif

    thank you for reply , about these information( that will appear if i clicked button )where cen i store it and how to connect it with project ?? note that , places that will be detected stored in vuforia account in net…do i stored these information in sql data base or what ??

  • Srinivas Rao

    Can you download the github code mentioned in my tutorial and check whether the stones image is working? Can you also tell me the star rating for your images?

  • tafadzwa joseph Dube

    hie great tutorial thanx a lot. Now can you kindly assist on how i can be control a character using arrow keys. for example the samurai character or even a 3D sphere , controllling its movements on the augmented reality scene using key arrows. if they are tutorials to this effect kindly direct me to them. thank you very much

  • magda saif

    ok , thank you so much for helping me :) @disqus_tKPKWDLGFq:disqus

  • Alex

    Beautiful tutorial. One question: is it possibile to open a photo gallery using Unity3d?

  • fraizor

    I made an account only to thank you.
    Thank you very much your greate tuturial helped me a lot , keep on the good work

    • Srinivas Rao

      Thank You!!

  • Al

    hI, is there any way to download the entire 3d model from the cloud?

  • Varun Khare

    is there any other good and free option to unity for linux

  • Kamel Bakir

    HELP, ? i did everything like the tutorial but when i run it , the webcam works and i can see the sumurai in my cam but i have a warning saying that trackable stones_hire lost, Sethint,UnityEngineDebug .log (object) but when i build it and run it in my device its not working at all , i cant see the sumuai , PLZ help me

    • Kamel Bakir

      everything is fine now thank u so much

  • Kaveh Khorramian

    Hi Srinivas, Thank you for the workflow, it is really nice.
    I’m new i using Unity+Vuforia, I got a problem during a project, maybe you can help me. I want to use Vuforia to detect and scan an image but later instead showing augmented reality show an animation which I captured before, Is it possible? Do you have any idea how can I do that? thanks a lot in advance.

    • Srinivas Rao

      Yes, you can show an animation but if you don’t want to use augmented reality then why not use a qr code based thing? You can simply make a qr code and on detecting it you can trigger your animation.

      • Kaveh Khorramian

        Yep, The problem is how can I detect the QR code? because its necessary to scan the QR code then show animation. I couldn’t find any Tutorial. I appreciate if you sent me some links or tutorials.

  • Kaveh Khorramian

    Big thanks Bro, I solved my problem somehow, but I’m curious about your suggestion as well, I will try it.

  • Jadie Naicker

    Hi Srinivas. Thank you so much for this tutorial. Everything worked perfectly, I have the app installed on my device but when I try to scan the image, the samurai doesn’t show up. Will it only work on a printed image on paper or can I scan the image on my PC? Thank you again.

    • Srinivas Rao

      It should also work by pointing the device’s camera to an image shown on PC screen.

      • Jadie Naicker

        Thank you. So since it is not working, something went wrong. Do you the typical errors or should I retrace all steps? Thanks

        • Srinivas Rao

          I would suggest downloading the apk mentioned at the end of the tutorial and trying to install it. That should definitely work, if not then it may be a mobile problem. If it works then you can try retracing steps.

          • Jadie Naicker

            Thank You. Will try it out nd give you feedback

          • Jadie Naicker

            The demo works so it must be something incorrect in my build. I will redo. Is it possible to build a front end where you can simply upload the image target (stones hi res) and the associated animation/media (samurai)?

  • MyLaptop Samsung

    hii.. so how about Android Augmented Reality application for Google Cardboard using Unity3D tutorial??

  • dikri reksajaya Wiguna

    you give thanks tutorial very helpful.. :)

    • Srinivas Rao

      Thank You!!

  • Jadie Naicker

    Is there a way to privately message you Srinivas? Hangouts?

    • Srinivas Rao

      Yes, you can message me on Hangouts.

  • Aamir Azhar

    What to do if I want to increase the size of samurai

  • Mark Pyrozen

    Is there a way that the application would display a different 3dmodel when you scan another image while still being able to display the previous model when you scan the image for it?

  • Tamuka Caswell Chikanyairo

    how are you sir ,Im actually planning to do my final year project in augmented reality being a n e commerce app with that touch and feel augment reality app whats your suggestion on resources ,,,,,caswelltamuka@gmail.com

    • Srinivas Rao

      Hi, what are you exactly planning to make? Do you want to work only on Android or are you open for other platforms?

      • Tamuka Caswell Chikanyairo

        I want to make an android app but now I have narrowed my research I think Im more familiar with the tools Im now focused on either the education sector or medical

  • Mark Pyrozen

    Thank You very much. Now I have a problem regarding my device camera not focusing when using the app and trying to scan an image. I have tried using the scripts in the developer help section and placed my script on the ARCamera but it still wont focus.

  • Srinivas Rao

    The marker is the image which is being tracked. Like in the final demo gif, marker is the stones_hires paper.

  • Afro

    hello sir, whether it can be implemented on a web application?

  • Justine Francisco

    Will this work on unity 5.0.0 with vuforia

    • Kiptoo Towett

      yes. I ran it on 5.4

  • Kiptoo Towett

    Amazing tutorial. Thank you very much Srinivasan.

  • chamath

    is there a way to show image instead of 3D samurai ?

    • Srinivas Rao

      Yes, you can make a plane and apply an image as its texture. This plane can be treated as a 3D object.

      • chamath

        thanx . is there a way to maintain database for it?
        is that so , can you please tell me what kind of tools or programing language i should use.
        i am new to augmented reality

  • TimeTraveler

    Thanks for such a wonderful tutorial…

  • Eduardo Hillebrand

    Sorry, i don’t know what im doing wrong, i’ve spent the entire week to make this work and nothing happend.
    everything was updated and is not exactly equal to tutorial but is very similar.
    i don´t know if i miss something but the red guy not show up.

  • Eduardo Hillebrand

    don’t work in webcam even.
    and yes, works fine on phone. :/

  • Eduardo Hillebrand

    With your image works fine.
    With mine own 5star-rating image don’t.

    • Srinivas Rao

      With your own image you will have to upload it on developer portal, download the dataset again and set your image as the marker. Did you do all these steps properly?

  • chamath

    thank you sir , but i want to create this way project . there is a mobile app to read image from augmented reality, and a desktop machine and desktop application to assign images to that target image(QR code, name perhaps ). we should see those assign images from that same target that desktop application assigned (if desktop application assign image to “1.jpg” image target then our mobile application can see that assign image through “1.jpg” using augmented reality), and also desktop application can assign new image to same target by removing previous assign image.

    also is there a way to get those details to mobile app without using internet . by using wifi network can we get values that desktop application provide assign image to target ?

    if there a way please let me know sir it is huge help….

    • Srinivas Rao

      Hi, sorry for the late reply (Somehow, I don’t seemed to have received a notification about this message). I don’t understand your query completely. Can you explain it again?

  • Avatar

    Where can I find the second part?

  • Ahmed Nasser

    How are you sir, so i’m wounding how to recognize large buildings with Victoria ,What’s your suggestions

    • Jorge Obregón Pérez

      i am interesting too, if you know how to recognize large buildings, please share de info

      • Srinivas Rao

        Hi, what exactly do you want to do with large buildings? Do you just want to recognize them or do you want some kind of AR as well?

  • Kanagalingam Jothi

    That’s an awesome tutorial! how can i make a touch listener to the animation added? Do i need to modify Script files? It would be helpful for beginners like me if you can make a tuto on this!

  • Fabian Tjandra

    Hi ! I’ve been using your tutorial and can make AR camera works. I already download the material, and the samurai show’s up on my image target. But his skin/material is missing, and I’m completely never use unity before soo.. I just followed your tutorial, and I got this missing skin, can you help me kind sir? :D

  • Alvin Hernandez Revilas

    Hello,

    I am using Android Studio 2.1.3 and I’m not getting the camera at all on my test device, a Verizon Ellipsis tablet. When I run it on my laptop inside Unity, it works fine. When I try to import to the tablet, the camera doesn’t even activate. The imported app is only a “Hello World” program, with the imported file name as its title.

    What else needs to be done so that this sample program can work??

    Sincerely,
    Alvin

  • http://www.behance.net/naqiuddinihsan Ihsan

    Thank you!

  • Alvin Hernandez Revilas

    Unity is 5.4 and Android is 4.2.2, so those are supported. The above app works on the tablet. So, it looks like camera APIs are not working. So, is there a fix you know, or should I Google it?

  • Ajit

    Good one. I would give it a try.

  • Fabian Tjandra

    Hi! sorry for my late reply, my model only show black skin because I don’t give any light on my scene lol, my bad, never use unity before. Anyway, I’ve talked to the Vuforia marketing, then they say, even if I make simple app that recognize around 4-6 image target, I get charged full price, what’s your opinion sir? Can I have your personal contact such as mail if you don’t mind?

  • http://www.xdx.cba.pl Damian Kuciński

    There is a lot of tutorials made about it but they always left you with lots of hard question and there is this one. Complete, great tutorial! Thank you soooo much, I made my AR App and it works perfectly, thanks to you!

  • Srinivas Rao

    Hi, Unity is a game engine with same interface for Windows as well as Mac. The tutorial should be the same for both of these platforms.

  • Srinivas Rao

    Hi, it can easily be done with the cardboard SDK and Vuforia. Pokemon Go is basically like an object drifting on a camera background. It does this with gyroscope which tracks rotation. But, gyroscope being a sensor has an error which causes the drift in the object. So, when you combine AR camera and Cardboard SDK, you need to put your 3D object(samurai) as the child of ‘CardboardMain’ and on the same level as ‘Head’. This should do it.

  • Vinoth Kumar

    Hi sir,
    I am using Android studio on Ubuntu operating system i dont know how to make augmented reality apps in android…..if import existing projects some errors will be occurred…please can i have clear assistance from you step by step

  • Nur Athirah Rosli

    hi sir i wanna ask u when at the build setting stage why my unity cannot switch the platform from pc,mac & linux to android.,?

Recommended

Learn Coding Online
Learn Web Development

Start learning web development and design for free with SitePoint Premium!

Get the latest in Front-end, once a week, for free.