Try-on glasses with ARCore Augmented Faces

Following Try-on makeup tutorial, in this tutorial we will make a try-on glasses app using Augmented Faces API from Sceneform. You will learn how to use 3D models and apply those for creation of face filters.

Project setup

If you are new to ARCore and Sceneform, check out this Augmented Faces codelab to get you up to speed.

Let’s find and download some 3D models of sunglasses. I use Poly and here are the models for this tutorial:

Prepare your 3D model

Download canonical_face_mesh.fbx from Sceneform github project and open the model in Blender or any other 3D modelling software

Instructions for Blender:

  • File - Import - FBX (.fbx)

  • Delete Cube, Light and Camera object that are created for every new project

  • File - Import - FBX (.fbx)

  • Import “Yellow glasses” model

  • Zoom out to see the model. Now we need to adjust sunglasses model to fit out the face model.

  • Following the instructions from ARCore documentation, add the glass model under “asset“ object. Check out this short tutorial to learn about parenting in Blender.

Glasses as a child of canonical face mesh object

  • Delete “facemesh“

  • Export the end result: File - Export - FBX (.fbx)

Now do the same procedure for gray glasses model.

Adding 3D models to Augmented Faces

First we will create ModelRenderable for our 3D models:

Next step is to find all AugmentedFace “trackables” and attach our ModelRenderable as faceRegionsRenderable. For full code sample, check out the source code:

Change ModelRenderable at runtime

In the sample app I added a simple functionality to switch between models at runtime. The main idea is to go through all existing trackable faces and change faceRegionsRenderable to a new model:

Previous
Previous

Face Landmarks for ARCore Augmented Faces