Instagram - like filter "Which ... are you?" with ARCore Augmented Faces

You might have seen many filters on Instagram recently that will tell you “Which pokemon are you?” or “Which Happy Potter character are you?”. In this short tutorial, I will show you how to make filter like this with ARCore Augmented Faces API.

There are several ways to solve this challenge and for this filter we will use face region poses to position a question view. According to the ARCore documentation, there are 3 face region poses available at the moment:

• Left forehead (LEFT_FOREHEAD)

• Right forehead (RIGHT_FOREHEAD)

• Tip of the nose (NOSE_TIP)

We are going to use left and right forehead pose to calculate the position of the view and ViewRenderable to create the question view.

Create custom AugmentedFaceNode

We need to override onActivate and onUpdate methods in AugmentedFaceNode. cardNode will be a child of FaceNode and will contain a ViewRenderable as a renderable object. Check out onActivate method for more details. onUpdate method will be used to update the position of our cardNode.

After several tests I found this magic formula for positioning of the card using LEFT_FOREHEARD and RIGHT_FOREHEAD poses (0.05 is for padding between the face and the view ) :

In the sample code below you can find an example of custom AugmentedFaceNode implementation. Check out the final source code linked below for the full demo code that includes quiz animation and data.

Apply the filter

To apply this filter to a face, we need to create FilterFace object and set the scene as parent in onUpdateListener as follows:

Final result

Check out the source code below for the working demo where you can find more details on how I created animation and the layout: