Welcome to part 3 of the render pass tutorial! In the last tutorial in Part 2, we went through how to use render passes for layering and here's a recap:
Background texture for a Scene Render Pass patch will always remain as the background behind the rendered object
The objects you want to appear on the top layer will always be the last to be rendered out and objects at the bottom-most layer will be the first to be rendered out
In this part 3 of the tutorial, we are going to go through how you can use render pass to scale down face meshes while ensuring it follows the face tracking proportionally with the background. And here's how it looks like.
For the facemesh material, we will be using this SDF animated texture we provided in our other tutorial previously, which is also included in the part 3 project file.
First, what we need to do is to create a normal full-size rectangle with 1 face mesh using the SDF animated texture as material. For the material type, we will be using physically based material at around 50% metallic and 50% roughness. Remember to also add an environment texture from the AR library when you are using physically based material.
Next, we use render pass to render out our full-size rectangle to the Device Screen Output. At the same time, we render out the face mesh with camera texture as background. However, instead of connecting the output texture to a Device Screen Output, we connect it to the material of the full-size rectangle as follow:
The end result is just a normal face mesh over the face as usual.
But why are we doing this? We will get right on to it. Now that we have rendered out only the full-size rectangle to our Device Screen Output, what will happen if we scale the rectangle down to half the size for both width and height?
If you haven't realised, we are already 1/4 way there like literally! The background is black because we didn't assign any texture to the background input, so it will take the colour input by default. Since the texture used for the rectangle is the rendered texture output of the facemesh with camera texture, we can adjust the size of the rectangle however we want and the facemesh will still stay proportionate to the user's face.
So, to achieve the effect we want, which is 4 equal rectangles at 4 corners, each with a different facemesh, we just need to repeat the steps above with 4 different rectangle and facemeshes.
Render canvas with 4 rectangles to Device Screen Output
Facemesh0 rendered out as material for Rectangle0 (pinned to top left-hand corner)
Facemesh1 rendered out as material for Rectangle1 (pinned to top right-hand corner)
Facemesh2 rendered out as material for Rectangle2 (pinned to bottom right-hand corner)
Facemesh3 rendered out as material for Rectangle2 (pinned to bottom left-hand corner)
Here's the render passes setup to render the 4 face meshes to 4 different rectangles. The one on the top render out a canvas, which consist of 4 different rectangles to the Device Screen Output.
And here's the patch setup for the material of 4 facemeshes using the SDF animated texture.
Here is the Scene setup. Only the Canvas has the purple arrow as we rendered the whole canvas, which consists of the 4 rectangles. Each facemesh is rendered out individually as we need it to be the texture for each of the 4 rectangles.
YAYY! Thank you for making it this far! Here's a recap as usual.
Output texture from Scene Render Pass can be used as a texture for any material
The rendered texture of facemeshes on a rectangle/plane can be scaled and still stayed proportionate to the user's face
Conclusion
Render pass may seem complicated at first, but it is definitely worth your time to learn about it so you can create not just cooler Instagram filters but also making your developing time easier! The best way to be better is to keep creating AR filters!
There are much more applications of render passes such as Frame Buffer and such. There are already lots of tutorials online but if you enjoy our article tutorial, do let us know so we might consider writing on ore topic in the future! OR, you can follow us on Instagram: @gowaaaofficial !
Again, you can find the working files for the 3 parts tutorial HERE.