Unreal Engine 4 introduced a new depth buffer along with their PBR rendering system. It’s called “Custom Depth” and can be used for effects like the selection outline that is built-in the Editor, rendering of occluded meshes or custom culling of transparency. I will discuss some of the basics of using this buffer in the material editor and explain some of the features I used it for in my own projects.
When using the techniques described for your own effect that utilize Custom Depth it’s important to read through the Issues & Limitations at the bottom of the page to prevent you hours of headache and unnecessary debugging.
Custom Depth 101
For those who have never used this feature before, you can enable it for both Static and Skeletal meshes under the Rendering category named “Render Custom Depth”.
To view this particular buffer go to Buffer Visualization in the viewport-options.
Note that objects close to the camera are near-black and may be difficult to see!
Rendering Object Outlines
To render outlines around specific meshes (like a fallen ally, usable loot container etc.) we need a buffer to get their silhouettes. Custom Depth is perfect for this.
With the buffer filled with our object depth info we can perform a simple depth comparison by sampling the neighboring pixels in our post process. If a neighbor has depth info but the pixel itself does not – we color it to our outline color.
We can use another technique for drawing a translucent overlay for all occluded geometry by comparing custom depth against SceneDepth. If CustomDepth is larger than SceneDepth we blend in a white tint with SceneColor.
Combining both techniques results in the post effect below:
I skipped most of the implementation details – if you have any questions feel free to ask them in the comment section below! I’ve made this effect available for download, feel free to modify and use this in your game.
Post Process material for outlines with translucent veil on occluded pixels.
After copying the .uasset files into the Content folder of your project you must add a Post Process Volume into your scene and enable the Unbound property, also don’t forget to add the post effect material under Blendables.
Culling Inner Triangles
When using transparency on models like character that have inner triangles (eyes, mouth, arm or bit of armor sticking through) you will see a noticable highlight in your material that attracts attention and simply doesnt look right. We can solve this issue using Custom Depth.
By rendering our character into the Custom Depth buffer we can cull any pixels that are behind the outter shell of the mesh. A slight depth offset should be added when comparing depth of Scene and Custom to prevent too many pixels to be culled including the outer shell. The material setup for this is quite simple:
I made a comparison of depth culling enabled and disabled in the setup below. The character on the right displays all occluded pixels in red. I rendered each character TWICE to work around a limitation where transparent materials do not render into custom depth at all (this is normal behavior for SceneDepth, but undesirable when explicitly enabling Custom Depth on a mesh with transparency applied) You can download the project source to see exactly how this was done.
For Switch we use this culling technique on a stealth effect giving it a more consistent ‘shell’ and highlight on a mesh that has a lot of overlapping pixels.
It’s important to know that if multiple materials use this technique that only the object closest to camera is culled correctly. In this image you can see the errors can would occur if you have multiple meshes overlapping. This should be taken into consideration when considering this technique – it may never be an issue if you use this effect sparingly or consistent isolated scenarios.
This issue can possibly be solved by adding a max depth delta when culling. I have had no need to dive into this further, but may come back to this in the future when it starts to cause practical issues.
For our game Switch we are using Custom Depth for a number of features, one of them includes an x-ray material that exposes players behind a wall – I added a short video of an early proof of concept below.
Multi-Color Outline Effect
A few engine versions ago Stencil Index buffer was added alongside Custom Depth. This enables us to create multi-colored outline effects among many other cool effects! Click here to read about this new effect, included with a download link for the material.
Issues & Limitations
- Custom Depth does not work on translucent materials. In that case you need a second copy of the mesh using a simple opaque material with Custom Depth enabled and Render Main pass disabled. View on AnswerHub
- Because the buffer is filled with all objects using Custom Depth it will not work when having multiple overlapped meshes (one behind the other) depending on how you’re using the depth value – a transparent mesh might be entirely culled if you use custom depth to cull inner triangles. A possible workaround is to add a maximum depth difference when looking for inner triangles to reduce the errors.