Exploring DLSS 2.0 in Unreal Engine
Last year Nvidia announced DLSS 2.0 with the ability to open this new anti-aliasing solution for all games. The game-specific deep learning is no longer required as it was with prior iterations. For an indie dev that is especially exciting as the chance of getting onto Nvidia’s SuperComputer was pretty slim. It’s now easier than ever to add DLSS support to your Unreal Engine title!
Besides the new game agnostic algorithm, we also get much-improved anti-aliasing results which honestly looks pretty fantastic and somewhat unbelievable in games like Control (See Digital Foundry’s DLSS Comparison). You’ll get a chance to see how DLSS performs in Unreal Engine through my own experiments below.
Update: Since this article was posted it has become much easier to access the DLSS Plugin for UE4.
For testing, I used my open-source SurvivalGame (available on GitHub) and Dekogon Studios’ City Subway Train asset.
What is DLSS?
DLSS stands for Deep Learning Super Sampling and in the 2.0 iteration it can use the Nvidia RTX cards’ Tensor Cores to upscale lower resolution render buffers to the desired ‘native’ scale much better than any existing upscale algorithm inside UE4 currently. At the same time that DLSS upscales, it is performing anti-aliasing effectively gaining performance from rendering the scene and post-processing at reduced input resolution. This AA solution replaces TAA and so dithered materials don’t seem to render the same with DLSS currently (where TAA would soften the dither pattern).
Remember that aliasing itself occurs from rasterizing a scene to pixels. Fewer pixels will cause higher aliasing, so the fact that DLSS actually fixes most aliasing while we provide it a much lower input than native is pretty amazing if you ask me.
One title using DLSS to improve performance while maintaining visual fidelity is Deliver Us The Moon, built using UE4.
Getting Started
For this article, Nvidia hooked me up with an RTX graphics card and access to the Unreal Engine RTX/DLSS Branch on Github. With the new graphics card, I got to play with ray-tracing for things like shadows, reflections, ambient occlusion, and even full Path Tracing that is now available. But what excited me most in practical terms is DLSS 2.0 and it has been the least covered in the context of Unreal Engine.
Update: It’s now much easier to get access to the DLSS Plugin! Download it directly from Nvidia without an AppId! The .zip includes detailed setup instructions too. (original instructions posted here removed)
The internal resolution is downscaled automatically based on the DLSS quality-setting. You can choose between 5 modes from Ultra Performance, Performance, Balanced, Quality, and Ultra Quality (Although this last mode was ‘not supported‘ for my setup). Finally, you have the option to sharpen the output.
In my tests, the internal resolution can go down to 33% (in Ultra Performance, meant for 4K Displays) which is a huge saving in screen-space operations such as pixel shaders, post-processing, and ray-tracing in particular. Even at 50% for Performance & Quality modes it’s still x4 fewer pixels to process. Judging from the provided UI the internal resolution can change on the fly between a predefined range (eg. from 50% to 69% in Quality-Mode) I’m not sure at this time how DLSS decides which exact internal resolution to use.
Anti-Aliasing Quality (DLSS vs. TAA)
The default anti-aliasing solution for Unreal Engine is Temporal AA (TAA) and so this is the main competitor for Nvidia’s DLSS in the engine.
Since DLSS is using a lower internal resolution, the real test is whether it can maintain final image quality while improving performance. A second major benefit is how it scales to 4K Displays (unfortunately my monitor is only 1440p) as it can enable 4K Gaming on mid-range graphics cards by using a more reasonable internal resolution.
It’s telling that often I had to double-check the screenshots to make sure I had the correct one between TAA and DLSS. Below you’ll see a few zoomed-in comparisons so that image resizing can’t interfere and honestly it’s necessary in order to see the difference.
Nearly indentical quality, slight error in the ceiling lights where a white line in the original texture got blown out by the upscaling algorithm causing a noticable stripe. I reckon this should be ‘fixed’ in the source texture instead.
Reflections look ever so slightly different, this scene used ray-traced reflections on highly reflective materials.
DLSS vs. TAA Performance
For the quality and performance comparison I’ve made a quick video toggling between the 3 different AA modes (TAA, DLSS Quality & Performance) to see the difference. As you’ll notice the visual quality is often difficult to see while the framerate takes a big leap at the same time. I would even argue that Quality-mode can conjure a higher quality image in some cases (I found that my foliage scene ‘felt’ crisper with DLSS enabled).
(Make sure to watch in 1440p and fullscreen.)
The Numbers
Please keep in mind these numbers were taken from my unoptimized scenes, running in standalone-mode (outside the editor) but not a cooked build. Several RT features were turned on to strain the system (including raytraced reflections).
Forest Scene (RTX On 2560×1440)
This scene was likely bottlenecked by the ray-traced reflections and so you’ll see a huge gain in framerate as the internal resolution is reduced.
- TAA Baseline ~35 FPS
- DLSS Quality ~56 FPS (+60%)
- … Balanced ~65 FPS (+85%)
- … Performance ~75 FPS (+114%)
- … Ultra Performance ~98 FPS (+180%?! – Noticeably blurry on 1440p, intended for 4K)
Subway Train (RTX On 2560×1024)
The camera used a cinematic aspect ratio hence the 1024p height. This scene used similar RTX settings and even ray-traced ambient occlusion on top.
- TAA Baseline ~38 FPS
- DLSS Quality ~69 FPS (+82%)
- … Performance ~100 FPS (+163%)
Subway Train (RTX Off 2560×1024)
Without any further RT-options enabled the difference in performance between internal resolutions appears to diminish somewhat. Although this was just a single test and your mileage may vary (as with all performance metrics, GPUs are a complicated beast)
- TAA Baseline ~99 FPS
- DLSS Quality ~158 FPS (+60%)
- … Performance ~164 FPS (+65%)
Render Pass (DLSS)
The extra renderpass occurs during Post Processing much like traditional AA solutions. On my Nvidia RTX 2080 Ti the cost to upscale to 1440p was about 0.8-1.2ms. This number seems to be consistent regardless of which quality mode is used. For reference, TAA at full 1440p costs about 0.22ms on my machine.
You can measure performance of individual render passes by either using “ProfileGPU” or “stat GPU” console commands.
Performance Conclusions
The performance potential of DLSS is huge depending on your game’s rendering bottleneck. Reducing internal resolution by 50% has an especially large benefit on RTX-enabled games with ray-tracing cost highly dependent on resolution.
Ray-tracing features appear to be working better with DLSS enabled from a visual standpoint too. RTAO (Ray-traced Ambient Occlusion) specifically causes wavy patterns almost like a scrolling water texture when combined with TAA. However, Enabling DLSS above Performance-mode completely eliminates these issues and provides a stable ambient occlusion.
Conclusion
Throughout my experiments, I’ve been super impressed with the results of DLSS 2.0. The fact that this magically works on any game out of the box without Nvidia SuperComputer pre-processing is impressive.
The image quality remains high and sometimes even managed to be crisper than TAA. There are some artifacts I ran into, that were overall small compared to the large performance gains we saw across the board. I’d love to test this on a 4K Display which is where the tech can shine even more.
If you want to see more, follow me on Twitter and Subscribe below to receive new articles straight to your inbox!
Hi Tom, we have been granted access to the NVIDIA DLSS Access program, but unfortunately they did not send us any App Id and they seems to not reply to emails. We can’t use 4.26 plugin integration cause 4.26 has still poor performance. That said, 4.25.4-dlss branch can be downloaded from github and dlss seems to work in editor environment. Is the app id / production approval still a thing?
I am not sure what their new plan is with AppIDs, I just know that with the new plugin the AppID is no longer required (but still an option, maybe for Ultra Quality Setting?) I can’t help you out with any of the Nvidia side of things as I am not even fully in the DLSS program myself and just received a temporary ID directly for testing only.
Hi Tom, thanks for your prompt reply.
Nvidia guys are totally missing, but I did some research: in fact, leaving default app id “0”, production build fails initializing DLSS (LogDLSSNGXRHI saying “error: Invalid app id was passed in. Contact Nvidia if you don’t have an app id.”)
As stated, the 4.26 plugin does not need an app ID, but the good news is that it also gets a valid ID (LogDLSSNGX saying “[….] app ID is xxxxxxxxx”) and that app ID works also on the engine version of DLSS.
Hi Tom,
Very informative and helpful article, thank you!
I wonder if we still need “NVIDIANGXApplicationId” to run DLSS in UE4? Though you mentioned we do not need it to download the plugin-in for UE4.
Thanks!
Shouldn’t need it for the plugin anymore which is the great part about that release!
I have a feeling Nvidia limiting DLSS 2.0 to at least acceptable reputation to test it as to protect it’s image from unknown studio/personal that could possibly use the tech in non representative way that’s not benefit to them. As DLSS 2.0 is pretty new and there’s many pro and cons from fans think about it.
You know if it’s more accessible to public, it’s possible someone use it and show only bad side of it (if there’s one, I’m don’t know).
It’s a shame as I think DLSS 2.0 will benefit more in for example Archviz, where mostly the scene is not as optimized as games. Many small lights, more high poly mesh, etc. And with raytracing is already benefit Archviz tremendously in term of visual, but performance hit is massive. So DLSS definitely helps.
They seem to have just opened up the DLSS plugin to everyone. Check out the ULRs in the updated post.
While this was intended to showcase DLSS 2, it really highlights how crap UE4’s TXAA really is. Blurry mess is what many gamers think of.
can you comment on the image quality during movement? I’ve found that in many games DLSS just makes everything blurry until you stand still in a game. And TAA just looks blurry all the time for example in modern warfare.
You can kind of compare this through the video (although YouTube will alter this quality somewhat) I did not notice bluriness during movement or a specific difference between standing still or moving. I noticed this a lot more with TXAA on the cables in the scene (and pointed this out in the video).
My time with DLSS has still been limited though, think we’ll need to see more scenes and games to fully understand the quality.
Thanks for the reply. Yeah it’s tough to tell due to the compression of youtube videos. I imagine it’s one of those things that not everyone may notice, but for me I find it hard to focus with the dlss and taa blur that ive seen in games. It’s also one of those things that screenshots wouldn’t show since they are stills. I think it’d be interesting to compare the texture clarity between no AA and having dlss on.
This is mostly useless. You have to obtain a key from nvidia to run this and they will just ignore your request.
I’d argue it isn’t. It’s a look into the future if anything. Some studios already have access and I suspect this will be opened up more in the future as the tech matures, the plugin is still in beta afterall. Even if you can’t use it yet, you have something to look forward to.
Of course that’s your stance, they gave you a free rtx card and access to it. Never mind the developers that purchased their top end flagship 3090 only to be ignored. The fact is, this technology is hidden behind Nvidia’s “because I said so” policy and that’s absolutely just ignorant and awful for the industry in general.
By the time this is main stream, there will be lots of other solutions and DLSS will have been yet another vaporware Nvidia marketing strategy.
You seem awfully bitter about a breakthrough technology that makes games better. This isn’t some easy-to-replicate tech, its based on long-run gains with vast compute training. DLSS is not going anywhere, and AMD ought to be working hard on their alternative.
This article was posted on reddit on an amd specific subreddit. The guy is just a troll from there.
I came from the UE4 subreddit sir. I have an intel system and a 3090. Sorry you feel like I’m a troll. I just have a strong opinion on the matter because I’m directly affected.
You think deep learning is breakthrough technology? Where have you been? You literally prove my point about Nvidia’s marketing strategies to people who don’t know any better.
I said that DLSS, not deep learning, is the technology. Read more carefully. I have worked as both a game developer and in general software development, so yes I think this technology is a game-changer.
They seem to have just opened up the DLSS plugin to everyone. Check out the ULRs in the updated post.