NVIDIA DLSS 5 Analysis Covering Screen Space AI Filter Technology Technical Limitations Performance Hardware Requirements and Developer Creative Control
I have been trying to get a straight answer from Nvidia about the inner workings of DLSS 5 for a few days. It turns out that some information has finally surfaced through recent technical discussions. To be honest, the tech seems a lot less complex than I originally assumed. While the demos at the conference looked high end, the actual process under the hood appears to be quite a bit simpler.
The thing is that Nvidia has confirmed that DLSS 5 primarily uses a 2D image and motion vectors as its input. It does not actually understand the 3D world or the geometry behind a scene. It essentially takes a snapshot of the game and applies a very fast AI filter over the top. This might suggest that the heavy lifting is more about the speed of the computation rather than a deep understanding of the game engine itself.
I find it a bit strange that a pair of RTX 5090 cards were needed for the demo. One card handled the game and the other was a four thousand dollar unit dedicated just to the AI path. If the system is just looking at a flat image, that is a massive amount of power for what is basically a post processing effect.
One major concern is how the model handles materials and lighting. Since it does not talk to the game engine about the specific textures, it has to take an educated guess. It looks at the frame and tries to figure out if a surface is meant to be wet, rough, or metallic. This appears to be a probabilistic approach rather than using the actual data the developers built into the world.
- Input Only a static 2D frame plus motion vectors.
- Awareness Zero understanding of 3D geometry or depth outside the screen space.
- PBR Handling Materials are inferred from the pixels rather than the engine data.
I suspect many developers will find the lack of control frustrating. Right now, there is no way to tell the AI exactly what to do with a character. There are no text prompts or fine tuning tools available. Instead, developers get a simple slider to change the intensity of the effect or the ability to mask off certain objects so the AI ignores them.
We saw this with the character Grace in the Starfield demo. The AI painted over her face and gave her a look that did not really fit the scene. If a developer does not like how a character looks, they can really only turn the filter down or off. They cannot ask the AI to try again with a different style. This could lead to a situation where characters in different games start to look a bit too much like each other because they are all being painted by the same model.
Actually, the whole reveal feels like a bit of a disaster. Jensen Huang spoke about a fusion of structured data and generative AI, but it currently looks like a simple layer of frosting on top of a cake. There is a clear gap between the real game data and the AI output. I think Nvidia has a lot of work to do if they want to convince people that this is more than just a fancy filter. For now, the messaging has been a bit of a headache for everyone involved.
