DLSS 5 - photorealism lighting and materials

[FourWheelDrift]

https://www.nvidia.com/en-gb/geforce/news/dlss5-br...

DLSS 5 introduces a real-time neural rendering model that infuses pixels with photoreal lighting and materials. 3d models are not altered, it uses AI to completely overhaul a game's lighting in real time, applying photo-realistic illumination, subsurface scattering on skin, more convincing hair rendering, and improved ambient occlusion to existing game worlds without changing geometry, textures, or materials.

Starfield - https://www.youtube.com/watch?v=-vMVlfxUDe4

Zorah - https://www.youtube.com/watch?v=GXpTyq-YbPM

Hogwarts Legacy - https://www.youtube.com/watch?v=t50AdVMuxZE

Resident Evil Requiem - https://www.youtube.com/watch?v=MhLWH18vXH4

[Lucas Ayde]

It looks incredible ... worth noting that it was a development preview and Nvidia were using TWO 5090s to run the demos .. one of the cards to render the games and the second 5090 to actually do the DLSS5 processing.

Of course it will be rolled into something that is usable on a single card when it's ready for release but I would guess you'll have to buy one of their 60xx series cards (when they are available) to get it and probably one of the higher-end ones at that, to get good performance.

See the DF report:

https://youtu.be/4ZlwTtgbgVA

[IanH755]

Might be a VRAM eater looking at it, so the next 6000 series mid/low tier cards might not to run it with less than 16Gb as a "finger in the air" guess.

I'm also not 100% sure I want "photorealism" for the FPS games I play, might be a bit too much realism

[JoshSm]

Looks like it tended to fk up the lighting and shadows, went for the 'brighter & more saturated = better' look like people do when showing off TVs in the shop.

Some of it was quite impressive but it's a bit like a lot of what's in UE5 - looks shiny in the demos but there's a lot of horrible compromise involved if you want it to do your own specific thing that doesn't fit their idea (as I hear from the developers who have to fight its codebase).

[Mr Whippy]

More layers of “stuff”

If they just made optimised high quality content to render in the first place…

I’m not sure this kinda approach is sensible long term.

I assume a multi-gigabyte model sits alongside this tech, and if it gets updated/changed the appearance over time is gonna change.

Plus in 10yrs when the tech is abandoned will older games suddenly look even worse than they should?

[Brainpox]

Looks like complete dogst. Why would you want to run an AI slop filter over your games? Even their cherry picked examples with dual 5090s look absolutely awful and in many cases totally change the feel of the scene.

[FourWheelDrift]

Brainpox said:

Looks like complete dogst. Why would you want to run an AI slop filter over your games? Even their cherry picked examples with dual 5090s look absolutely awful and in many cases totally change the feel of the scene.

It's what people already do to Bethesda games to get rid of the potato faces.

[AlexC1981]

It has huge potential. I watched the Digital Foundry video on Youtube. The faces, clothes and details look great, however the outdoor scenery shots look too bright, lost atmosphere, and were not pleasant to look at. I hope it will be a lot better by the time it is released.

I remember when the 3dFX VooDoo cards came out and everyone was amazed at the leap forward. If Nvidia get it right, this could be a similar revolution. I've been wanting realistic graphics for years and frankly, the progress over the last 20 years has been poor.

Look at the Oblivion re-master for example:

https://www.youtube.com/watch?v=dWIub1PZIMk

It looks better, but not 20 years better in my opinion.

Mr Whippy said:

Plus in 10yrs when the tech is abandoned will older games suddenly look even worse than they should?

It runs locally, so as long as you still have a graphics card that supports it, it should be fine.

Brainpox said:

Looks like complete dogst. Why would you want to run an AI slop filter over your games? Even their cherry picked examples with dual 5090s look absolutely awful and in many cases totally change the feel of the scene.

You're never going to get something approaching realistic graphics without using AI at some point. Either when the game is being made or later, such as this. If older games like Morrowind can be modified to use it, it would be fantastic. Morrowind is considered by many to be the best of the Elder Scroll games, but after playing Oblivion and Skyrim, I couldn't deal with the massive visual downgrade.

[PhilboSE]

AlexC1981 said:

Look at the Oblivion re-master for example:

https://www.youtube.com/watch?v=dWIub1PZIMk

It looks better, but not 20 years better in my opinion.

That’s because it’s a remaster, not a ground up rewrite with a state of the art engine.

Look at (say) RDR2, an 8 year old game now and tell me there’s not 12 years advancement over Oblivion original. The game world, richness, textures, lighting effects, shadows, early morning mist…all a staggering step forward.

The trouble is that a game as rich as RDR2 takes a simply phenomenal amount of effort to design and populate the world, and 99% of games studios don’t have those resources.

Which is why they might take some post-processing free smarts, it’s zero cost to them.

AlexC1981 said:

If older games like Morrowind can be modified to use it, it would be fantastic. Morrowind is considered by many to be the best of the Elder Scroll games, but after playing Oblivion and Skyrim, I couldn't deal with the massive visual downgrade.

No amount of post-processing can overcome the fundamental lack of polygons and textures in very old games. There just isn t enough data to extrapolate from.

Edited by PhilboSE on Friday 20th March 08:33

[Brainpox]

Graphics don’t need to be photorealistic… and the problem is this filter ignores all lighting in the scene and applies a light source behind the camera, as if taking a photo, like all AI slop images do. All the ray tracing that we love gets ignored. It makes the scene less realistic, so even if that is it your number one priority in games, it doesn’t achieve it.

https://youtu.be/H1a8YEOlpeY?t=545

[AlexC1981]

PhilboSE said:

That s because it s a remaster, not a ground up rewrite with a state of the art engine.

Look at (say) RDR2, an 8 year old game now and tell me there s not 12 years advancement over Oblivion original. The game world, richness, textures, lighting effects, shadows, early morning mist all a staggering step forward.

The trouble is that a game as rich as RDR2 takes a simply phenomenal amount of effort to design and populate the world, and 99% of games studios don t have those resources.

Which is why they might take some post-processing free smarts, it s zero cost to them.

RDR2 looks good, but like you say it relies on sheer human effort to produce that. Would it even be possible for a human to produce the art needed for realistic graphics? RDR2 looks more like a painting, where's DLSS 5 looks like an over processed photo. I think AI is going to be the only option to get close to realism.

PhilboSE said:

No amount of post-processing can overcome the fundamental lack of polygons and textures in very old games. There just isn t enough data to extrapolate from.

Edited by PhilboSE on Friday 20th March 08:33

I understand it won't look realistic due to he lack of polygons, but I don't see why the textures can't be enhanced.

[AlexC1981]

Brainpox said:

Graphics don t need to be photorealistic and the problem is this filter ignores all lighting in the scene and applies a light source behind the camera, as if taking a photo, like all AI slop images do. All the ray tracing that we love gets ignored. It makes the scene less realistic, so even if that is it your number one priority in games, it doesn t achieve it.

https://youtu.be/H1a8YEOlpeY?t=545

I don't think that's correct. In this example, you can see the shadow on one side of the face, and you can see the shadow of the guys head outlined on his shoulder etc. and this is consistent on the other players. It definitely needs more work though! Not sure what's going on with those muscle outlines on the chest and abs.



I agree you don't need realistic graphics to make a fun game, but I have enjoyed every single graphics upgrade that's come from upgrading from a Commodore 64 to Amiga, Megadrive, Playstation, Voodoo, GTX, RTX (and a few more between). This is just one more step on the road to actual realistic graphics that I've been waiting 40 years for.

[JoshSm]

AlexC1981 said:

RDR2 looks good, but like you say it relies on sheer human effort to produce that. Would it even be possible for a human to produce the art needed for realistic graphics?

There's loads of tooling & techniques that'll help out, even free stuff like MetaHuman can do a really good job.

You don't need AI post processing to do fantastic graphics, it's just a lot cheaper and lighter to slap a AI image over the top instead of doing it the hard way.

Downside is it can do some very weird things that don't reflect the original intent.

[mmm-five]

AlexC1981 said:

...snip...
I don't think that's correct. In this example, you can see the shadow on one side of the face, and you can see the shadow of the guys head outlined on his shoulder etc. and this is consistent on the other players. It definitely needs more work though! Not sure what's going on with those muscle outlines on the chest and abs.

In that football game example, you can see how much they've changed the representations of real-life players in the way that a dodgy-looking VVD turns into a Rashford or Bellingham when DLSS5 is turned on

It must be using the same AI that Samsung used for its moon photos!