New Vision 2.0 - Deus Ex, AI, and you

Discussion related to the New Vision mod.

Moderator: DaveW

User avatar
DaveW
New Vision
New Vision
Posts: 2351
Joined: Sat Nov 19, 2005 10:03 am

New Vision 2.0 - Deus Ex, AI, and you

Post by DaveW »

Chances are some of you have heard of the hot new emulation trend involving AI upscalers. Rather than using a basic algorithm like bicubic sampling, this uses a trained AI model to estimate what detail is and how best to scale it up to a higher resolution. This is mostly intended for photos, but it's had some success with game textures as well.

You can see where this is going; over the past day I've been dicking around with these using the Deus Ex textures, and the results are pretty promising. The most common system used in 'the scene' is ESRGAN (since it's free), but I didn't think the results were very good for textures. Instead I've been using a commercial application, Topaz A.I. Gigapixel, and the results are pretty good.

Some comparison screenshots with Vanilla / New Vision 1.5 / New Vision 2.0 (i.e. AI Upscale):
https://imgsli.com/MjY4Mw
https://imgsli.com/MjY4NQ
https://imgsli.com/MjY4Ng
https://imgsli.com/MjY4Nw
https://imgsli.com/MjY4OA
https://imgsli.com/MjY4OQ
https://imgsli.com/MjY5MQ
https://imgsli.com/MjY5Mg
https://imgsli.com/MjY5Mw
https://imgsli.com/MjY5NA

What's so great about this?
  • -It's ALMOST completely automatic, for most textures. Textures can be up-res'ed in batches and quickly previewed in game using the DX10 sideloading feature
  • -It maintains the original art style and direction of Deus Ex, as there's no other artist's hand involved
  • -It can be used on literally all game textures, including character models, which HDTP never had the manpower to touch
  • -This is the big one: this can be made to work with any mod, using the same batch export / conversion process.
What are the limitations?
Well, it doesn't look as high fidelity as New Vision, obviously. There's limits to what the AI can do, especially as it's built on pattern recognition in photos. There are several factors that limit how successful the upscale is on textures, like the original resolution, how 'abstract' it is, etc.

I also haven't figured out a good technique for masked textures yet.

Enough talk - I want to try it out!

DOWNLOAD HERE

Alright, so to test this out you'll want to uninstall any other graphical mods. That includes New Vision 1.5 and HDTP.

Take the contents of the archive and put them in your DeusEx\Textures folder.

Launch the game with Kentie's Direct X 10 Renderer - this is important, as I'm currently using his sideloading technique to load the textures.

This leads to some jumpiness in game; this would disappear if I used the original method of DXT Merging; so it's a solvable problem.

Also thanks to Björn for helping me extract all the original textures using Hanfling's UCC script.
Made in China
MJ12
Posts: 466
Joined: Thu Apr 02, 2009 7:55 pm

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by Made in China »

First of all, it great seeing you working on NV! Welcome back!

Now, the thing about AI upscalers is that it makes everything clearer. It looks great in UNATCO and places that are supposed to look shiny and, well, clear - but it doesn't work well when things are supposed to look grimy and it indeed hurts the artistic vision. For instance, the first image (UNATCO) looks superior to everything before it, while the second image (Brooklyn tunnels) looks inferior.

I feel like just AI upscaling everything will make for a lesser definitive version. It could prove to be better if more post processing would be done afterwards - re-adding the grime, dust, and maybe noise. Heck, maybe it'll even be easier to just mix and match NV1.5's textures with the AI ones to have the most complete texture set. This is also extremely true for the mangled Chinese parts that NV1.5 fixed.

Just my $.02, though - you really know more about it than me. And once again, glad having you back!
User avatar
FastGamerr
MJ12
Posts: 312
Joined: Sat Jul 10, 2010 10:06 am
Location: Finland
Contact:

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by FastGamerr »

Made in China wrote:Now, the thing about AI upscalers is that it makes everything clearer. It looks great in UNATCO and places that are supposed to look shiny and, well, clear - but it doesn't work well when things are supposed to look grimy and it indeed hurts the artistic vision. For instance, the first image (UNATCO) looks superior to everything before it, while the second image (Brooklyn tunnels) looks inferior.
I think it indeed would be hit-and-miss in certain areas of Deus Ex (however, UNATCO, Hong Kong, Area 51 and the character models look quite fantastic, though).

But since concepts of such fidelity don't exist in Deus Ex: Nihilum... well, damn. And it hasn't even been two months since I released the "final version". +o(

Thanks for the heads-up on this one, DaveW, I'll test NewVision2a.7z out as soon as I can!
User avatar
DaveW
New Vision
New Vision
Posts: 2351
Joined: Sat Nov 19, 2005 10:03 am

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by DaveW »

I think it should be possible to mix and match; I personally wouldn't mix and match New Vision 1.5 and these textures, though, since they are vastly different styles and there is no way you could isolate a certain area of the game (e.g. have a grungy New York, but not UNATCO). Hong Kong is probably the only area in the game that is almost completely standalone in it's texture set.

You could, though, use the character textures with HDTP keeping the original character models. The AI upscaler works really well for most characters since it can determine detail like hair, faces, etc.

The way I picture releasing this, is that there's the main release and then subsequent 'mod packs' that will replace specific mod textures. I'm not sure how many of these I would do personally, but I would make sure the instructions are there for anyone to do them. The only difficulty is with masked textures which requires a bit of a manual touch with Photoshop.
Made in China
MJ12
Posts: 466
Joined: Thu Apr 02, 2009 7:55 pm

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by Made in China »

I agree that it looks great on character textures. There are some edge cases on Tiffany Savage and Mechs - as they do have some unnatural elements to them (camouflage and mech parts) which might looks even more unnatural after the AI upscaling - which might prove beneficial to the latter, and detrimental to the former.

Do you think that going over and applying a layer of dust or grime using will be too much work? Because I feel like that intelligently adding those layers will be the best solution, if the same texture isn't reused in an inappropriate context.
User avatar
DaveW
New Vision
New Vision
Posts: 2351
Joined: Sat Nov 19, 2005 10:03 am

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by DaveW »

Even with the mechs, the AI seems to pick up on the detail fairly well because the original textures are comprehensible. It only struggles when the original image is vague or really low resolution, which is the case for 50% of the environment textures. It doesn't help that Deus Ex has a really 'shiny' art style either.

Image

An extra grunge pass might be something someone else can do; to be honest I'm 90% done, I just need to package it up with the last few manually updated textures. But I'm going to make a tutorial so someone else can follow this up if they want to.

I'm updating the download link, it now includes all the masked textures, so characters, deco objects, and third person weapons are now complete.
User avatar
Jonas
Off Topic Productions
Off Topic Productions
Posts: 14224
Joined: Sat Apr 24, 2004 9:21 pm
Location: Hafnia

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by Jonas »

This is really cool, damn! Nice work :o
Jonas Wæver
Chief Poking Manager of TNM

I've made some videogames:
Expeditions: Rome
Expeditions: Viking
Expeditions: Conquistador
Clandestine
User avatar
FastGamerr
MJ12
Posts: 312
Joined: Sat Jul 10, 2010 10:06 am
Location: Finland
Contact:

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by FastGamerr »

Agreed! Well, it's not always an automatic success, as with these Queen's Tower signs:
Shot0000.jpg
But overall, very very nice.

Is there a major difference in quality if the original texture is enlarged to, say, 1024x1024 or above instead of 512x512 like in the zip?
User avatar
DaveW
New Vision
New Vision
Posts: 2351
Joined: Sat Nov 19, 2005 10:03 am

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by DaveW »

Pretty much anything with text scales quite poorly, due to how low-res and blurry the originals were. So there's a bunch of signs like the UNATCO nameplates that look a bit rubbish, and could probably do with a manual remake. I've already manually recreated some of the more basic text signs (e.g. ones that are text over a black background) but these would require a lot more work which I'm not feeling up for with my other projects taking priority.

The 4x upscale is actually a limit of Topaz AI which seems to cap at 6x resolution, but when I've forced it to upscale to 1024x1024, the results are actually worse. On some of the textures where it struggles, you get what looks like JPEG compression artifacting, which is a lot more obvious at a high resolution. It's actually better to leave it at 512 and then this detail kind of 'blurs out' as the game upscales the texture.
User avatar
FastGamerr
MJ12
Posts: 312
Joined: Sat Jul 10, 2010 10:06 am
Location: Finland
Contact:

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by FastGamerr »

Alright, thanks for the heads-up.

Would be really cool if the issue with enlarging masked textures could be solved, someone could possibly finish the UI rescaling projects (started by kocmo and the like) with the aid of this tool.
User avatar
DaveW
New Vision
New Vision
Posts: 2351
Joined: Sat Nov 19, 2005 10:03 am

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by DaveW »

It's mostly a case of finding the right process; I've found that up-scaling the image with the usual pink mask colour and then using Photoshop's Select and Mask tool (with colour decontamination) is a fairly efficient way of doing it. It's not automatic, but it is quite fast.

I doubt this technique would work for the UI, since it's relying on pattern recognition with a model trained on photos. Stuff like that will still have to be done manually, though the D3D10 renderer side-loading would make it relatively simple to load those textures without messing around with the .u files.
Defaultplayer001
NSF
Posts: 81
Joined: Sat Nov 05, 2016 1:18 pm

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by Defaultplayer001 »

This is awesome, thanks so much! Great to see you modding again <3.

I've actually been looking into this sorta thing for a while for the community update https://trello.com/c/U1DaBwEd/46-experi ... resolution.

I've tried waifu2x and the results were OK, but wanted to experiment with other tools. It's great to see what Gigapixel looks like!

Certainly seems better then w2x!

Do you mind if I include this an an option for the Community Update? It's really great, and a lot more lightweight then NV 1.0!

I think I might get Gigapixel myself, looks like it might be the best / easiest option; would love to be able to help with this.

Also, posted about this to https://old.reddit.com/r/GameUpscale/ and r/Deusex, thans again for all the work you've done for the DX Community!
User avatar
DaveW
New Vision
New Vision
Posts: 2351
Joined: Sat Nov 19, 2005 10:03 am

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by DaveW »

Oh, neat - I didn't know there was a subreddit dedicated to this kind of thing. I stumbled upon Gigapixel months ago by accident and only recently saw people experimenting with ESRGAN for game textures.

The results definitely vary a lot based on the model; most models (at least, the example ESRGAN and Gigapixel) rely on photos so are great at extracting real world detail from textures, but when they're more abstract it struggles. Hopefully as more people build models trained on actual game textures (there seems to be an effort to do this) we'll start seeing better results. I'm going to upload a sort of mini-SDK with all the Deus Ex textures extracted so people can quickly run different models on the Deus Ex texture set and test it out.

I've got no problem with you including it in the community update, this version isn't final though, I might go back and create some more manual textures and also improve some of the masked ones.
User avatar
Jaedar
Illuminati
Posts: 3937
Joined: Fri Mar 20, 2009 3:01 pm
Location: Terra, Sweden, Uppsala.

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by Jaedar »

I actually think 1.5 looks better than ai upscale.

2.0 is closer to the original, but 1.5 has more detail and overall sells the scenes better imo.
"Delays are temporary; mediocrity is forever."
odio ergo sum
User avatar
CrocMagnum
NSF
Posts: 86
Joined: Sun Nov 18, 2012 4:19 pm
Location: Brussels

Re: New Vision 2.0 - Deus Ex, AI, and you

Post by CrocMagnum »

Holy Smoke, this is promising! My subjective opinion:

I agreed with Jaedar that this upcoming v.2.0 looks globally more faithful to original Deus Ex, I like it very much. Previously depending on areas, textures (especially floors) looked garbled sometime. This reminds me how ENB and SweetFX tools (nothing to do AI Upscalers, I guess) could often "shave off" textures and deprive them from their substance.

Also I agree that the effect on characters, especially their faces, is truly impressive: they look sharper and more vivid.
DaveW wrote:...this version isn't final though, I might go back and create some more manual textures and also improve some of the masked ones.
Good catch. Considering your solid credentials with previous versions of New Vision, I'm confident that you'll make something worthy. I bet you wouldn't go out of your way to work on a new version of "New Vision" if this AI Upscaler wasn't so full of promise. So sincere Good Luck on your endeavours, friend.

Finally, DaveW, my dear! I’m so happy to hear from you again. For the record you're one of the reasons why I registered here in 2012: just to thank you for "New Vision". In other words:

"Great Modders are just like retired Rock Stars,

sometimes they do come back!" :mrgreen:

Defaultplayer001 wrote:...I've tried waifu2x and the results were OK...
Well! Out of context that’s a rather funny statement. ^^

Thanks for the pointer though, this tool sounds super useful and I might just need it.
Pity the world if you fail Spirit Monk, for you are the last. (Jade Empire)
Post Reply