Idea and practice: Creating an independent localisation pack

A refuge for those migrating from the fallen DXEditing.com and a place for general discussion relating to Deus Ex editing (coding, mapping, etc).
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Idea and practice: Creating an independent localisation pack

Post by Bird »

I would like to evolve my Hungarian Localisation MOD further to be independent of any MOD that comes around. I think it would be a good idea to have this topic for reference to others who will have the same issue in the future.

Goals:
  • To use .××T files in an extensive, yet inobtrusive manner
  • To create a unique LanguageMOD-package by merging DeusExText.u and DeusExUI.u (and DeusExConText.u maybe)
  • To have and maintain proper typography by hacking the hell out of TTF files and to assign rarely-used, or non-used characters for advanced typography
Problems:
  1. I still need to figure out how to hook translated DeusExText entries from DeusExHUTMOD.u via .HUT files in order to have DeusExText.u coexist with DeusExHUTMOD.u (i.e. to have DeusExText.u as a failsafe, fallback option).
  2. I still don't know what to do with DeusExConText.u and how to make it independent of any other project (however, it is unlikely that this file will be used by other MODs)
Things I managed to do so far:
  • Merged DeusExText and DeusExUI and created new TTF Fonts based on Handel Gothic
  • Experimented in an unsuccessful way to make .HUT files read InforationDevices from DeusExHUTMOD.u rather than DeusExText.u
Tried to edit 00_training.hut as below:

Code: Select all

[DataCube0]
TextPackage="DeusExHUTMOD"
textTag="00_DataCube01"
This did not do the trick.
I changed every instance of Language=int to Language=hut and localised strings from DeusEx.hut are being read. Any ideas and/or critical comments will be appreciated.
Hanfling
MJ12
Posts: 406
Joined: Sun Oct 04, 2009 6:54 pm

Re: Idea and practice: Creating an independent localisation

Post by Hanfling »

Why so complicated? It's fairly easy. Just name your files DeusExUI.hut_u, DeusExConText.hut_u, etc. And they will get loaded if you have Language=hut. However, you should consider using hun, and not hut for hungarian, as this seems to be used by at least UT3.

Also again in this thread. The exported fonts using DEED are *NOT* correct. Bitmap fonts get converted to latin1 during import, which is not bijective. However, you can produce exported textures which result in correct reimported textures nontheless, but DEED does not do this, so you end up with lots of bad non ANSI characters. If this still is not enough, I might consider figuring out a way to get at least the full latin1 charset, and not just parts of it imported, however, you would need new textures for that anyway, but they would be imported in order.
Last edited by Hanfling on Mon Aug 10, 2015 9:13 pm, edited 1 time in total.
I demand my DXE User ID back. Launcher for DeusEx, Rune, Nerf, Unreal & Botpack. HX on Mod DB. Revision on Steam.
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Re: Idea and practice: Creating an independent localisation

Post by Bird »

Thanks for the heads-up, I just wanted to follow legacy naming conventions as it were set here: http://wiki.beyondunreal.com/Legacy:INT_File

Back then it used to stand for INternational Text and for DEutsch Text, hence the HUngarian Text, not .HUN. Maybe they changed their minds 10 years after.
Just name your files DeusExUI.hut_u, DeusExConText.hut_u, etc.
I don't get it. Mustn't it be DeusExUI_hut.u? I am not getting it whether it should be an extension or something else.

BTW my 00_training.hut gets loaded OK, but the last entry with the [DataCube0] gets ignored:

Code: Select all

[SecurityCamera2]
FamiliarName="UN_obst_swim"

[PickupDistributor0]
NanoKeyData[0]="(ScriptedPawnTag=Test_Subject,KeyID=Body_Search_Room,Description="Orvosi szoba kulcskódja")"

[SecurityCamera3]
FamiliarName="UN_obst_swim2"

[SecurityCamera1]
FamiliarName="UN_obst_jump"

[SecurityCamera0]
FamiliarName="UN_obst_ramp"

[LevelSummary]
Title="Untitled"

[DeusExLevelInfo0]
MissionLocation="UNATCO kiképzõpálya"
startupMessage[0]="UNATCO Kiképzõközpont"
startupMessage[1]="Szárazföldi Egyesült Államok"

[NanoKey0]
Description="Gyakorlóajtó kulcskódja"
FamiliarName="Gyakorlóajtó"

[DataCube0]
TextPackage="DeusExHUTMOD"
textTag="00_DataCube01"
Hanfling
MJ12
Posts: 406
Joined: Sun Oct 04, 2009 6:54 pm

Re: Idea and practice: Creating an independent localisation

Post by Hanfling »

No. .hun_u instead of .u. UE1 does check for these files first. If I rename the german DeusExConText.u out of the latest german patch DeusExConText.det_u and use det as language I get german conversation dialogues.

/edit:
And probably just variables which have the localized specifier are loaded out of localisation files.
I demand my DXE User ID back. Launcher for DeusEx, Rune, Nerf, Unreal & Botpack. HX on Mod DB. Revision on Steam.
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Re: Idea and practice: Creating an independent localisation

Post by Bird »

EDIT:

Will check this out in some minutes. Thank you!
The exported fonts using DEED are *NOT* correct.
I am using

Code: Select all

#exec new TrueTypeFontFactory Name=FontLocation					FontName="HandelGotDLig"			Height=38	GROUP="Fonts"	AntiAlias=1
to import all of my TTF fonts in my pre-defined .uc file.
Hanfling
MJ12
Posts: 406
Joined: Sun Oct 04, 2009 6:54 pm

Re: Idea and practice: Creating an independent localisation

Post by Hanfling »

Well I'm not sure that you need to replace all fonts with TTF ones to get all characters you need. Try with just an hun file with vanilla DeusExUI.u / Extension.u. If that does not work try a different encoding for the .hun files.
I demand my DXE User ID back. Launcher for DeusEx, Rune, Nerf, Unreal & Botpack. HX on Mod DB. Revision on Steam.
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Re: Idea and practice: Creating an independent localisation

Post by Bird »

EDIT:
If I rename the german DeusExConText.u out of the latest german patch DeusExConText.det_u and use det as language I get german conversation dialogues.
That trick worked perfectly and flawlessly! Much appreicated.

I'd really love to use the default character set what we have in .PCX files, but I am obsessed with these Hungarian characters which I cannot exclude: őűŐŰ

I tried to edit .PCX files manually a long time ago with that red bracketish-like palette, but I failed miserably.

The vanilla Deus Ex has Û, û (not a surprise) instead of Ű, ű and surprisingly has Ő and ő by default. ÉÁÍÓÚÜÖ and its lower-case counterparts are being displayed as expected. Oh, and not to mention that typography-menace I would like to accomplish (like to put … &endash; &emdash; „ ” in). If this is the case, I surely would need to have those TTF files integrated.
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Re: Idea and practice: Creating an independent localisation

Post by Bird »

I just realised I am not sure which principles I ought to follow.

First way:
Shall I stick to vanilla user interface design, force everything hoping to be used at 640×480 and leave every font's size as it were, so it is painstakingly difficult to read on higher resolutions than 1024×768.

Second way:
Shall I adapt and experiment how things are supposed to look at even 720p or 1080p or even at 4K and drop support for screens lower than 1024×768, which leads to ultimately better readable fonts in the end, but breaks legacy support.

Nowadays, DataVault Images are being doubled, there is Deus Ex Enhanced, New Vision, just to name a few. On which way shall I go?
Hanfling
MJ12
Posts: 406
Joined: Sun Oct 04, 2009 6:54 pm

Re: Idea and practice: Creating an independent localisation

Post by Hanfling »

I would stick to making a localisation project localisation only, as the language is seperated from the actual UI logic and affects different files. So head for making a set of int files and localized DeusExText.u/DeusExConText.u[/DeusExConversations.u/DeusExConAudio*.u's], so at that level it would work with everything out of the box.

But well, there is still the font issues for it to be rendered correct. The difference between bitmap fonts and ttf based ones is the time when these get rendered. For the bitmap fonts it's when you create them with a program, for the ttf fonts it's during import. Whats really odd about bitmap fonts is that the charset order inside the image file is a different then what the game uses in the end (cp850 -> latin1). However let me see what I can do to improve the font stuff (sth. which I planed to do anyway) while you work on the localization itsself.

/edit:
Can you run batchexport on the packages which do include the fonts (The raw Texture with lines removed end up as Texture<NUM>.pcx/bmp after export). And check the textures file which include more then just ASCII characters if they include all characters needed for hungarian. If thats the case an approach to reuse the alreadys rendered fonts would be feasable to remain the visual style (and safe a lot of work).
I demand my DXE User ID back. Launcher for DeusEx, Rune, Nerf, Unreal & Botpack. HX on Mod DB. Revision on Steam.
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Re: Idea and practice: Creating an independent localisation

Post by Bird »

Thank you for providing me help Hanfling.
Basically Õ and õ and Û and û should be changed to their Double Acute Accent (Ő and ő and Ű and ű) counterparts.
Texture13: Û and ûŰ and ű
Texture14: Û and ûŰ and ű
Texture16: Û and ûŰ and ű
Texture17: Û and ûŰ and ű
Texture18: Û and ûŰ and ű
Texture19: Û and ûŰ and ű
Texture20: Û and ûŰ and ű
Texture21: Û and ûŰ and ű
Texture25: Û and ûŰ and ű
Texture26: Û and ûŰ and ű

Texture27: Û and ûŰ and ű | Õ and õŐ and ő
Texture28: Û and ûŰ and ű | Õ and õŐ and ő
Texture29: Û and ûŰ and ű | Õ and õŐ and ő

Unfortunately I am not able to get this issue fixed, since all of my efforts turned into a set of garbled in-game fonts.

P.S: Besides Hungarian text, there are localised DataVaults as well in DeusExHUNMOD. See: http://deusex.hu/files/raw/deus_ex_hung ... isation.7z
These are the raw files, installer can be found at: http://www.moddb.com/mods/magyardx
Hanfling
MJ12
Posts: 406
Joined: Sun Oct 04, 2009 6:54 pm

Re: Idea and practice: Creating an independent localisation

Post by Hanfling »

Ah ic, so Ű and ű and Ő and ő are mostly missing inside the fonts.
I demand my DXE User ID back. Launcher for DeusEx, Rune, Nerf, Unreal & Botpack. HX on Mod DB. Revision on Steam.
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Re: Idea and practice: Creating an independent localisation

Post by Bird »

Yes, and fortunately nothing else needs to be edited apart those, so it saves a lot of tedious work. I mean in-game characters gets drawn as Û instead of Ű and õ instead of ő, just to name a few.
Hanfling
MJ12
Posts: 406
Joined: Sun Oct 04, 2009 6:54 pm

Re: Idea and practice: Creating an independent localisation

Post by Hanfling »

Digged a bit around in code and brain regarding 16 bit/char unicode support.
Engine handles fonts as 16 bit/char internally. Engine supports loading int files with a 16 bit character set. Font rendering supports it too. TTF font importer has Count= option (which defaults to 255), which is afaik the last supported character, so you could use it to import more characters.

So what would be useful is to figure out all fonts/font sizes used to create the bitmap fonts, so one could reimport them with more glyps, while remaining the same visual style. So if one would start such a list. Great! I would really like to see such a list.

DeusExText (Datavaults, emails, etc)/Conversations are assumed to be 8 bit/char, so you can't import 16 bit/char texts there. However after import they are 16 bit/char anyway, so one would basically just change/add/modify an importer for them to be able to import 16 bit files. For DeusExText files this is fairly easy and I'll add that to HTK, for *.con files this is a really huge problem as they use 8bit/char, so one would not just need the importer to support the 16 bit files but also an conversation editor which supports writing the 16 bit files. But well I'm working on my own *.con file exporter and some QtConEdit might follow at some point.
I demand my DXE User ID back. Launcher for DeusEx, Rune, Nerf, Unreal & Botpack. HX on Mod DB. Revision on Steam.
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Re: Idea and practice: Creating an independent localisation

Post by Bird »

Hanfling wrote: So what would be useful is to figure out all fonts/font sizes used to create the bitmap fonts, so one could reimport them with more glyps, while remaining the same visual style. So if one would start such a list. Great! I would really like to see such a list.
According to \Engine\Classes\Canvas.uc:

Code: Select all

// Objects.
#exec new TrueTypeFontFactory Name="SmallFont" FontName="HandelGothic BT" Height=8 AntiAlias=1 XPad=2 CharactersPerPage=64

// DEUS_EX AJY
#exec new TrueTypeFontFactory Name="MedFont" FontName="HandelGothic BT" Height=18 AntiAlias=1 XPad=2 CharactersPerPage=64
#exec new TrueTypeFontFactory Name="BigFont" FontName="HandelGothic BT" Height=22 AntiAlias=1 XPad=2 CharactersPerPage=64
#exec new TrueTypeFontFactory Name="LargeFont" FontName="HandelGothic BT" Height=32 AntiAlias=1 XPad=2 CharactersPerPage=64
Maybe I can dig something up in the near future in order to find an identical and genuine font. It seems they used Handel Gothic BT for all the fonts, except for that sans-serif font in Behind The Scenes, some Helvetica-like typeface for conversations and for TechFonts in Extension.u
Hanfling wrote: DeusExText (Datavaults, emails, etc)/Conversations are assumed to be 8 bit/char, so you can't import 16 bit/char texts there. However after import they are 16 bit/char anyway, so one would basically just change/add/modify an importer for them to be able to import 16 bit files. For DeusExText files this is fairly easy and I'll add that to HTK, for *.con files this is a really huge problem as they use 8bit/char, so one would not just need the importer to support the 16 bit files but also an conversation editor which supports writing the 16 bit files. But well I'm working on my own *.con file exporter and some QtConEdit might follow at some point.
Thanks for sharing these! I can barely wait for those features implemented. Wish I knew how to write code properly (as in I am soo far away from programming stuff), so I could help somehow.
User avatar
Bird
NSF
Posts: 86
Joined: Sat Sep 19, 2009 3:24 pm
Location: between your synaptic gap

Re: Idea and practice: Creating an independent localisation

Post by Bird »

I think there won't be any other option rather than buying a Handel Gothic font from Bitstream foundry (for like 30 Euros), but it doesn't support and Cental European characters. I tried Tidle's Handel Gothic but it looks far more alterior than Bitstream's.
Post Reply