How to tell about Unicode support ?

Ask other users about problems encountered with portable apps or help by posting solutions to existing problems.
Post Reply
Message
Author
castman
Posts: 179
Joined: Sat Jun 28, 2008 5:41 am
Location: Brazil, Sao Paulo
Contact:

How to tell about Unicode support ?

#1 Post by castman »

One thing I want to learn about is what tells you that a software has full, partial or none unicode support.

Because I even don't know what is a unicode only character and what isn't.

So, I would be glad in learning from you.
Last edited by castman on Sun Dec 07, 2008 10:57 am, edited 1 time in total.

User avatar
m^(2)
Posts: 890
Joined: Sat Mar 31, 2007 2:38 am
Location: Kce,PL
Contact:

#2 Post by m^(2) »

Huge topic, really.
All letters that are not in the English alphabet like Polish ąćęłóśżź are not ANSI characters. Some non-letters too.
However you don't need Unicode to display them correctly because somebody found code pages (CP) years ago:
Some original ANSI characters are changed to fill local alphabets. As an example, in ANSI-1250 a letter with hex code 0xB9 is ą.
The problem is that with different code pages different characters get the same code. B9 won't be displayed as ą in any other CP.
Therefore as long as you don't have to deal with characters unavailable in your code page, Unicode doesn't give you any benefits(*).
But if program has to work with multiple languages at once or can't assume that the input it gets has the same language as OS, Unicode helps.

However Unicode brings a lot of mess regarding file encoding. There are 5 standards used in practice and a dozen of others. Many of programs that claim to be Unicode aware supports just 1-2 of them. That's a big problem.

Ad. (*): Almost. Windows API doesn't treat Unicode and non-Unicode programs equally, there are some things that have to be dome with Unicode. Minor ones.

castman
Posts: 179
Joined: Sat Jun 28, 2008 5:41 am
Location: Brazil, Sao Paulo
Contact:

#3 Post by castman »

Well, I understanding however, I am thinking how to apply to a tiny program.

I just have to place a non ANSI character in the filename, like "ç" to get the answer?

I will try this with Slower and some Win9x programs. Hehe

Thank You

User avatar
m^(2)
Posts: 890
Joined: Sat Mar 31, 2007 2:38 am
Location: Kce,PL
Contact:

#4 Post by m^(2) »

castman wrote:Well, I understanding however, I am thinking how to apply to a tiny program.

I just have to place a non ANSI character in the filename, like "ç" to get the answer?

I will try this with Slower and some Win9x programs. Hehe

Thank You
You have several programs and want to test whether they are Unicode?
Not easy. Mainly, because the problems are known since forever ans Windows has good ways of dealing with them and because programs can use Unicode in may ways.

If the program reads a file, Unicode characters in the name usually don't help...
The only way that would give positive answer would be to check whether it calls Unicode Windows functions.
Read the imports section and search for function names that end with capital W, i.e. CreateFileW. How to find the imports section?
I guess there are other tools, but I use Total Commander's lister with FileInfo plugin. Total Commander is commercial, but standalone lister is free.

castman
Posts: 179
Joined: Sat Jun 28, 2008 5:41 am
Location: Brazil, Sao Paulo
Contact:

#5 Post by castman »

Ohh! Sorry by my late answer. I saw a tool that tells you about all the functions exported by especified DLLs of the OS.

At NirSoft called DLL Export Viewer. Is That you are talking about?

I actually have only 2 applications I want to test: Slower and EndorFun.

EndorFun is a lovely game made for Windows 95. This Program has a simple GUI. It can save a game set. When I run it on Windows XP it consumes 50% (1 core of 2) of CPU usage. I use Slower to avoid this problem.

I tryed modifying the default.pif and some ntvdm optimizing programs both didn't work. I tryed also some dpmi applications for DOSBox, the program accused running from DOS. However, I couldn't try HX due to avast! keeping blocking as a virus.

Then I still need to reinstall Windows 9x in a VM.

User avatar
m^(2)
Posts: 890
Joined: Sat Mar 31, 2007 2:38 am
Location: Kce,PL
Contact:

#6 Post by m^(2) »

castman wrote:Ohh! Sorry by my late answer. I saw a tool that tells you about all the functions exported by especified DLLs of the OS.

At NirSoft called DLL Export Viewer. Is That you are talking about?
It's totally different thing, but should be OK.
castman wrote:I actually have only 2 applications I want to test: Slower and EndorFun.

EndorFun is a lovely game made for Windows 95. This Program has a simple GUI. It can save a game set. When I run it on Windows XP it consumes 50% (1 core of 2) of CPU usage. I use Slower to avoid this problem.

I tryed modifying the default.pif and some ntvdm optimizing programs both didn't work. I tryed also some dpmi applications for DOSBox, the program accused running from DOS. However, I couldn't try HX due to avast! keeping blocking as a virus.

Then I still need to reinstall Windows 9x in a VM.
Windows 9x by default is ANSI only, so programs designed for it are ANSI too, no need to check more.
There are patches that give very limited Unicode support in 9x, but no reasonable programmer assumed that user had them installed.

Why do you care whether they are Unicode? Does it change anything?

I have Slower and I looked into it. 99.9% that it's ANSI only.

User avatar
Andrew Lee
Posts: 3064
Joined: Sat Feb 04, 2006 9:19 am
Contact:

#7 Post by Andrew Lee »

I typically just try to get the app to load files with names that contain characters unsupported by the system's current ANSI code page eg. Korean or Japanese (야후! 코리아 / オークション).

You don't really have to read the language, just have a Unicode font installed that supports these languages. A popular font is Arial Unicode MS.

There are other things to consider depending on the nature of the app. For word processors, you want to be able to enter Unicode characters into the document. For MP3 players, you want to be able to enter/display Unicode characters in the ID3 tags etc. For certain apps, Unicode support is actually not very important.

User avatar
m^(2)
Posts: 890
Joined: Sat Mar 31, 2007 2:38 am
Location: Kce,PL
Contact:

#8 Post by m^(2) »

Andrew Lee wrote:I typically just try to get the app to load files with names that contain characters unsupported by the system's current ANSI code page eg. Korean or Japanese (야후! 코리아 / オークション).
I tried it before posting with the letter given by castman (it's not in my codepage) and Total Commander 7.0x. TC is ANSI and allows me to view the file.
Now I tried with your letters too...the same.
That's because TC is smart and uses short file names in this case - when viewing 야.txt, it opens 74D4~1.txt..and it works.

castman
Posts: 179
Joined: Sat Jun 28, 2008 5:41 am
Location: Brazil, Sao Paulo
Contact:

#9 Post by castman »

That's why when I will compile an AutoIt Script, I can choose between a non Unicode and a Unicode build.

I don't have anymore Win 9x applications. I threw all my OLD Dirty CD's to the trash. :D

This game was the only thing I had left.

Post Reply