Comments on: Them Character Encoding Blues http://blog.isecore.net/2007/10/06/them-character-encoding-blues/ A Kevlar-Burrito Full Of Meat Fri, 09 Jan 2009 07:50:46 +0000 http://wordpress.org/?v=abc hourly 1 By: isecore http://blog.isecore.net/2007/10/06/them-character-encoding-blues/comment-page-1/#comment-83118 isecore Sat, 06 Oct 2007 11:26:43 +0000 http://blog.isecore.net/2007/10/06/them-character-encoding-blues/#comment-83118 Well, that would make sense. I've read about several Windows-users who discover that their OS uses some pretty weird encodings, but UTF-16 would make sense in some ass-backwards way. Anyhoo, everything here is going to be converted to UTF-8 in the future. I'm tired of seeing weird characters in my filenames :) Well, that would make sense. I’ve read about several Windows-users who discover that their OS uses some pretty weird encodings, but UTF-16 would make sense in some ass-backwards way.

Anyhoo, everything here is going to be converted to UTF-8 in the future. I’m tired of seeing weird characters in my filenames :)

]]> By: Mind http://blog.isecore.net/2007/10/06/them-character-encoding-blues/comment-page-1/#comment-83103 Mind Sat, 06 Oct 2007 07:33:55 +0000 http://blog.isecore.net/2007/10/06/them-character-encoding-blues/#comment-83103 Ah, I found some more info. Apparently, in windows the standard is UTF-16, that is, the cheap version of UTF-8. The pros is that characters are always 16 bits wide instead of varying length as it is in UTF-8. That makes for extremely easy encoding as you can beforehand reserve space for said amount of characters. But, UTF-16 is not UTF-8 and therefor does not support all the characters UTF-8 supports. Now, windowsdevelopers refer to UTF-16 as unicode and Linuxdevelopers refer to UTF-8 as unicode. Hence the confusion on my part. A more correct term would be to not refer to UTF-16 as unicode at all but rather keep using the term "wide" and nothing else. The standard libs to handle unicode under Linux handles UTF-8 and on windows UTF-16. Pretty annoying, this probably means I have to get some external lib to be able to support UTF-8 on windows. Ah, I found some more info. Apparently, in windows the standard is UTF-16, that is, the cheap version of UTF-8. The pros is that characters are always 16 bits wide instead of varying length as it is in UTF-8. That makes for extremely easy encoding as you can beforehand reserve space for said amount of characters. But, UTF-16 is not UTF-8 and therefor does not support all the characters UTF-8 supports.

Now, windowsdevelopers refer to UTF-16 as unicode and Linuxdevelopers refer to UTF-8 as unicode. Hence the confusion on my part. A more correct term would be to not refer to UTF-16 as unicode at all but rather keep using the term “wide” and nothing else. The standard libs to handle unicode under Linux handles UTF-8 and on windows UTF-16. Pretty annoying, this probably means I have to get some external lib to be able to support UTF-8 on windows.

]]> By: Mind http://blog.isecore.net/2007/10/06/them-character-encoding-blues/comment-page-1/#comment-83090 Mind Sat, 06 Oct 2007 05:57:45 +0000 http://blog.isecore.net/2007/10/06/them-character-encoding-blues/#comment-83090 Itīs funny, I was just looking up much of this stuff with character encoding while programming on a chat-bot. Sort of my long term project to do a megaHAL replacement that is sligthly more modern and portable. Anyway, I canīt seem to find exactly what sort of character encoding unicode is in the various libraries of C++. The encoding is simply referred to as "wide". I know that UTF-8 is "teh shit", Itīs not really a lack of knowledge on my part about how the character encodings work or what they are good for, itīs more of a lack in documentation on the libraries part or a lack of my ability to find the correct information about it. Really annoying. I have found a standardlib that I use now that I suppose is the same everyone that makes "unicode" supporting programs use. But I am not certain, and I don`t really know what will happen when people start entering text into it with various keyboardsettings and languages or from different media such as the input-box on a webform kontra a well formatted textfile. Itīs a djungle really... at least until I find more reliable information about it. Itīs funny, I was just looking up much of this stuff with character encoding while programming on a chat-bot. Sort of my long term project to do a megaHAL replacement that is sligthly more modern and portable. Anyway, I canīt seem to find exactly what sort of character encoding unicode is in the various libraries of C++. The encoding is simply referred to as “wide”. I know that UTF-8 is “teh shit”, Itīs not really a lack of knowledge on my part about how the character encodings work or what they are good for, itīs more of a lack in documentation on the libraries part or a lack of my ability to find the correct information about it. Really annoying. I have found a standardlib that I use now that I suppose is the same everyone that makes “unicode” supporting programs use. But I am not certain, and I don`t really know what will happen when people start entering text into it with various keyboardsettings and languages or from different media such as the input-box on a webform kontra a well formatted textfile. Itīs a djungle really… at least until I find more reliable information about it.

]]>