utf-8 console

[Albretch]

~
I code java and I use international characters that are not part of latin-1
~
How can I make the console print these characters as they are supposed to be?
~
this is the conf I am using
~
$ cat /proc/cmdline
ramdisk_size=100000 lang=en vt.default_utf8=0 apm=power-off vga=791 initrd=minirt.gz nomce quiet loglevel=0 tz=localtime BOOT_IMAGE=linux init 2 xmodule=vesa fb1280x1024 depth=16 fromhd=/dev/sdb5
~
LXDE's leafpad does show characters as utf8, but the console and kate (which I install after starting up) do not
~
Thank you
lbrtchx

[mecho]

For the console setup you may want to have a look at this site:

http://sourcemage.org/HowTo/Language

It is well explained. About Kate I don't really know. I use Geany and it has support for international locales. I personally use Cyrillic fonts with Geany. It is also very feature rich

[Albretch]

~
Well I tried to follow the instructions on that link:
~
http://sourcemage.org/HowTo/Language
~
but I wan't able to reset my terminal. I have noticed leafpad does display non latin-1 chars (as those with accents)
~
// __ /etc/sysconfig/keymap file not there even though a number of apparently similar files were found
~

# cat /etc/sysconfig/keymap
cat: /etc/sysconfig/keymap: No such file or directory

~


# find / -type f -name keymap
/media/sdb6/knet/knoppix/.xine/keymap
/UNIONFS/usr/share/initramfs-tools/hooks/keymap
/UNIONFS/usr/share/initramfs-tools/scripts/init-top/keymap
/UNIONFS/lib/udev/keymap
/KNOPPIX/lib/udev/keymap
/KNOPPIX/usr/share/initramfs-tools/hooks/keymap
/KNOPPIX/usr/share/initramfs-tools/scripts/init-top/keymap

~
// __ I guess knoppix 6.2 uses console-tools
~


# ls -l /usr/share/keymaps
total 18
drwxr-xr-x 2 root root 2048 Jul 19 2009 amiga
drwxr-xr-x 2 root root 2048 Jul 19 2009 atari
drwxr-xr-x 8 root root 2048 Jul 8 2008 i386
drwxr-xr-x 2 root root 2048 Jul 19 2009 include
drwxr-xr-x 2 root root 6144 Jul 19 2009 mac
drwxr-xr-x 2 root root 4096 Jul 19 2009 sun

~


# cat /usr/share/kbd/keymaps
cat: /usr/share/kbd/keymaps: No such file or directory

~


# dpkg -l | grep console
ii console-common 0.7.80 basic infrastructure for text console config
ii console-data 2:1.09-2 keymaps, fonts, charset maps, fallback table
ii console-setup 1.46 console font and keymap setup program
ii console-terminus 4.28-2 Fixed-width fonts for fast reading on the Li
ii console-tools 1:0.2.3dbs-65.1 Linux console and font utilities
ii consolekit 0.4.1-2 framework for defining and tracking users, s
ii libconsole 1:0.2.3dbs-65.1 Shared libraries for Linux console and font
ii libjline-java 0.9.94-1 Java library for handling console input
ii libsvga1 1:1.4.3-27 console SVGA display libraries
ii linphone-nox 3.2.1-1 SIP softphone - console-only client
ii pppstatus 0.4.2-10 console-based PPP status monitor

~
so I couldn't set KEYMAP
~
Are you using knoppix 6,2? How exactly did you reset your terminal to use UTF-8?
~
Thanks
lbrtchx

[mecho]

Hi, I am sorry for the late answer. This is how I set my console to display Cyrillic character properly. I removed completely console-tools package and setup kbd package instead.
Then enter this commands as root:

echo -n -e '\033%G' - tells the console that incoming data is UTF-8
setfont Cyr_a8x16 - loads my Cyrillic font
kbd_mode -u - sets the keyboard mode to be UTF-8
loadkeys bg_pho-utf8 - loads my Cyrillic phonetic keyboard layout

I have this in a shell script and run it whenever I need Cyrillic in console. I still don't know how to save the configuration. Hope this was in any way helpful

[mecho]

Please discard my previous message. I figured it out how to set special characters without the kbd package-only using what's available in knoppix.
You have to open and edit the configuration file /etc/default/console-setup
There you can change your encodings, codesets and load different fonts. The file is pretty explenatory. Here is mine just to give you an idea how I load cyrillic characters and end even switch between latin and cyrillic in console :
charmap for cyrillic phonetic is cp1251
codeset for my cyrillic is CyrKoi
I am loading fontface Terminus
Then I have my keyboard info and I am switching between latin and cyrillic using Alt+SHift
using : grp:alt_shift_toggle

# A configuration file for setupcon

# Change to "yes" and setupcon will explain what is being doing
VERBOSE_OUTPUT=yes

# Setup these consoles. Most people do not need to change this.
ACTIVE_CONSOLES="/dev/tty[1-4]"

# Put here your encoding. Valid charmaps are: UTF-8 ARMSCII-8 CP1251
# CP1255 CP1256 GEORGIAN-ACADEMY GEORGIAN-PS IBM1133 ISIRI-3342
# ISO-8859-1 ISO-8859-2 ISO-8859-3 ISO-8859-4 ISO-8859-5 ISO-8859-6
# ISO-8859-7 ISO-8859-8 ISO-8859-9 ISO-8859-10 ISO-8859-11 ISO-8859-13
# ISO-8859-14 ISO-8859-15 ISO-8859-16 KOI8-R KOI8-U TIS-620 VISCII
# CHARMAP="ISO-8859-15"
CHARMAP="CP1251"

# The codeset determines which symbols are supported by the font.
# Valid codesets are: Arabic Armenian CyrAsia CyrKoi CyrSlav Ethiopian
# Georgian Greek Hebrew Lao Lat15 Lat2 Lat38 Lat7 Thai Uni1 Uni2 Uni3
# Vietnamese. Read README.fonts for explanation.
# CODESET="Lat15"
CODESET="CyrKoi"

# Valid font faces are: VGA (sizes 8, 14 and 16), Terminus (sizes
# 12x6, 14, 16, 20x10, 24x12, 28x14 and 32x16), TerminusBold (sizes
# 14, 16, 20x10, 24x12, 28x14 and 32x16), TerminusBoldVGA (sizes 14
# and 16), Fixed (sizes 13, 14, 15, 16 and 18), Goha (sizes 12, 14 and
# 16), GohaClassic (sizes 12, 14 and 16).
# Set FONTFACE and FONTSIZE to an empty string if you want setupcon to
# set up the keyboard but to leave the console font unchanged.
FONTFACE="Terminus"
FONTSIZE="16"

# You can also directly specify nonstandard font and ACM to load.
# Use space as separator if you want to load more than one font.
# FONT='lat9w-08.psf.gz /usr/local/share/braillefonts/brl-08.psf'
# ACM=/usr/local/share/consoletrans/my_special_encoding.acm


# The following variables describe your keyboard and can have the same
# values as the XkbModel, XkbLayout, XkbVariant and XkbOptions options
# in /etc/X11/xorg.conf.
XKBMODEL="pc105"
XKBLAYOUT="us,bg"
# XKBVARIANT="nodeadkeys"
XKBVARIANT="altgr-intl"
# XKBOPTIONS="lv3:ralt_switch,terminate:ctrl_alt_bksp"
XKBOPTIONS="grp:alt_shift_toggle,terminate:ctrl_alt_bksp"

Hope that helps :D

[mecho]

You can use UTF-8 encoding even though its not mentioned in the config file. Also I forgot to mention - you have to run SETUPCON in console to apply the configuration

[Albretch]

~
Once the desktop finishes the initialization all the way to init 5
~
$ sudo ls -l /etc/default/console-setup
-rw-r--r-- 1 root root 1882 May 21 06:57 /etc/default/console-setup
~
knoppix@Microknoppix:~$ sudo vi /etc/default/console-setup
~
knoppix@Microknoppix:~$ cat /etc/default/console-setup
VERBOSE_OUTPUT=no

ACTIVE_CONSOLES="/dev/tty[1-4]"

#CHARMAP="ISO-8859-15"
CHARMAP="UTF-8"

CODESET="Lat15"

FONTFACE=""
FONTSIZE=""

# FONT='lat9w-08.psf.gz /usr/local/share/braillefonts/brl-08.psf'
# ACM=/usr/local/share/consoletrans/my_special_encoding.acm

XKBMODEL="pc105"
XKBLAYOUT="us,gb"
#XKBVARIANT="nodeadkeys"
XKBVARIANT="altgr-intl"
#XKBOPTIONS="lv3:ralt_switch,terminate:ctrl_alt_bk sp"
XKBOPTIONS="grp:alt_shift_toggle,terminate:ctrl_al t_bksp"

~
I have also tried:
~
CODESET="Lat38"
~
$ sudo setupcon -v
We are not on the Linux console, exiting.
~
$ sudo setupcon -v --force >& ~/tmp/setupcon.out
~
$ cat ~/tmp/setupcon.out
Loading /etc/console-setup/cached.kmap.gz
~
$ sudo ls -l /etc/console-setup/cached.kmap.gz
-rw-r--r-- 1 root root 3886 May 21 07:21 /etc/console-setup/cached.kmap.gz
~
$ javac -encoding UTF-8 esp_Silabas00Test.java
~
$ java esp_Silabas00Test

"?Ay!"
~
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
~
If I open the file using leafpad I can see what I need to see
~
"¡Ay!"
~
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
~
I have also done a few searches and I don't find much explanation it, even though many people should have these problems (I guess). Where did you get your info from?
~
Thank you
lbrtchx

[Albretch]

Also when you say:
~
"using what's available in knoppix."
~
you are not specifying what version of knoppix you are using
~
With other versions of knoppix I didn't have any problems, with 6.2 I do and it doesn't seem to be documented well enough or I can't find the info
~
Thanks
lbrtchx

[mecho]

I am also using knoppix 6.2
I don't understand where you want your special characters to appear - in Linux console or lxterminal ???
In the console if you do what wrote in my previous message you should be fine, but you have to issue the setupcon command after every boot of knoppix. I still can't figure it out how to save the changes. Some knoppix script is resposible for this dynamic changes.
I have achieved cyrillic characters in lxterminal by editing knoppix-autoconfig script which is in /etc/init.d
Because I boot knoppix with default US I had to make some changes in this line:

LANGUAGE="us"; COUNTRY="us"; LANG="C"; KEYTABLE="us"; XKEYBOARD="us"; KDEKEYBOARD="us"; CHARSET="iso8859-1"; KDEKEYBOARDS="de(nodeadkeys),fr"; TZ="America/New_York" ;;

I made it look like this:

LANGUAGE="us"; COUNTRY="us"; LANG="bg_BG"; KEYTABLE="us"; XKEYBOARD="bg"; KDEKEYBOARD="bg"; CHARSET="microsoft-cp1251"; KDEKEYBOARDS="de(nodeadkeys),fr"; TZ="America/New_York" ;;

This way I have the cyrillic CP1251 charset and I am able to type in cyrillic in lxterminal.Everything is a bit messy in KNoppix because of this boot scripts, but I am running my laptop entirely from usb, because my hard drive is too noisy and I have the bonus to have my OS with me everywhere I go. The information I have from bulgarian site:
http://bg-howto.hit.bg/
Google translates it quite well

[mecho]

You said you that you are coding java - this will be a bit more helpful for you.

http://blog.andrewbeacock.com/2007/01/how-to-change-your-default-locale-on.html

This problem is explained there. The difference is that in knoppix /etc/default/locale and /etc/environment are automatically generated by knoppix-autoconfig and you will have to edit that script the generate the proper values for you

[mecho]

Notice also the difference - in knoppix 6.2 the the list of supported locales is in /var/lib/belocs/locales.dep not in /var/lib/locales/supported.d/local as the article says. Knoppix uses belocs-locales package instead of locales that's the reason for this difference

[mecho]

Here is the final polished solution to the problem of displaying characters properly in terminal and all other programs using default locale.
Open and edit /etc/locale.gen
There are many preconfigured locales inside, just find the right one and unmark it. If it does not exist - create it. For example If you want all your menus and programs to be displayed in English, but you need Cyrillic phonetic characters to be properly displayed then type:
en_US.CP1251 CP1251 and save the file. Unmark any other locales which you would need in the future as well.
Next step is to generate the locales using:
sudo dpkg-reconfigure locales
By default knoppix uses belocs-locales, so you might have to to this using:
sudo dpkg-reconfigure belocs-locales-bin
After this is done open /etc/init.d/knoppix-autoconfig and navigate to line 290, just below
*) # American version.
Edit LANG="C" to LANG="en_US.CP1251" or whatever locale you created. I also changed the line CHARSET="iso8859-1" to CHARSET="microsoft-cp1251" - although I am not sure if it's necessary.
Save and reboot. Done
If you want knoppix to boot by default in any of the preconfigured languages which are in knoppix-autoconfig, you will have to edit:
/mnt-system/boot/syslinux/syslinux.cfg - go to the line bellow LABEL knoppix,KERNEL linux and change LANG=en to whatever language you want knoppix to boot by default

Hope this was useful