Did you write your own /etc/X11/xorg.conf? According to xorg's man page exists an option to disable virtual terminal support (DontVTSwitch). If this option is set to true then you cannot use the virtual terminals while the xserver is running.