Try changing /etc/sysconfig/i18n to something like:
LANG="en_US"
COUNTRY="US"
LANGUAGE="en"
CHARSET="iso8859-1"
XMODIFIERS=""
or
LANG="en_GB"
COUNTRY="GB"
LANGUAGE="en"

You could also install localepurge with apt-get install localepurge