yes.
use apt-get remove --purge <package>
the --purge makes it also remove the config files