I believe those keys were created when I ran the sshstart script.
In /etc/ssh I have:
moduli
primes
ssh_config
ssh_host_dsa_key
ssh_host_dsa_key.pub
ssh_host_key
ssh_host_key.pub
ssh_host_rsa_key
ssh_host_rsa_key.pub
sshd_config

I reboot and started again and re-run that script, changed root passwd and it's made no difference.
Can still ping but not ssh to or from the live boot machine.