Are you referring to console redirection over serial port? Or are you just using remote KVM over network?
Not sure how first one works. Second, it is not related to esx, it could be your java version, try different computer, or reinstall java or something.
Try hitting ALT + F12, it should show you live vmkernel logs. ALT+F2 takes it back to main screen.