Hi,
yes, you first install ESXi on the hardware. doesn't matter it you do it for one host first or all 4. Has to be done 4 times anyway 
then you connect to one of your ESXi server with the vSphere Client ( can be downloaded at http://your-esxi-ip/ ) create a Virtual Maschine and install a windows Server VM and install vCenter on this OR you simply deploy the vCenter appliance which can be downloaded as a OVF template from the VMware website ( deploying the appliance is much faster and will be fine for your environment )
Take a look at the vSphere Setup and Installation Guide: http://pubs.vmware.com/vsphere-51/topic/com.vmware.ICbase/PDF/vsphere-esxi-vcenter-server-51-installation-setup-guide.pd…
Regards