The fix is to put the vCloud Direcotr console portal address in the proxy exclusion list. We need to put exact address in the list. put something like *.contoso.int will not work. After add vcloud-con.contoso.int in the proxy bypass list, the console access worked.
IE, such a masterpiece...