He's too small and he's not durable enough.

Holyfield was able to jump from cruiser to heavyweight and people still thought he was too small and he was 6'3 as well. But Evander had great stamina, work ethic, and he took a good punch.....Haye doesn't