Warren hates Haye so much that he would not want to be embarrassed as he was when he fought Enzo.

Haye/Booth thinks Chisora is not worthy of fighting him.

Forget what us all think that is the bottom line, personality issues and not enough money.