Haye would back out of the fight 2 weeks before it happens