I think Haye don't want to be around him because he is being made to look a bit foolish with his insults and put downs. Klit is being respectful and saying it how it is so Haye cant find any reason to fire insults at him anymore. So what else does he have to say? not much and he doesnt want to be sat there looking awkward and uncomfortable. Easier to sit back stage and say he sounds like Borat - a bit childish but Klit still managed to turn it around and make a joke about it.