Maybe he has left because there is some truth in him caling out Khan ?
Or maybe he actually realised warren was holding his fights in a morgue.