You have to be careful about the eyes. Emile Griffith would "eye-feint" you into knots, I've been told.
I liken this to driving on a busy road. You can't focus one one car or the other. Instead, you have to see the whole picture. Hands, shoulders, all of it. And there is an element of mind-reading as well, because you and the opponent are constantly trying to out-think one another.
And, of course, in certain situations you know the guy is going to throw a hook because that is what is done in that situation.