I think he just struggles against quality opposition. Saying southpaws "rocked him" or "gave him trouble" may be true but so did the orthodox fighters. I don't think it has anything to do with stance but more so fighting style. Cotto and Maidana both gave him trouble in recent years, more than any southpaw, and maidana and mosely both rocked him harder than any southpaw has. I think his issue comes with boxers that aren't conventional. WHen he fights awkward fighters he runs into a bit of an adjustment period. On the other hand, when he fights pure boxers (like Canelo) he cruises to a victory.