Yeah this isn't a bodybuilding contest.
There are guys who are at the top of their game who, looking at them, you'd pick to lose if you went by 'physical appearance'. Guys like Calzaghe, Enzo, Juan Diaz, even Cotto doesn't come in ripped. Part of that is to do with genetics.
When you're in that ring, looking pretty isn't gonna help. The only advantage I can think of if you're totally ripped is that you'll probably have the muscular size advantage over your opponent.