If floyd loses I think it depends on how he loses, if he gets fucked up like Hatton then yes it's going to seriously hurt his legacy but come on pacman is an all time great. If it's a close fight and floyd wins a rematch them it doesn't do much to his legacy IMO.
I actually really like pacman Im obviously a floyd fan and really want him to win but it's roach I really dislike I don't think I could handle the shit that guy would talk if pacman was to win.