I truly understand your concern, and others are giving you the benefit of their experience and advice. The one thing that would concern me as your trainer would be that you become so fixiated on the possibility of your nose being broken, that you start getting very defensive in the ring, or start doing weird things with your guard and stance. In fact, you may become so obsessed, that in a fight, you put yourself in a situation to where it WILL happen.
We play basketball knowing the possibility of an elbow coming down could break our nose, we play football knowing the possibilty of injuries are out there....we even cross a street or get in a car, knowing accidents may happen. If you truly love boxing, tell yourself, it may, or may not happen and get on with it. Otherwise, you may let this start gnawing at the inside of you.