People break bones in their hands all the time. With time, your hands will toughen up, but be careful not to over do it. I'd stick with the shadow boxing for at least a few more days or until your hand feels better.

I broke a knuckle several years ago in a fight. I just recently bought a heavy bag for exercise and within a couple of days, I felt a little pain in the same spot. Anyway... it's been probably 4-6 weeks and my hands feel good and are noticeably stronger and tougher than they were when I first bought the heavy bag. Just make sure you're not pushing it too hard because injuries can set you back far more than taking it easy.