one thing to remember...speed equals power. If two (hypothetical) people are exactly the same mass and have exactly the same form, then the one who is faster will have the most power. That's why you often see a smaller person who can hit harder than a larger one. Some things that help increase punching power:
i) use the legs and hips to generate torque, adding to power
ii) turning the wrist to generate torque
iii) staying relaxed until right before the moment of impact...helps give the whip-like snap

as far as which muscles to work, work all of em. Having a lot of forearm, wrist, and hand strength never hurts.