Might depend on what you start with. I'm a small female and my muscles pretty much all increased in size a bit with boxing. I also know a large male body builder who became a bit smaller when training to fight.

it probably also depends on how you spend that one hour "punching".

Can you give us more information about what you do?

A lot of people on this forum who know more about this than me might be able to suggest some other things you can do.