Do what you enjoy.
What are your goals? Just fitness? Why not do both. Just watch when you box and the guy dips low you don't knee him in the face, I nearly did that once it's a reflex action