When I had to cut weight, and essentially lost a lot of my body fat doing so for a fight, I also lost speed, power and energy. Felt like shit too.

Shortly after, had to gain weight for a fight, and lost the 'pretty ab' look...but gained speed IMO, and power and felt a hell of a lot better.

I think core strength is one thing. I mean, core strength I believe will help with speed and power and overall effectiveness for boxing. In both delivering and receiving blows.

I think having a bit of fat, not an overabundance of it, is likely the healthier way to go and if anything, will help more with speed then a fat depleted body per se.