Duk Koo Kim and Frankie Campbell, have to mention them even if they weren't great in the sense of the OP's intention with this thread, as both left a big impact on society and the sport.

ALSO, Obviously the stipulation implied is that we lost them when they were still active, but I want to add Rocky Marciano as he was only 45 years old when his plane went down on approach in heavy rain to Des Moines, Iowa, August 31, 1969, piloted by Glen Belz. 45 is certainly too early. We can then add Trevor Berbick who was no slouch.