The ever-so "slight" difference between Hitler and all the Jeffersons of the past is that Nazism was a pseudo-scientific ideology deliberately exterminating everything it didn't agree with; it was a destructive movement.

Whereas all those white (oh my gosh!!) men from the enlightenment were merely displaying the flaws of their times (what else would you expect?), whilst building the democracies and scientific development we have today, i.e., they were entirely and deliberately constructive movements.

Why is it so hard to tell the difference between the two?

Like I said before, in 100 years' time someone will want to tear down a statue of Elon Musk because the cobalt in his phone was mined by children and he didn't even give a fuck (just like all of us in the 2020's).