I think people just don't like excessive immigration or think it normal that London became a minority white city in a handful of decades. It would be like Tokyo being half white by 2050. Most would consider that extreme change. Nobody minds a bit of immigration but it should be consensual and incremental.