The problem the Tories have is that most people like the idea of less immigration in the abstract: and they (like all parties to some degree) have stoked and exploited this for electoral gain. Windrush shows that most people really don't like the reality of less immigration, which is harsh immigration controls - inevitably resulting in their neighbour or co-worker or aunt being deported.
So now here we are.
So now here we are.