Right.
I have vague memories of a conversation I had in the Navajo nation...during which time it was explained to me that anyone, Navajo or not, could move in and live in the Navajo communities, but with the proviso that their neighbours agreed to it.
Now, in smaller, more rural communities you could easily imagine a similar scenario playing out hypothetically. An incomer wants to move in, the community discusses it and decides whether they want it to happen or not.
Seems fine to me.
Now, in larger, more transient populations such matters are far more complex. partly because of scale, partly because of pace and partly because the concept of community is less easily nailed down. But still a possible starting point. After all who better to make judgements about whether a community would benefit from "incomers" than the community itself.
*By community, I'm using geographical populations as a starting point (village, street, block of flats, whatever) rather than ethno/cultural/religious/social groups.