Not sure it makes much odds tbh. Whatever the exact motivation behind Leave activists or voters the effect of them winning will be to erect a wall of border and residency bureaucracy where it doesn't currently exist. Refugees and others from outside the EU already face a wall. EU citizens currently don't, but will in the event of Brexit. All of them/us.
As for "lack of democratic accountability", I haven't noticed any in the negotiations Cameron has been carrying out. Some tosh about in-work benefits, a populist but financially irrelevant hit on children and something incomprehensible about protecting the City. Was that what the campaigners have been going on about for decades? No of course it wasn't, it was decided on behind closed doors.
In the event of Brexit, who will be accountable for the size and shape of the wall, and of all the other changes that will be imposed on us? Who will decide the precise details of the negotiations between our Plucky Island and the EU, between the Jolly Good Chaps and the US government/Wall St (is there an acronym for TTIP targetted specifically at us?) and what the poxy City can and can't do? I'll tell you- the Tory government between now and 2020, unfettered so long as their majority holds. Any new incoming government in 2020 will have to renegotiate anything they don't like. That isn't democratic accountability, it's a blank piece of paper for a redesign of what it means to be British.