It's the responsibility of the UK government to come to some sort of agreement with the EU over the details of how the UK leaves the EU.
I, who am a Leave voter, didn't vote in or for any of the politicians who make up the UK government, so my influence on or responsibility for how they conduct themselves is non-existent. I suspect this is true for a significant % of Leave voters, whereas there is also a significant % of Remain voters who did vote for the current UK government (I don't know what the actual figures are, perhaps someone can enlighten both of us).
If you're really seeking to apportion blame or responsibility to individual voters, I suggest you focus on those whose votes at the 2017 GE left us with the incompetent Conservative government led by Theresa May, though TBH this is a fairly pointless exercise.