we did have some thoughts about this when the vermin were in one of their turmoils, and the 'what if that twat johnson was re-elected party leader now he's not no longer an MP?' question came up.
taking it more from the 'what if the leader of the party who won the general election lost their seat?' angle,
institute of government says dunno although it raises the question of how a PM who wasn't an MP could deal with prime minister's questions and so on.
there doesn't seem to be a specific rule that PM absolutely has to be an MP. Some parties have a rule that party leader has to be a sitting MP, but that's up to each party. The leader of the SNP is an MSP not a westminster MP, but since they only stand for election in a small proportion of UK seats, there's little chance of the SNP forming a government at westminster.
alec douglas-home was PM for 4 days as a member of the house of lords, then for a few weeks without being an MP while a by-election was set up for him.