Move the lot in my view. Parliament, Whitehall and any associated functions. It would help another city in the country and create a counterweight to the malign influence London has on the nation. The decrepitude of Parliament is the perfect opportunity - move that first, then government departments (accounting for commitments to building leases, etc.). It would probably take at least ten years to do but would be worth it in the long run.
Given technological developments, there could even be a multi-centre model, with departments scattered in different cities across the North and Midlands, but with Parliament somewhere in the North (i.e. as far from London as possible in England).