Having now scanned the transcript of the interview I'm still not a lot clearer on his 'fairness' position.
None of the LTNs have made a road that was previously really quiet into a busy road - at worst they've made an already busy road a bit busier (though as Leigham Court Road data a bit upthread, quite possibly only as busy as it was a decade or so ago before satnavs were being widely used). Whereas the satnav impact on minor road in many cases has made roads that were quiet much busier.
Main road housing isn't disproportionally social housing (based on the income distribution data above) so the vast majority of main road residents have chosen to live there - trading off a larger home, or a home in a more "expensive" area against a smaller home or one in a less expensive area on something other than a main road.