Maybe that's it - I can't think of an example of how they're different accents so it could be down to perception. Will have to listen out next time I'm in north London
If I was to tell you I was from West London and aforementioned pal from East London, you might well think "ahhhh, there's an East/West accent" and labour under the impression that West Londoners have lazier Rs (or whatever) for the rest of your life
As it happens my mum was bought up in West London and hers in East London, and their accents probably did influence our own. I've also got more posh friends than her, which I'm sure has a similar effect, or did somewhere down the line.
It could also be that people from certain parts of London are more transient than others. I know people from North and East London who'll never move far because that's where their respective communities are based.