Have you tried it with a winter jumper underneath? Personally I nearly always size up with coats.
I'll be able to wear a hoodie or something underneath it, don't really have any big thick jumpers that I'd wear with it so I should be ok. They've run out of medium now so I can't exchange it anyway.