Cheapest upgrades; arm warmers, leg warmers (or just lycra leggings), ear warmers, overshoes, gloves. If you want to go cycle-specific the layering of core warming stuff can get a bit expensive... but it certainly exists. As bees says, thermal base, jersey (potentially long-sleeve), then jackets and rain capes, then the gilet (sleeveless jacket), which can be worn on top of any other layer (though usually over a jersey). The advantage of cycle stuff is that it's often designed to fit in a jersey back pocket, and is very layer-oriented (better fine-tuning of temperature).
The cheapest things I mentioned (arm warmers etc) you can find stuff that does the job on riverine commerce sites... Overshoes can get expensive, but I have some cheap ones from Planet X that work fine (out of stock mind), and given that they are on your feet, dropping £60+ on them seems a bit daft. Gloves also can get expensive, but are kind of essential either way. Foot warming is definitely better dealt with using covers than socks, as thick socks will change shoe fit and can cause injury, particularly if circulation is poor due to cold.
Personally have yet to fully embrace cycle clothing, have some old commuter stuff I use. T-shirt, Lightish thermal, Gore-tex commuter jacket for coldest days. Under armour leggings over bib shorts, with MTB shorts on top of that. Optional leg warmers and arm warmers, headbandy fleecy thing that covers ears. But I end up carrying a small backpack... At some point I'll probably move to cycle stuff to keep things minimal. But I am not ready for external lyrca just yet.