I ride off and on in the cold, depending on how I feel.
I think the coldest I've ridden in is about 10 degrees farenheit for about 30 minutes. not really the most comfortable ride ever, but i did it in pretty much regular gear (nothing heated).
How i gear up for the cold:
ski mask/balaclava w/ full faced helmet
scarf for the neck
long sleeve shirt/under armor
2nd jacket under my thick leather jacket
glacier gloves (
http://www.glacieroutdoor.com/products_cycling.php)
long johns/thermal underwear
leather chaps (cheap and decent, but not great:
http://stores.ebay.com/My-Leather-Mall_Leather-Chaps-Pants_W0QQ_fsubZ20787792...)
wool socks
winter boots (leather boots with thermal insulation that zip up on the side)
couldn't move my hands all that well when i got where i was going, but it was a great way to wake up in the morning.
a nice trick to keep your hands warm, depending on the number of red lights you get and your gloves, is that you can "hug the jug", or reach down and grip the engine with your gloves. gives you a nice warm feeling.
oh, and this might sound silly, but tuck your shirts in. makes a WORLD of difference.
... back to your actual question, i don't know of any real cheap place, but i usually shop around at
http://www.motorcycle-superstore.com/ it has some decent deals every now and then.