It is now officially winter and if it isn't already cold where you live, there is a very good chance that it will be very soon. Dressing to go out into the cold and be active can be very tricky, especially if you are doing something like cycling which can generate a lot of heat and sweat. Obviously, some areas will be more wintry than others, but there are a few guidelines that will help you dress as effectively as possible.
Wherever you live, the key is to be as warm as possible and to eliminate as much unnecessary water(sweat) as possible. If you get wet, whether it is from sweat, rain or snow, water conducts heat and cold much more efficiently than being dry which means you can get chilled much more easily. So, the two key goals are to be warm and also to be dry.
There are two keys to keeping dry. The first is to dress with just enough clothes so that you are warm, but not overly so. While it is probably better to err on the side of being a bit too warm, there is definitely a balance. Secondly, try to wear clothes which "breath". You want your fabrics to wick water away from your skin and out into the atmosphere.
One mistake I see, especially in climates that are only moderately cold, is cyclists wearing a jacket to keep themselves warm. Nylon or Goretex wind/rain jackets are very poor when it comes to breathing. Avoid these at all costs. A good winter jacket is made of a very breathable fabric like polyester.
In moderately cold climates, unless there is a significant wind factor (that includes going downhill) a jacket is just about the worst thing you can wear. Instead, what you want is a whole bunch of breathable layers.
Where I live in Northern California the morning temperatures can hover in the mid 30F's, but usually warm up into at least the mid 40F's. These are ideal conditions for layering. I wear a base layer t-shirt. Over that I put a short sleeve jersey and arm warmers. Over that I put a medium weight long sleeve jersey. That gives me three layers on my body and two layers on my arms.
On my legs, I wear full-length leg warmers. On my feet, I have wool socks and toe covers or full shoe covers. On my hands I wear knit gloves which breath way better than gloves with a nylon or similar fabric shell. I don't carry a jacket unless I am going to be doing a lot of climbing and then I only put it on for prolonged(>5 minutes) descents.
Obviously, conditions vary across the United States, but wherever you are, I would suggest trying to use layers rather than a nylon windshell or Goretex jacket. I think you will be surprised at how warm you can be and dry as well.