Since it's gotten cold I have a belt(s) that make a squeaking noise for a split second right when I start my car (cold), no other time. Is it just because of the cold or is this a sign of a wearing belt?

 

It could be loose, try tighting it up.

 

You could try tightening up the altenator belt and A/C belt or you could shell out the $16 or so and get new belts that will be tighter and won't flex that much. I should've done that instead of trying to tighten mine for a week after I started hearing the squeaky belts (embarrassing).