I've got the factory stereo with the cassette slot and the 6 CD changer in
back.

The volume knob has trouble controlling the volume when the dash is cold.
When I turn the knob the volume goes up instead of down. Once the dash is
warm from the heater or a warm day, the knob works pretty well, but still
can't be turned up or down real fast or the volume just goes up.

Any ideas?

I don't want to replace the stereo with after market products (I really
prefer the factory models), and certainly can't afford a factory replacement
(I think it's like $700).