I recently changed the battery due to the fact that I saw the battery light come on and a/c dim the car.

I took it to Sears, they tested the battery, it was bad, they changed it.

Today as I was driving I felt the a/c get warm, then saw that the battery light was on. After driving it I kind of revved it to a high RPM and then the battery light eventually went off completely.

Is it my alternator that is causing this problem?

 

yup, it's the alternator. you can take it out and get it tested.