My road bike wheel wobbles?

I was adjusting the brakes to give me better leverage (less squeeze before I started slowing down) when I realized my wheel wobbles slightly. I was spinning the wheel and on the left side it hits for a second or two and then aligns again indicating that there's a wobble. I've checked and the axle itself isn't loose. Could it be the spokes or something? How can I fix this?