You should be fine. Although, your 8-ohm speaker might sound less powerful than your 4-ohm one, but then again, if you are really pumping up the volume, there's a point where you'll be hard at telling a difference with the sound. The 8-ohm and 4-ohm impedance rating are their to guide you in terms of how much power your amplifier can handle. Normally, the lower your impedance rating, the more current your speaker draws. This would translate to a higher power requirement, which may be too much for your amplifier.