I think it probably does need rest from strenuous cardio activity (biking, lots of running, etc.) But to do any of those activities you have to use some major muscle group (in the named examples your thighs), and those get tired too, making you want to rest.
When you rest the major muscle group you exercised, you rest your heart as well.
Edit: There is a reason why people with heart disease shouldn't do -very- strenuous cardio activity (i.e. no 130 mile bike rides, at least in one day anyway.)