You have to options I can think of immediatly to accomplish your task.
Option 1 is using Remote software:
Connect to Computer B through Remote Desktop Protocol (RDP) which is now built into XP Pro. Being that it is on your internal LAN you shouldn't have to worry about any ports or what not.
On the host computer, computer B in this case, just right click on My Computer and click properties. In the systems properties dialog you should see a remote tab on top. Click the box to allow users to remotely connect to this computer. You may have to set up a login anad password here as well.
On the client computer (Computer A in your example) Just go to Start -> All Programs -> Accessories -> Communications -> Remote Desktop Connection. In the dialog that pops up type in the IP or hostname of ComputerB. Then type in the login and password you specified above and blamo you will have full control just as if you were sitting at that machine.
If you don't have XP Pro you could do the same thing with VNC which is an Open source download of a remote control app. More details on VNC can be found
here
The other option would be to purchase a hardware KVM and just switch to the other unit when you want to start up the MP3 player or whatever. You can get little cheapy 2 port KVM's for less than $40.
Here are some options
Good Luck and hope that helps somewhat!