There are loads of different types of smart cards - the problem is that you need the appropriate software for the type of card you wish to use.
Some cards are relatively basic, and simply provide a secure way to store confidential data (e.g. a security certificate for logging onto a secure web-site, or for connecting to a secure Wireless LAN) - the digital certificate file is much more secure than a password, and the smartcard provides a method of storing the data so that it cannot be copied or tampered with, as well as requiring a PIN to release the data. We have these at work to restrict access to workstations with access to confidential data. Instead of logging on with a username and password, you put your card into the reader and enter your PIN. No card, no access. Pull your card out - instant log out.
More advanced cards have more sophisticated CPUs or OSs on them - e.g. JavaCard, if you haven't already guessed, uses a Java CPU and stripped down OS. You can install simple applications onto the card for managing secure data. Even more sophisticated cards may add encryption accelerators for stronger encryption (128 or 256 bit encryption is a bit much for a standard smart card CPU).
This allows smart cards to actually perform processing themselves, instead of just referring to a server. E.g. several mass transit systems use smart-cards as reusable tickets. The card communicates with the reader, and the card records where you started the journey, where you got off, etc. and deducts the appropriate fare (upgrading the ticket as required, depending on the exact journey you take, but ensuring that you are only charged the lowest possible fare) - even if the card reader isn't connected to a server (e.g. on a bus). The secure communication system makes it highly tamper resistant, so that hackers can't add credit to a card, etc. At the same time there is no need for the readers to record the transactions, so no liberty-infringing database of people's movements.