A proxy server must be running for the "non-server" PC to gain access to the internet.
The machine with the proxy server installed, may or may not access the internet via the proxy server. ( This is configurable ).
I have installed WinProxy by Ositis Software at several locations and find it easy to use and their website is very helpful.
A proxy server can work with a Dialup or Constant Connection such as DSL or Cable.
Two computers on a network can share a single internet connection if a proxy server is installed. If you do not have a proxy running, the computers may steal the IP from each other. This depends on your provider. Generally, they allow you 1 dynamic IP. When 2 computers share this, they can steal it from each other if you do not install a proxy server.
Win98se also has modem sharing/internet sharing sofware. I have not used it, but it is probably a lite proxy server.
Strike