For sniffing/scanning, one of my favorite tools is the NT/2K port of nmap. Let me know if you want me to e-mail it to you. It's small, fast, and freeware; it does port scanning, spoofing, open share checks, etc. I would just give you an URL to get it yourself, but the few sites that I knew had it must've pulled the program recently...

You'll need a packet driver (here):
WinPcap.
Another nice, free, and very fast port scanner is 7th Sphere PortScan. It's less than 30k. Let me know if you can't find it after doing a web search.
The Foundstone site has some nice tools; also try
Hacking Exposed. The guys who wrote the Hacking Exposed book (a MUST have, IMHO) run the Foundstone/Hacking Exposed sites. Make sure you visit the tools section:
HE Tools.
As far as "attacking" goes, I wouldn't if I were you. Besides the fact that you risk the chance that you may quickly find yourself out of your depth in a war against someone who has a lot more experience (plus all his buddies), you'll also lose credibility with the offender's ISP, etc. if you are able to track him that far. If you're really worried about security I recommend a good SOHO hardware firewall -- but if you can't spend $500+, you might want to consider asking a Linux guru to help you set up a basic Linux firewall/router box. Damaged might be willing to help in that dept.
If you want to stick with Windows-based software, I recommend a couple programs: BlackIce and AtGuard. I think you can still dig up AG
here. Unfortunately, it was bought by Symantec and was "bloatified" into Norton Personal Firewall or something. Let me know if you can't find it; IMHO, it's one of the best incoming/outgoing port monitoring/blocking programs out there. It also blocks cookies, popups, ads, refers, etc.
Edit: More links below...
Retina/Iris (nmap used to be here, too):
eEye
Ping Plotter (GUI traceroute):
PP
VisualRoute (another graphical traceroute):
VisualRoute
NeoTrace (yet another GTR):
NeoTrace
Atelier Port Scanner:
AWSPS
*nix nmap (may have a link to NT version):
nmap
Misc small, free utils:
AnalogX