I wrote something like this, except instead of sounding an alarm, it logs it. A number of years ago, when my ISP was denying that they were having reliability issues, I was able to rattle off to them the exact times and durations of each outage that week, which eventually earned me an apologetic call from some regional manager offering me service credit.

I still left them, though.
These days, I use monitor.us to send me e-mail alerts (and my phone has been set to ring an alarm when I get one of them) when my VPSs, the home server at my parents' house, or my home server goes down (since the servers at my home and my parents' home virtually never go down on their own, if I lose connectivity to them, it's almost always indicative of connectivity problems--this recently helped uncover the increasing flakiness of my parents' old cable modem, which I'll be replacing the next time I fly out there). The downside is that the free service has a resolution of only 30 minutes, but, eh, that's good enough for now.