Noooooooooooooooooooooooooooooooooo

[Red Squirrel]

My online web/mail/etc server had an uptime of several years, it was at over 1200 days last I had checked. The other day I got a heartbeat not received alarm but by the time I checked everything seemed to be fine, so figured the agent just crapped out, did not think more of it and figured I'll go in and restart it later.

Today I noticed I did not get an email I sent myself from work last night, I decided to investigate further so while I was sshed in I started the agent again. Then I got an alarm that my UT3 server was down, odd, but no biggie, have not logged into that in months. Then I get an alarm that postfix is down... ok this is really weird. I type uptime. 1 day! Apparently the server rebooted. Not yet sure why, if it was a power issue at the data centre or an issue with the server itself, but that's quite a bummer. I also lost a few days worth of email because I was too lazy to check into it immediately. Oh well, I don't really get much email. Good news is, everything did come up smoothly more or less, no hard drive failure or anything like that. I ensured that postfix is set to auto start though, not sure why it was not set. I should probably do the same for the agent. Program I coded and it does not have a "service" type setup coded in but I'll have to throw it in /etc/rc/local or something.

At least everything came back up is what counts, uptime is just a number. But damn... going to take a while to hit that again. I'll be in my 30's.

 

[SunnyD]

My online web/mail/etc server had an uptime of several years, it was at over 1200 days last I had checked

You, sir, are the scourge of the internet.

1200 days... I bet you never update packages or libraries either.

 

[Printer Bandit]

What's the point of running your own server?

This isn't 1999.

 

[DesiPower]

 

[Red Squirrel]

You, sir, are the scourge of the internet.

1200 days... I bet you never update packages or libraries either.

It's Linux. Yum update, done. No need to reboot. Though after a while the repos stop updating, but this is production so kind of hard to get around that without having another server to put a newer distro on so you can have both running while you move stuff over. Typically I switch providers by now as I find another deal so while I have both servers running it gives me a chance to transfer stuff over with little to no downtime. Hate rebooting stuff unnecessarily.

 

[JulesMaximus]

How will you go on living now?

 

[SunnyD]

It's Linux. Yum update, done. No need to reboot. Though after a while the repos stop updating, but this is production so kind of hard to get around that without having another server to put a newer distro on so you can have both running while you move stuff over. Typically I switch providers by now as I find another deal so while I have both servers running it gives me a chance to transfer stuff over with little to no downtime. Hate rebooting stuff unnecessarily.

a yum update (or apt-get upgrade, etc) does put new packages, but it doesn't unload and unlink old shared libraries that are in use which leaves you with lingering security vulnerabilities. This is especially true with kernel packages.

In 3 years you probably should have rebooted at least half a dozen times, even on Linux.

The "if it ain't broke" mentality... *sigh*

 

[Jeff7]

a yum update (or apt-get upgrade, etc) does put new packages, but it doesn't unload and unlink old shared libraries that are in use which leaves you with lingering security vulnerabilities. This is especially true with kernel packages.

In 3 years you probably should have rebooted at least half a dozen times, even on Linux.

The "if it ain't broke" mentality... *sigh*

"If it ain't broke, reboot it."

What's the point of running your own server?

This isn't 1999.

Control over your own data.

 

[olds]

Cool story, bro.

 

[MrDudeMan]

You, sir, are the scourge of the internet.

1200 days... I bet you never update packages or libraries either.

lol.

my god... 1200 days? while impressive in some sense, that's way, way too long to go without a reboot.

 

[Platypus]

my god you're even more of an idiot than previously thought, which is saying a LOT.

 

[Insomniator]

What's the point of running your own server?

This isn't 1999.

This

 

[TwiceOver]

My colo server died last Thursday. Don't know what the uptime was, it had been out there for a couple years. Another project I'll have to work on.

 

[Red Squirrel]

my god you're even more of an idiot than previously thought, which is saying a LOT.

Yeah the server somehow turning off on it's own is somehow my doing. Probably a power issue at the DC. I put in a ticket just to make sure. Edit: Just found out it turns out the data centre experienced a secondary power failure.

"If it ain't broke, reboot it."

Control over your own data.

This. Cloud stuff, no thank you.

I could probably do fine with shared hosting, but at one point I was running a MMO game server and needed to host a Windows VM for it so I just ended up keeping that server. It's also nice to be able to manage my own DNS, mail, backups etc...

I typically don't do this much uptime but after 1 year it was a challenge to see how far I can go. 1 year is typical but usually I'd reboot or upgrade something at some point. My home server was going on 1 year but had to do some power upgrades and it only has single psu so could not walk power over to new feeder. I actually have some more plans for power upgrades as I'll need more power if I decide to get into bitcoin mining. Looking into a telco grade -48vdc setup most likely with couple rackmount rectifiers and inverters.

I could almost host all my stuff from home but my ISP does not allow servers.

 

[dave_the_nerd]

Multi-year uptimes are not uncommon for a light-duty server that's got a solid OS on it. I'm impressed, but not surprised. Sorry for your loss.

My favorite sort-of-uptime story was an acquaintance who interned at a small local telco. They had a Sun box in a rack, that had been there for years. Nobody had the passwords to it, they didn't know what it did, but it was putting out a lot of network traffic, it was probably important, and nobody was willing to risk turning it off.

 

[Red Squirrel]

Multi-year uptimes are not uncommon for a light-duty server that's got a solid OS on it. I'm impressed, but not surprised. Sorry for your loss.

My favorite sort-of-uptime story was an acquaintance who interned at a small local telco. They had a Sun box in a rack, that had been there for years. Nobody had the passwords to it, they didn't know what it did, but it was putting out a lot of network traffic, it was probably important, and nobody was willing to risk turning it off.

Hahaha Good ol telco stuff. We have this box we dial into, it's to program operator intercept messages such as "*3 beeps* The number you have dialed is not in service, the new number is ..."

Nobody actually knows where this server physically is, but it's very old and just keeps going. Did I mention we have to DIAL IN to it?

The phone switch is a marvel as well. Uptime of over 30 years. They don't make stuff like they used to. Interesting walking in that room thinking that those fans have been humming non stop for longer than I've been alive.

Heard some crazy stories about Netware servers as well. Those were pretty solid. One story involved a full rack that was completely walled in. They figured it out like 10 years later when doing an inventory.

 

[BudAshes]

Since the original post is about as interesting as dry bread and water, this thread is now about pie.

cherry is best.

 

[Platypus]

Yeah the server somehow turning off on it's own is somehow my doing.

Err.. no I'm referring to the fact that you think it's completely fine to run a web/mail server exposed to the Internet for over 3 years without rebooting to load updated kernels. Unless you're using something like ksplice you've been running some seriously vulnerable code.. and proud of doing so.

 

[Red Squirrel]

Err.. no I'm referring to the fact that you think it's completely fine to run a web/mail server exposed to the Internet for over 3 years without rebooting to load updated kernels. Unless you're using something like ksplice you've been running some seriously vulnerable code.. and proud of doing so.

It gets updated, and logging is watched quite carefully. I don't think Postfix or Dovecot has had very many major issues in the past years anyway.

It's not like Windows stuff where there are remote code execution vulnerabilities and other crazy stuff like that on a regular basis. Heartbleed was a bad one but that was an exception and not the rule. I patched that pretty much right away.

If anything it was irresponsible of me to ignore that agent down alarm, had there really been something odd going on with the server I would have been too late.

 

[Platypus]

It gets updated, and logging is watched quite carefully. I don't think Postfix or Dovecot has had very many major issues in the past years anyway.

It's not like Windows stuff where there are remote code execution vulnerabilities and other crazy stuff like that on a regular basis. Heartbleed was a bad one but that was an exception and not the rule. I patched that pretty much right away.

If anything it was irresponsible of me to ignore that agent down alarm, had there really been something odd going on with the server I would have been too late.

This forum needs a syduck: emoticon.

Do tell me more in detail about this 'Windows stuff' please. I am all ears.

What web server software are you running? Willing to bet that its been problem free for the last 3 years?

You can update packages all you want, your kernel does *not* get updated unless you reboot or ksplice it live, period. Your packages do but as SunnyD already mentioned, that's only a partial win in some cases. The Linux kernel has had at least 455 public CVEs posted since you last rebooted your server. While true a RCE vuln via the kernel alone is rare, there is so much attack surface in your scenario it's mind boggling. If you think that running the latest packages alone is enough to stop someone from exploiting a known, vulnerable kernel once inside you've got bigger problems than thinking anyone gives a dusty fuck about your uptime. All that means is it's an easier target to pick off. This isn't 1996.

Anyway, keep doing what you're doing, you're ensuring a lifetime of ample, high paying work for people.

 

[_Rick_]

Wasn't there a kernel exploit only a few months back, that was present in kernels way back to who knows when?

Hey, OP, can I haz shell?

 

[amdhunter]

This thread reminded me to do my Windows Updates. My uptime is like 10 minutes right now. :awe:

 

[mikeymikec]

This thread reminded me to do my Windows Updates. My uptime is like 10 minutes right now. :awe:

Ditto My Windows server's uptime is about 30 days on average, being the usual WU update cycle.

I'm hoping the next thread will be about dust accumulation inside a computer, where fellow dust enthusiasts can compare the amount, type and/or layout of debris inside their PC, because it works so well that it never gets cleaned or something.

 

[ultimatebob]

a yum update (or apt-get upgrade, etc) does put new packages, but it doesn't unload and unlink old shared libraries that are in use which leaves you with lingering security vulnerabilities. This is especially true with kernel packages.

In 3 years you probably should have rebooted at least half a dozen times, even on Linux.

The "if it ain't broke" mentality... *sigh*

Yeah, this. Kernel security updates do not take effect until you reboot the box.

Maybe Linux should be more like Windows in this regard, where they do not let you install new security updates until you've rebooted from the last batch of security updates that require a reboot to activate. It kind of gives you a false sense of security the way it works now.

 

[SunnyD]

Yeah, this. Kernel security updates do not take effect until you reboot the box.

Maybe Linux should be more like Windows in this regard, where they do not let you install new security updates until you've rebooted from the last batch of security updates that require a reboot to activate. It kind of gives you a false sense of security the way it works now.

I forget whether it's yum or apt-get (or both; I run CentOS and a Ubuntu LTS servers for various reasons), but I know that one of them explicitly gives you a warning to reboot your box if kernel or shared libraries get updated. But it won't ever automatically reboot on its own. Just gives a warning, nothing more.

Ironically, I'm not terribly proficient at *nix personally. I know just enough to get by.