Man deletes entire company with one line of code

[werepossum]

Who needs malware when you're doing it yourself?

Nobody wants to lose any income. That this would be a business large enough to have enough employees that the owner could just sit back and watch isn't how that game is played.

lol True.

Yup. You just access a shared drive over a WAN link, e.g. an NFS shared volume. No reason why this can't be done over VPN.

Of course, you'd be nuts to do this with a backup. If you have the shared drive mounted read-write, then any malware on the system could toast the backup.

That's why you don't backup to a network share. You either use a different type of protocol, e.g. rsync, FTP, etc. Ideally, you would use a snapshotting file system. Trigger a snapshot, and then rsync or FTP the snapshots off to the backup server. Alternatively, you run the backup software on the backup server, and on your production machine you expose the data drive or snapshot files as a read-only share. The backup server connects to the share read-only and copies all the data off.

For the truly paranoid, the backup server can be firewalled off, except for when a backup is due. When a backup is due a cron job on the backup system temporarily lowers the firewall, initiates the backup, and when the backup is complete the firewall is brought back up. Taking this to extremes, you can use multiple backup servers which operate in rotation, but where only one has an open path through the firewall at any time.

Thanks for the info. Apparently he did mount it only during backups, but managed to screw up his script to the point of wiping everything. Obviously the dude needs to be running Windows. Are you sure you want to delete everything? Yes, I - wait, what?

 

[Red Squirrel]

I can easily see it. rm -rf / on a workstation or any machine that has lot of network mounts would pretty much delete a good chunk of the network, because it would also delete all the network shares that are mounted. Even if you were to do it as a unprivileged user... you obviously want to have write access to your network shares otherwise it defeats the purpose, so even as non root you'd lose all of that. Or doing it on a VM server would be bad too because all the luns are mounted, and obviously require write access for the VMs to work.

A while back I was reorganizing my movies on my file server, and because Linux does not have permission inheritance a lot of files had wrong permissions set as they may had been added through a different way than others so they took whatever default permissions for that process (Ex: samba, or NFS, or another machine as another user etc) so I could not just do it from another machine via NFS but had to be right on the file server as root. Basically the movies were organized by letter folders, A, B, C etc... I found this cumbersome so just wanted them all in the same folder. So I was doing mv A/* . and then hitting the up arrow, erasing A and adding B, rinse and repeat. Well I fumbled the keyboard and ended up doing mv /* . I quickly realized what happened and hit ctrl+c but by that point most of the system folders like /etc/ /usr /bin /boot had been moved. I was able to move them back by typing the full new path to the mv command but a lot of system folders like that often have to be at a specific location on the hard drive platter, especially /boot and the boot loader. To this day I still have not rebooted that box. I compared the folder permissions and attributes with other servers and I think it's ok, but still not sure if that system will ever boot again and rather not try unless I have no choice. Server is sitting at 681 days uptime.

What I should have done is reset all permissions properly as root, then still do that rename process as a more restricted user. Live and learn.

 

[flexy]

1500 shared hosting customers is like having 1500 customers on a paper route. He probably makes pennies a month off of most of them.

Yes. Possibly just some guy who got re-seller hosting or something, that he had 1500 clients doesn't necessarily mean he made big money.

There's lots of incredibly crappy hosting companies out there where they sell one-year plans for $5 or something...and I have reason to believe that guy is one of those.

 

[rh71]

I always keep backups of my websites (that live on other webhosts) locally. His business may be gone, but my website files wouldn't be... just common sense no? I wouldn't be suing for his misfortune.

 

[Red Squirrel]

I don't even get how hosting companies afford to sell hosting for that cheap. I tried to do the math once and just could not figure it out, based on the cost of leasing or collocation or even buying hardware outright. Even if you are fortunate to have an ISP that allows hosting stuff you still have to pay for the hydro costs and need tons of clients to pay off the initial cost of the hardware. Some of these providers offer like 100GB or even more too for dollars per year. As a rough estimate 1TB of redundant space is probably about $100 so if you're offering 100GB you can only put 9 clients on 1TB (remember 1TB is actually like 900GB or so) so if they're only paying $5 per year... it just makes no sense to be able to cover even just initial 1 time costs let alone ongoing costs to run the servers. You'd have to be overselling like mad and hope nobody actually uses their allocated space... but if they do, then they can sue you for not being able to provide what they're paying for.

 

[Markbnj]

If I had been on the forum when he posted this I like to think I would have been one of the nice ones, but honestly seeing "rm -rf {foo}/{bar}" made me want to rage quit life. Nobody "deserves" to have this happen to them, but some people come close.

I wonder which distro he was running because ubuntu (and perhaps debian, not sure) won't let this happen without confirmation unless you mod the policies.

 

[Sonikku]

Should probably have had a backup.

 

[Red Squirrel]

Should probably have had a backup.

Sounds like he did, but they were mounted, so they got deleted too.

Always always always have offline backups in addition to regular online backup routines.

 

[JoeBleed]

I have to wonder if his offline/offsite backups that happened to be online at the time don't have a backup of their own. Wonder if that's how he got most of it back. If any of it is true. Unless his offsite backup was at his house maintained by him, i would think any other company offing a backup service would have their own backup or at least snapshots.

That's how the one my company uses is supposed to work. i say that as i'm not the one that works with the offiste backups. I just hope they're testing it. my onsite/offline backups are fine though.

What do the few of you in this thread at actually do this, work with offiste backup companies, think?