• We’re currently investigating an issue related to the forum theme and styling that is impacting page layout and visual formatting. The problem has been identified, and we are actively working on a resolution. There is no impact to user data or functionality, this is strictly a front-end display issue. We’ll post an update once the fix has been deployed. Thanks for your patience while we get this sorted.

Outage

Page 2 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.
Originally posted by: Doboji
Ever feel like investing into some real data storage let me know. I work for Network Appliance...

RAID 4 Double Parity FTW

-Max

Then when you are done playing with the small stuff, you could give EMC a call 😉
 
Originally posted by: DVK916
Maybe you should have a backup system. A system that is a clone of the primary one. Where all of the data is duplicated and save, so if one goes down the other can just pop up and take its place.

that won't work in his case....he ran into data corruption, the system you described would duplicate the corrupt data as well...

Originally posted by: TGS
Originally posted by: Doboji
Ever feel like investing into some real data storage let me know. I work for Network Appliance...

RAID 4 Double Parity FTW

-Max

Then when you are done playing with the small stuff, you could give EMC a call 😉

sure...if he wants to fork over 1/2 million easy 🙂
 
Originally posted by: FreshPrince
Originally posted by: DVK916
Maybe you should have a backup system. A system that is a clone of the primary one. Where all of the data is duplicated and save, so if one goes down the other can just pop up and take its place.

that won't work in his case....he ran into data corruption, the system you described would duplicate the corrupt data as well...

Originally posted by: TGS
Originally posted by: Doboji
Ever feel like investing into some real data storage let me know. I work for Network Appliance...

RAID 4 Double Parity FTW

-Max

Then when you are done playing with the small stuff, you could give EMC a call 😉

sure...if he wants to fork over 1/2 million easy 🙂

I would suggest selling your souls to Google and use their servers. You would have your choice of server farms- East Coast, West Coast, overseas somewhere. Now the big question is, "Should I take the stock or the cash?". 🙂
 
Wow what a nightmare, millions of AT'ers came out of their homes yesterday to the shock and horror of those living around them. I quickly fled to the shelter of my computer, curled into a fetal position until the forums were revived.
 
I wanted to go out and see the real world, but I had to stick around to hit refresh every half hour to see if the forums were back up yet. You guys should get anandtechstatus.com, put the server on a different continent (so it's roughly guaranteed never to go down at the same time) and post status messages there in case of an outage. At the very least, couldn't you have put up a temporary index.html on www. to explain the situation (or was there corruption on the webserver too? Even then, a spare pc running linux or something...)?

If you think the stress of spending a day fixing data corruption is worse than the 'minor' trauma we suffered of not knowing what was going on with our beloved forums and/or having to face the real world for a day, consider that you have to take that 'minor' pain and multiply it by a few thousand because of all the users! It'd blow your head!




Alright, the above was somewhat tongue-in-cheek. It's fantastic you were able to get out of it without any data loss 🙂 Thanks for the update.
 
I liked how there was an outage and then when AT came back I saw this:

AnandTech is hiring, we're looking for an experienced .NET developer, if you're interested click here.

🙂 Made me chuckle.
 
idk, that ATstatus.com thing sounds good but AT doesn't go down too often so i'm not sure it's really needed. just my opinion

great to hear it's back up. i was refreshing every 10 minutes for 2 hours and even IMed a fellow AT member to see if he can't load the site either. i just went to do hw after that 😛. i already see the real world so saw no need to go out and explore
 
Originally posted by: DVK916
Maybe you should have a backup system. A system that is a clone of the primary one. Where all of the data is duplicated and save, so if one goes down the other can just pop up and take its place.

SO what have like RAID5+1, that would cost an Ungodly Amount, RAID 5 (depending on how much parity) is supposed to be able to withstand multiple disk failures without data loss.
 
Originally posted by: Kenazo
I liked how there was an outage and then when AT came back I saw this:

AnandTech is hiring, we're looking for an experienced .NET developer, if you're interested click here.

🙂 Made me chuckle.

AnandTech is hiring, not FuseTalk. Besides, it doesn't sound like this outage was software related. 😉
 
WOW! Sounds like a lot of work. I think I speak for all of us in saying we appreciate how fast you had everything back up!

Cheers Jason!
 
Originally posted by: FreshPrince
Originally posted by: DVK916
Maybe you should have a backup system. A system that is a clone of the primary one. Where all of the data is duplicated and save, so if one goes down the other can just pop up and take its place.

that won't work in his case....he ran into data corruption, the system you described would duplicate the corrupt data as well...

Originally posted by: TGS
Originally posted by: Doboji
Ever feel like investing into some real data storage let me know. I work for Network Appliance...

RAID 4 Double Parity FTW

-Max

Then when you are done playing with the small stuff, you could give EMC a call 😉

sure...if he wants to fork over 1/2 million easy 🙂

Depends on what device you are looking at. A small Clariion with a DPE/DAE isn't going to run a ton of money. It's all relative to the capabilities of the storage device you are looking for. If there is little "monetary" impact for the forums being down, then investing in a low-mid level storage array would be a waste. I would be much more suspicious of a host based controller card creating problems, than of these sort of problems coming from even a low-end EMC device.

I've personally seen data corruption messages on NetApp storage, though it was on unimportant data. I've only heard of a single problem in my area with data on EMC, with a customer having a bad memory module and it was causing data problems when transactions were made to a database. The customer was able to recover from a BCV, and was back in business. NetApp has similiar technology with snaps, so it typically comes down to a vendor preference or hopefully a price/performance evaluation.
 
Originally posted by: TGS
Originally posted by: FreshPrince
Originally posted by: DVK916
Maybe you should have a backup system. A system that is a clone of the primary one. Where all of the data is duplicated and save, so if one goes down the other can just pop up and take its place.

that won't work in his case....he ran into data corruption, the system you described would duplicate the corrupt data as well...

Originally posted by: TGS
Originally posted by: Doboji
Ever feel like investing into some real data storage let me know. I work for Network Appliance...

RAID 4 Double Parity FTW

-Max

Then when you are done playing with the small stuff, you could give EMC a call 😉

sure...if he wants to fork over 1/2 million easy 🙂

Depends on what device you are looking at. A small Clariion with a DPE/DAE isn't going to run a ton of money. It's all relative to the capabilities of the storage device you are looking for. If there is little "monetary" impact for the forums being down, then investing in a low-mid level storage array would be a waste. I would be much more suspicious of a host based controller card creating problems, than of these sort of problems coming from even a low-end EMC device.

I've personally seen data corruption messages on NetApp storage, though it was on unimportant data. I've only heard of a single problem in my area with data on EMC, with a customer having a bad memory module and it was causing data problems when transactions were made to a database. The customer was able to recover from a BCV, and was back in business. NetApp has similiar technology with snaps, so it typically comes down to a vendor preference or hopefully a price/performance evaluation.

You've seen data corruption messages on a Netapp? when? what was the context? Being in PS I spend all day everyday working on Netapps in a billion different environments and never have I seen data spontaneously corrupt on a Netapp box.

Do you work for EMC? I'm not going to argue that Netapp beats EMC on the high end enterprise level massive storage market. But I would argue that we're definitly the way to go for Mid-Small level implementation, and certainly the way to go for ISCSI implementations. Take AT for example... why on earth would they want to buy an expensive EMC solution, when a nice low end FAS270c would more than do the job?

All the lovely ultra redundant features of the high end Netapp boxes is duplicated on the low end... a 270c for example supports redundant heads, raid dp, all the snapshot technology you could want, and the same OS as the higher end boxes.

-Max
 
Well thats what I mean, we've had data errors pointing to NA740s the customer uses. Though I've had messages come up telling me my internet explorer cache on the network profile has been corrupted. Like I said unimportant data. I think we've had several occurances of that sort of thing, which isn't a customer impact obviously.

Though to say that EMC doesn't have products for the smaller customers isn't exactly correct. There is the NS350 to the NS500 or NS700 series devices, with the ability to go to a gateway device with Clariion, Symm, or DMX backend. Though like I said the performance of the NAS devices is roughly in the same ballpark, and I prefer a price/performance evaluation rather than picking a single vendor over another. Both of the devices offer similiar functionality, FAS270 - NS350 wise. What I do see is the EMC box scaling a bit further, but I doubt the AT DB is going to scale to 10TB anytime soon.
 
Originally posted by: TGS
Originally posted by: Doboji
Ever feel like investing into some real data storage let me know. I work for Network Appliance...

RAID 4 Double Parity FTW

-Max

Then when you are done playing with the small stuff, you could give EMC a call 😉

Just because it is EMC doesn't make it immune. We had a EMC lightpulse controler get fubar'ed and do almost exactly this same thing about a year ago, and corrupt one of 2 TB sized arrays. I had better things to do that day besides going though database transaction logs looking for some stuff. Arrg.

That being said, I still love EMC 🙂
 
Originally posted by: Evadman
Originally posted by: TGS
Originally posted by: Doboji
Ever feel like investing into some real data storage let me know. I work for Network Appliance...

RAID 4 Double Parity FTW

-Max

Then when you are done playing with the small stuff, you could give EMC a call 😉

Just because it is EMC doesn't make it immune. We had a EMC lightpulse controler get fubar'ed and do almost exactly this same thing about a year ago, and corrupt one of 2 TB sized arrays. I had better things to do that day besides going though database transaction logs looking for some stuff. Arrg.

That being said, I still love EMC 🙂


Techinically thats an Emulex HBA. Also if you have a production system using fibre chanenel attached storage you should be using at least two of them in conjunction with power path just in case one of the cards goes tango-uniform. Though in any case things that typically cause the most problems in storage are at the host level, as most storage systems have a great deal of redundancy and error checking when the data is processed.
 
Originally posted by: TGS
Originally posted by: Doboji
Ever feel like investing into some real data storage let me know. I work for Network Appliance...

RAID 4 Double Parity FTW

-Max

Then when you are done playing with the small stuff, you could give EMC a call 😉



The call HP when EMC's 30 engineers can't figure out how to install the damn thing. 😉
 
Back
Top