'World of Warcraft' battles server problems

[Schadenfroh]

Text

With 6 million subscribers, each of whom pays $15 a month, Blizzard Entertainment's online game "World of Warcraft" has become a billion-dollar enterprise.

Now comes the hard part: Making sure WoW is always up and running. Some players are angered by ongoing server problems that have led the game to crash without warning while they were playing. Complaints have also surfaced about long lag times and frustrating waits to even play.

Despite Blizzard's contention that it's been keeping WoW customers informed of system problems at all times, some players contend that the company has been slow to react to complaints and reluctant to offer support when problems arise.

According to players, the problems have been especially acute since Blizzard implemented its last major patch to WoW, in late March. At that time, the company acknowledged it had some temporary server problems but said they'd resolve themselves within hours. But some players say that ever since then, they're routinely encountered "urgent maintenance" that can result in being booted from the game at any time.

"Being a system administrator myself, I have some understanding of what goes on in a corporate data center," said Evgeny Krevets, a sometimes-frustrated WoW player. "I don't know Blizzard's system setup. What I do know is that if I kept performing 'urgent maintenance' and taking the service down without warning for eight-hour periods, I would be out of a job."

Blizzard blames some of the problems--such as the disconnection, for several hours on Friday, of players linked to several servers--on AT&T, its network provider. (AT&T did not respond to a request for comment.)

It also argues that online games like WoW that have to manage hundreds of thousands, or millions, of accounts, are simply prone to network issues.

"Due to the complex nature of massively multiplayer games like 'World of Warcraft,' technical issues such as the ones some of our players have experienced recently may occur on occasion," Blizzard spokeswoman Lisa Jensen, said in an e-mail to CNET News.com. "Our commitment to our players is to provide effective solutions as quickly and carefully as possible whenever any such situation occurs."

WoW is what is known as a "sharded" online game. That means the game's many players are divided up among a large number of servers, or "shards," because no individual server could handle the full player base. This is common in the online games space.

As a result, players can usually choose which server they wish to play on, and each server can take on its own characteristics due to the specific guilds that play on them. In some games, like "EverQuest II," different servers can even have different operating rules.

Certainly, WoW is hardly the first online service to be hit by network and server problems. Over the years, services like eBay, Amazon.com and E*Trade have all dealt with various forms of outages.

And even some WoW players who are frustrated by the inconsistency of their game acknowledge that providing constant uptime is tricky, especially considering how fast the service has grown.

"I don't know how much I fault (Blizzard), since many of my own companies have had scaling problems," said Joi Ito, a venture capitalist who has put money into well-known online outfits such as Technorati, and who runs a WoW guild--or team--filled with other tech executives and well-known bloggers. "However, the uptime is really not (at an acceptable) level for a real commercial service, so I hope they get better."

Ito said the server problems have been particularly frustrating for him and his guild members because of the particular flavor of virtual "quests" they often run in WoW.

"Difficulty logging in (and) servers going down--it's become a normal part of our lives," Ito said. "It really does suck for us because we're running higher-level (quests) where it takes us a few hours to get to the (goal) and sometimes the server suddenly goes down right near the end before we finish. And they are unannounced (and) you just see people on the server--guild list--start dropping off."

Another member of Ito's guild said he too had been having problems with the WoW servers, though of a different nature.

"I have waited to get online the last couple times I have played," said Eric Haller, a San Francisco blogger and investor. "We moved (servers) because the old server was doing that, and now the brand new server is having the same issues."

Haller said he attributes the wait times--often about 10 to 15 minutes--to WoW's growth being so fast. He joked about how long he has to wait.

"When you live on Internet time, 10 minutes can seem like an eternity of delayed gratification," Haller said, "so it can be pretty frustrating."

Not all WoW players have experienced the server problems, and even some who have complained note that the issue may be slowly resolving itself.

"I decided to switch to another server over a week ago," Krevets said. "The amount of issues and problems I experienced were just too much for me. The new server that I've been playing on has not experienced any log-in problems or queues, so I've been quite happy so far."

In any case, with no shortage of massively multiplayer online games, such as "EverQuest," "City of Heroes," "Ultima Online" and others, on the market, some might wonder why angry WoW players don't just walk away.

But some say WoW has reached its 6 million subscriber threshold--no other American massively multiplayer online game has even broken a million--because its game play is easier to grasp for mainstream players. And because there are few other practical options for many such players, they feel Blizzard should take the performance problems more seriously.

"The thing is, there is no other real alternative" to WoW, Ito said. "So they sort of have a natural monopoly, and that's why people are so mad, I think. They can't vote with their feet. They just have to wait. And 'Blizz' has to realize that they have millions of hours of people's time hostage and should feel that responsibility."

 

[pontifex]

i just reinstalled wow and i'm trying to re-activate my account.
lucky if i can even get logged in to the account server. when i try to get to the page to enter CC info it always says its too busy.

maybe i shouldn't start playing again...

 

[CrackRabbit]

So when did Cptn. Obvious start writing for Cnet?

 

[erikistired]

welcome to a year ago.

 

[pontifex]

still can't get in over 2 hours later...

guess i;ll wait until tomorrow - oh wait, tomorrow is Tuesday, maintenance day - guess i'll wait until wednesday to try again.

 

[Markbnj]

Interesting. They are trying to distinguish massively multiplayer games from other networked systems in terms of complexity. The inference is that they are pioneers on the bleeding edge of distributed transactional systems.

I think they are pioneers, but they are pioneers in the game industry on the bleeding edge of scale. Is WoW more complex, in terms of network architecture and transaction volume, than the credit card, ACH, EFT, or online trading systems? I don't think so. Those systems don't go down, period. The clients may crash. The networks may pop in and out, but the databases stay up and stay online.

Blizzard's WoW network is approaching levels of utilization, and the associated revenue points, similar to those infrastructure networks. I say approaching even though the volumes are still way below even a small trading network. They will have to make the investments, just as those networks did.

Heck, buy a few of these and virtualize a few thousand servers and I bet you can get to five nines or better.

 

[Wingznut]

Originally posted by: pontifex
i just reinstalled wow and i'm trying to re-activate my account.
lucky if i can even get logged in to the account server. when i try to get to the page to enter CC info it always says its too busy.

maybe i shouldn't start playing again...

I'm in the exact same boat. *sigh*

 

[ubercaffeinated]

a wow free life makes the air sweeter and the sun brighter.

 

[Schadenfroh]

Originally posted by: makoto00
a wow free life makes the air sweeter and the sun brighter.

I was curious how long it would take the antiMMO zealots to get here, you are late.

 

[pontifex]

Originally posted by: Markbnj
Interesting. They are trying to distinguish massively multiplayer games from other networked systems in terms of complexity. The inference is that they are pioneers on the bleeding edge of distributed transactional systems.

I think they are pioneers, but they are pioneers in the game industry on the bleeding edge of scale. Is WoW more complex, in terms of network architecture and transaction volume, than the credit card, ACH, EFT, or online trading systems? I don't think so. Those systems don't go down, period. The clients may crash. The networks may pop in and out, but the databases stay up and stay online.

Blizzard's WoW network is approaching levels of utilization, and the associated revenue points, similar to those infrastructure networks. I say approaching even though the volumes are still way below even a small trading network. They will have to make the investments, just as those networks did.

Heck, buy a few of these and virtualize a few thousand servers and I bet you can get to five nines or better.

i'm not sure you can compare a video game to CC transactions. cc transactions are only a small amount of data. sure they do alot but its small amounts. wow is a virtual world and not only are they doing high volume, they are doing heavy loads.

 

[erikistired]

Originally posted by: Schadenfroh

Originally posted by: makoto00
a wow free life makes the air sweeter and the sun brighter.

I was curious how long it would take the antiMMO zealots to get here, you are late.

how is that antiMMO? that looks to be antiWoW to me. i agree, life without WoW is much sweeter. however i'm also playing auto assault. does that make me an antiMMO zealot?

 

[oynaz]

Originally posted by: pontifex
i just reinstalled wow and i'm trying to re-activate my account.
lucky if i can even get logged in to the account server. when i try to get to the page to enter CC info it always says its too busy.

maybe i shouldn't start playing again...

Re-activate your account? You do not need to do that after a re-install. Just login.

 

[skace]

WoW IS bleeding edge. I just wish the PR person would admit that "Hey, our design team is banging their heads against the walls trying to scale above and beyond 6 million people" instead of coming up with rediculous excuses.

 

[Markbnj]

i'm not sure you can compare a video game to CC transactions. cc transactions are only a small amount of data. sure they do alot but its small amounts. wow is a virtual world and not only are they doing high volume, they are doing heavy loads.

I'm not sure what your point is. In terms of load and volume banking networks and the supporting databases outstrip something like WoW by several orders of magnitude. Individual data packets may well be smaller. I'm not sure, but nor am I sure that this is a significant difference.

Most of what makes a game a game, from our player perspective, happens only on the client. That is where the graphics are stored and rendered, sounds generated, player input handled, collision detection enforced, etc.

The back-end is primarily a large online database that contains the current locations and states of all the game objects, including players and NPCs. When you do something to the game you cause messages to be sent from the client that update the database. At the same time your client is receiving a stream of world state updates from the database so that it can update your view.

So I would agree that each individual is sending/receiving more data in WoW, but I am not sure which is harder: 100 million people sending x bytes of data vs. 4 million people sending 10x data, or whatever it is.

My basic point is that at WoW's current levels of utilization Blizzard will have to make the investment in a high-availability transactional infrastructure that can handle those volumes. It can certainly be done.

 

[Genx87]

Originally posted by: pontifex

Originally posted by: Markbnj
Interesting. They are trying to distinguish massively multiplayer games from other networked systems in terms of complexity. The inference is that they are pioneers on the bleeding edge of distributed transactional systems.

I think they are pioneers, but they are pioneers in the game industry on the bleeding edge of scale. Is WoW more complex, in terms of network architecture and transaction volume, than the credit card, ACH, EFT, or online trading systems? I don't think so. Those systems don't go down, period. The clients may crash. The networks may pop in and out, but the databases stay up and stay online.

Blizzard's WoW network is approaching levels of utilization, and the associated revenue points, similar to those infrastructure networks. I say approaching even though the volumes are still way below even a small trading network. They will have to make the investments, just as those networks did.

Heck, buy a few of these and virtualize a few thousand servers and I bet you can get to five nines or better.

i'm not sure you can compare a video game to CC transactions. cc transactions are only a small amount of data. sure they do alot but its small amounts. wow is a virtual world and not only are they doing high volume, they are doing heavy loads.

On the server side what do you think it looks like? A bunch of transactions and calculations. The pretty game world is all client side.

 

[Genx87]

Originally posted by: oynaz

Originally posted by: pontifex
i just reinstalled wow and i'm trying to re-activate my account.
lucky if i can even get logged in to the account server. when i try to get to the page to enter CC info it always says its too busy.

maybe i shouldn't start playing again...

Re-activate your account? You do not need to do that after a re-install. Just login.

Yes you do, reinstalling the game only gets the client files on it, it doesnt turn on your account server side.

 

[fishbits]

Is Blizzard still doing the thing where when the game servers have problems, their web pages do too? So "If you're having problems with the game, check our web site for details" except that the web site is hosed as well.

a wow free life makes the air sweeter and the sun brighter.

Feel exactly the same way, and I've played other MMOs before and after. From the sounds of things, server issues have only gotten worse even as revenues have increased. Among other things, couldn't stand the thought of that many of us paying Blizzard so much money, and in return WoW underperforming other MMOs who are able to make things work much better with much less funding. Meanwhile the fanboys who never played any other MMO made every excuse under the Sun as to why this not only was acceptable, but that we should be kissing Blizzard's feet for the level of service they decided to bless us with.

I'm still guessing that WoW will see a decline in usage (player-hours or total players or both) in the near-ish future. Maybe when that happens, the hardware and infrastructure set up will be adequate. If however growth continues, Blizzard really doesn't seem interested in upgrading to keep pace with it. I mean heck, the customers will pay no matter what, so why not keep more profit instead of reinvesting it to to the customers' benefit?

 

[pontifex]

Originally posted by: oynaz

Originally posted by: pontifex
i just reinstalled wow and i'm trying to re-activate my account.
lucky if i can even get logged in to the account server. when i try to get to the page to enter CC info it always says its too busy.

maybe i shouldn't start playing again...

Re-activate your account? You do not need to do that after a re-install. Just login.

i canceled my account, i didn't just stop playing and then keep paying for it.

 

[pontifex]

Originally posted by: Genx87

Originally posted by: pontifex

Originally posted by: Markbnj
Interesting. They are trying to distinguish massively multiplayer games from other networked systems in terms of complexity. The inference is that they are pioneers on the bleeding edge of distributed transactional systems.

I think they are pioneers, but they are pioneers in the game industry on the bleeding edge of scale. Is WoW more complex, in terms of network architecture and transaction volume, than the credit card, ACH, EFT, or online trading systems? I don't think so. Those systems don't go down, period. The clients may crash. The networks may pop in and out, but the databases stay up and stay online.

Blizzard's WoW network is approaching levels of utilization, and the associated revenue points, similar to those infrastructure networks. I say approaching even though the volumes are still way below even a small trading network. They will have to make the investments, just as those networks did.

Heck, buy a few of these and virtualize a few thousand servers and I bet you can get to five nines or better.

i'm not sure you can compare a video game to CC transactions. cc transactions are only a small amount of data. sure they do alot but its small amounts. wow is a virtual world and not only are they doing high volume, they are doing heavy loads.

On the server side what do you think it looks like? A bunch of transactions and calculations. The pretty game world is all client side.

but there's a lot more info to keep track of and it is constant.

 

[Markbnj]

but there's a lot more info to keep track of and it is constant.

I'm not sure what your last point means. What's constant? If your first point is that the WoW database application has more complicated data, and more variable states than a banking system, I think you're right. I'm not sure that holds true for a trading system, however it would take an in-depth analysis to say for sure. In any case, I think the bottom line is still that Blizzard has scaled up from their entrepreneurial infrastructure (the game's launch architecture), and hasn't yet made the investments that multi-billion dollar systems make to achieve high-availability.

 

[NYHoustonman]

I cancelled, this time for good I think, about 2 weeks ago. Oblivion has taken up a lot of my time, as has school work, so I wasn't playing much anymore. What pushed me over the edge was that, each and every time I tried to log on (random times) to talk to people (I was the leader of my guild), I'd have connection problems. I simply don't have the patience for that anymore. Every server my friends and I tried (we were on our third, having gotten both previous characters to 60) had all kinds of issues after a while, especially the second (Moonrunner), where saturday through monday essentially became 'maintenance' time. 6 times that I remember, I was close to end bosses in UBRS/Strat/Scholo/the like, and the server decided to either crash or bring up the dreaded server restart message. You essentially couldn't do anything overly time consuming on weekends, because it wasn't a matter of if the server would crash, but when.

Yea, good riddance WoW. I'm happy without it.

 

[CKent]

Originally posted by: CrackRabbit
So when did Cptn. Obvious start writing for Cnet?

:laugh::laugh:

It's actually gotten quite a bit better lately, but indeed last night wasn't a good one for the login server.

 

[pontifex]

i'm not sure if i even want to start playing again. i tried a few months back and played for like an hour and then canceled. apparently if you reactivate and then cancel within a short time, you don't get charged.

i'm just so bored lately and there aren't any good games out anymore.

 

[DirthNader]

I quit WoW a couple of months ago, primarily because there was nothing left for me to do but raid (bleh) or roll alts.

The poor service certainly didn't entice me to stay. Huge lag and unplanned restarts are nothing new, and they were happening well before WoW hit 6 million subs. I didn't appreciate my small server (Azjol-Nerub) having not one, but two server transfers onto it either - coupled with Christmas sales, my last months in WoW were full of hour long queues on the weekdays, and queues long enough on the weekends that if you weren't on by 8PM EST, you may as well forget it.

 

[Fingolfin269]

I can't count the number of times I've quit this game. I'm playing again on a new PvP server but we'll see how long it lasts.

The funny thing about this is that people constantly talk about how 'if I performed like that in my job I would be fired!'. Yup, you would be fired because you could give the company a bad name or cause your company to lose money. Guess what people? Blizzard isn't hurting their name and they sure aren't suffering in the financials.