Question on Opteron vs Athlon 64

[imported_Toth]

I have been looking at processor options lately and I was a bit confused on the difference between a desktop platform like the Athlon 64 vs the server platform of an Opteron.

Besides the fact that Opterons have more hypertransport links enabled to allow a MP setup and their use of registered memory, what are the major differences? If I am looking at setting up a small cluster that is purely computational(aka not a webserver or such) of say 100 machines, why would I need to pay the price premium of the Opterons, when I can get Athlon x2 64s at a much more affordable cost that already act like dual processor setups? Does this change when you say had 1000 machines?

I just want to be educated on the matter, so if anyone has any links that would be great.

 

[uOpt]

Well, the registered RAM is a big deal. First of all, with the exception of most Asus boards you have difficulties even getting ECC in socket 939 boards. Registered RAM is of course a no-go for 939 but many server installations want it because it allows you to have much more RAM. In 939 you usually run into trouble even reaching 4 GB with remapping and DDR400 and whatnot is required.

Then, it is the question of chipsets. People like me don't think very high of NVidia chipsets, and if you don't use Windoze you'll probably neither. For socket 940 you can get a board with the rock-solid if a little underfeatures AMD chipset. For 939 you have to mess with Via. If you want 939, ECC and dual-core there is a single mainboard available, the Asus A8V (but not the A8V-E).

Also, if you want PCI-X you need socket 940, there are no 939 boards with PCI-X that I am aware of.

 

[Markfw]

Welcome to the forums !

Since I own both, and have researched a lot, I can tell you that for your purposes, there is no difference. That said, below are the +/- for each IMO

Opteron:
can have 2-8 cpu's for 4-16 cores on current motherboard (+)
require registered member (+ is stability and more memory, - is cost)
Can have PCI-X (+ for SCSI cards or other cards requiring a lot of bandwidth, 8x the speed of PCI I think)
Cost (-, motherboards and cpu's cost more)

Athlon64:
Can overclock ( BIG +)
Cheaper (BIG +)
only on per motherboard (small -, but X2 gives 2 cores)

And I am sure I forgot some

 

[coomar]

the single core opterons are moving to 939

 

[uOpt]

Originally posted by: coomar
the single core opterons are moving to 939

Not the single cores. The single CPUs, both single-and dual-core.

These will be equivalent to Athlon 64s for the aspects discussed here. It is not even clear these have the additional hyperlinks.

BTW, registered ECC memory is not much more expensive than unregistered ECC. So if you want ECC the 940 solution doesn't come at that much a higher price.

Another aspect of dual-opteron versus dual-core 939 is that you can now get the 2.8 GHz single-core 254 opterons, whereas dual-cores x2s max out at 2.4 GHz and dual-core Opterons at 2.2 GHz.

 

[Gamingphreek]

Opteron has a number of core enhancements geared towards Workstation and Server apps as well.

A number of redundant stability features.

Everything listed above.

As for PCI-X support that is all on the motherboards. If you need PCI-X you would have to look for something that has an AMD southbridge. While most big businesses wouldn't use Nvidia for corporate servers, the Nforce Professional with an AMD southbridge with a PCI-X tunnel is arguably the best choice.

-Kevin

 

[Furen]

Originally posted by: Toth
I have been looking at processor options lately and I was a bit confused on the difference between a desktop platform like the Athlon 64 vs the server platform of an Opteron.

Besides the fact that Opterons have more hypertransport links enabled to allow a MP setup and their use of registered memory, what are the major differences? If I am looking at setting up a small cluster that is purely computational(aka not a webserver or such) of say 100 machines, why would I need to pay the price premium of the Opterons, when I can get Athlon x2 64s at a much more affordable cost that already act like dual processor setups? Does this change when you say had 1000 machines?

I just want to be educated on the matter, so if anyone has any links that would be great.

If you want that many systems I'm sure you want SOME kind of support. Even if you plan on buying the systems without any enhanced tech support packages, I'd talk to Sun (over the phone) about their socket 939 opteron workstations. Here are the "official" prices on the dual-core 100 series:

Dual-Core AMD Opteron 175 $530
Dual-Core AMD Opteron 170 $475
Dual-Core AMD Opteron 165 $417

The 175 is actually cheaper than the 4400+ ($537), so your overall price would probably not be that much higher than going with the X2 and solaris is quite nice (in my opinion, anyway). ECC ram might be a bit expensive, though. Just be sure to get some crappy video on your systems, heh.

EDIT - Having to go opteron really depends on what your memory bandwidth needs will be. If your computations are relatively-low-bandwidth (like if you wanted to set up a F@H farm, for some reason) then you could get away with going X2 and just networking your cluster together. If your computations are very memory bandwidth intensive then opteron multi-cpu (not necessarily multi-core) is your best bet. The same applies if you need high-speed/low-latency interconnects like Infiniband+Infinipath.

 

[uOpt]

Originally posted by: Furen

Originally posted by: Toth

EDIT - Having to go opteron really depends on what your memory bandwidth needs will be. If your computations are relatively-low-bandwidth (like if you wanted to set up a F@H farm, for some reason) then you could get away with going X2 and just networking your cluster together. If your computations are very memory bandwidth intensive then opteron multi-cpu (not necessarily multi-core) is your best bet. The same applies if you need high-speed/low-latency interconnects like Infiniband+Infinipath.

I'm sorry, but that is nonsense.

The 939 X2 actually have a little more memory speed because the registered RAM takes a clockcycle more per access.

NUMA would be a factor but so far benchmarks have not shown it to have a big effect.

 

[coomar]

but the opterons have seperate hypertransport channels

 

[Furen]

Originally posted by: MartinCracauer

I'm sorry, but that is nonsense.

The 939 X2 actually have a little more memory speed because the registered RAM takes a clockcycle more per access.

NUMA would be a factor but so far benchmarks have not shown it to have a big effect.

Here's a benchmark that shows a bit of a benefit from NUMA:

Link

Just because you havent read anything that shows NUMA being beneficial to performance, it doesnt mean that it's worthless. Like I said, if he knows his usage will leave him memory-bandwidth-bound (which I cant tell without knowing exactly what he plans on doing) then NUMA will probably be helpful. It truly does depend on what he plans on doing with his compute cluster. Most applications are NOT memory-bandwidth limited on A64s because of their extremely efficient memory controller but you have to be aware that even though an X2 will have more memory bandwidth than a single opteron dual core, having two single-core Opterons will give each one two memory channels, instead of having to share a single dual-chan mem controller between two cores.

 

[uOpt]

Yes, it depends.

Opterons have more hypertransport channels and NUMA, but socket 939 has unregistered RAM. If you are willing to live without ECC you can buy much lower latency RAM for socket 939.

It would be interesting to benchmark but I bet that in the big picture neither matters, and if you want to look at details the 939 probably comes out a bit faster for most applications.

That doesn't mean I recommend socket 939 and X2, far from it. I'm just saying that memory performance, as long as you can get enough memory into your board at full speed, will not be noticably better on a socket 940 system.

 

[imported_Toth]

Thanks for the replies guys.

Well the major issue for us is price per box, because we are mainly CPU limited as it stands. Thats not to say that memory performance doesn't matter, but unless going with a X2 64 was crippling in comparison to a dual opteron setup then I don't think its as big of an issue. The actual latency is probably more important than the bandwidth limits... but that will need more testing.

Even though the cluster would be operating 24/7, the idea of NO downtime is not a huge issue for us. If a machine crashes we can survive as long as it can easily be brought back up and isn't dead dead... So unless Opterons intrinsicaly have a longer life span then I don't see stability features as being a must.

For us the major issue is how many processor we can buy in as few a boxes as possible. As it stands the Athlon x2 64's seem energy efficient, fast, and effective as a dual processor setup. However, almost all vendors exclusively sell dual opteron setups. If you talk to them about Athlon 64s they give you a blank stare, and I am just trying to understand why.

P.S. What are some good vendors for Athlon x2 setups in 1u/2u configurations? I have some at the moment but I could use some more leads. The reason I ask is I also don't know which ones are trustworthy in their ability to deliver.

Thanks!

 

[Furen]

I dont think anyone is selling 1U socket 939 systems other than SUN, and they're charging a pretty penny for them (~$2500 per Opteron 175 server). If you are moving enough volume (you did say 100 servers...) then you can probably push them hard to drop your prices quite a bit (especially if you are migrating from Dell, HP or IBM), as long as they make a decent profit on them. I, personally wouldnt pay much over $1500 (if that) per Sun X2100 server with a 175. If you buy the rack and software from them they'll be more amenable as well (though they'll rail you with that). So long as you have around $200k to burn then you should have no problem getting your ~100 dual-core servers.

Your other option would be to get the systems custom built but since Socket 939 Opterons are as rare as pink elephants (and, thus, the 1U motherboards for them) you probably wont get anywhere.

EDIT: Now, to get as many CPUs into as few boxes as possible you'd have to go for a 1U Dual Dual-core Opteron server. Monarch Computer could help you with that.

Also, there probably wont be any need for DVD roms. I'm guessing raid would be useless as well, since you'd probably have a separate storage server so I wouldn't take more than one harddrive per system. Hell, no need for harddrives at all if you network boot them, heh. More than 1GB of ram may be overkill depending on your application.

 

[imported_Toth]

Thanks Furen... but I was talking about 939 Athlon 64 x2s, not opterton dual cores. Sorry if I mislead you. However, I think that means I am even more hosed in trying to find a vendor.... any suggestions? Also I am talking more in the realm of 1000 machines...

You are right about no need for DVD roms, but I think we will want some local storage(eg 1 80GB hd). All in all they would be very stripped down machines. 1 dual core a64 x2(4200+), 1GB ram for the two cores, 1 80 gb HD. Since they are running in a cluster I don't even need a video card of any sort right? Everything should be able to be done through a network interface correct?

 

[Furen]

Yeah, I know you wanted to get X2s, BUT in order to do so you'd need to find a socket 939 mobo for a 1U server. This, of course, led me to get into Sun, once more, since they're the only people that I could find selling socket 939 1U servers (which are socket 939 Opterons).

 

[Remedy]

Originally posted by: Furen
Yeah, I know you wanted to get X2s, BUT in order to do so you'd need to find a socket 939 mobo for a 1U server. This, of course, led me to get into Sun, once more, since they're the only people that I could find selling socket 939 1U servers (which are socket 939 Opterons).

Tyan Tomcat K8E 1U X2/Opteron board. Now, supposivly, Tyan will be making Transport barebone 1U servers available. When? I have no idea. But, they have the boards done. I guess it's a matter of them building the barebone.

 

[Furen]

That's no a 1u motherboard. It could fit into a 1U case but the expansion slots would be useless.

 

[Hyperlite]

Originally posted by: Furen
That's no a 1u motherboard. It could fit into a 1U case but the expansion slots would be useless.

its just plain ATX, but apparently a pretty nice one at that...

anyway...

 

[Remedy]

Originally posted by: Furen
That's no a 1u motherboard. It could fit into a 1U case but the expansion slots would be useless.

Agreed.

Well. The moniker for this board isn't 1U. But, with the onboard RAID and onboard Rage video chip and Dual Lan adapters. I don't see what someone would need to use the 32bit PCI slots for(or x16 slot) in a 1U rack. But, they do make risers for each of those slots. So, useless or not is moot.

Tyan claims the Tomcat K8S as being RR (rackready). Yet, it isn't any different from the K8E. (Whic h isn't claimed)

Just trying to help.

 

[Furen]

Yeah, but with the volume he's looking into (1000 servers), anything we offer will probably not be sufficient for him. Certainly not self-builds, heh.

 

[Remedy]

Oh ok. Scratch that. I guess Sun's Socket 939 is the way to go.

 

[imported_Toth]

Arn't there other alternatives to a place like Sun? Their price markup is pretty high just to begin with... I realize with a bulk buy it would come down but I would be surprised if it would be enough. Arn't there more places like this : http://www.swt.com/ that would be able to accomodate my order? I just don't know where to look...

Thanks for all the suggestions so far!

 

[Furen]

There are a few issues with your requirements that limit which OEMs can provide your systems. The first is the fact that you are asking for single dual-core CPUs. As far as I know only Sun is selling these in a 1u form factor. Further complicating this same issue is the fact that there are no 1u socket 939 motherboards out there (that I know of). The second issue with your order is its volume. 100 servers is kind of hard to fulfill for a lot of smaller OEMs, 1000 is close to impossible.

You could try giving someone, like Monarch (they seem to have pretty good ties with AMD), a call to ask them if there's any way for them to hunt down the parts to make you a server that fits your need. If you're buying from an integrator such as this I'd try to get Opteron 175s over X2 4400s simply because they're the same price but AMD "validates" the opteron for server use (whatever this validation is). If one of these OEMs offers to charge you around $300-$400 for labor I think that's a pretty good deal. This whole process would take a long time, though, since they'd have secure the parts, create a test setup first and troubleshoot any possible issues and finally start "mass-production."

Sun, on the other hand, already has the parts (or can secure them rather easily... then again, 1000 pieces of a just-released setup could be slightly tricky if you require immediate deployment), has already tested the system extensively and (an important point, in my opinion) can offer you an IPMI service processor and preconfigured management nodes and software (if you buy the whole rack). Like I said, you can try giving them (and other OEMs) a call in order to get quotes, estimated deployment dates, etc. A smaller OEM would sell you 1000 individual servers, a large OEM can sell you a preconfigured solution. Dont be scared by Sun's $2300 price per server. You can drop that to around $2000 just by getting rid of 1 gig of ram and the DVD rom, they would not mind custom-building for a large order. And that's without getting a volume discount, which you undoubtedly can get, especially if they "find out" that you're interested in HP/Dell servers and have been talking to them as well. The biggest thing when dealing with ANY OEM is that you cant let their salesforce convince you that they're doing you any favors. You are the one purchasing so dont let them pressure you into anything, they're not doing you any favors--they want your cash. hell, you can ask them to send you a server to test out if you want to, to see if it fits your needs/expectations.

Hopefully you can get a 20-30% discount on your stuff from whomever you buy from, just 'cause of the volume.

EDIT: Sun's prices on these new servers are actually pretty low (or rather, decent), considering that Dell sells a Pentium D 1u rack server for around $3k.

 

[imported_Toth]

Furen I apperciate what you are saying, but looking at this from the standpoint of parts, I could theoretically go build a 1u box with the Tyan 2865 board that was mentioned above(technically a desktop), with a 4200+ 64 x2, 1 gig ram, and an 80gb hard drive for ~ $1130. Now granted I fully understand that is for only one box and anyone I go to will charge for labor and integration/testing, but that is also me going out to retailers directly. Therefore techinically anyone providing such servers would be able to pass a lower cost per part on to me, making $1130 a roughly accurate per box price.

So yes I could get a discount on the Sun servers(btw their default config at $2300 has no DVD drive) by dumping a 1 gig of ram and some sort of %20 bulk discount, but that would still be on the order of $1500 per box or perhaps more depending on the discount. So while I do see the advantages of buying from sun, right now the number of boxes we can afford is paramount.

Now, I also understand that what I said in the first paragraph above is bullshot because I have only found 1, maybe 2 vendors that claim they can provide that many servers with that specification.

So to relax things... what would it be like if I said I wanted my cofiguration to be 2u. Everything else would be the same part wise, but just in a 2u case. Persumably the MB choice would be less of an issue as well as cooling(type of cpu heatsink... etc). So ... now what are my options then? Who could provide such a configuration outside of a Sun or a Dell? I would still be looking at roughly 1000 machines...

 

[imported_Toth]

Bump anyone? What choices would I have if I went 2u?

Thanks