PrimeGrid Race: Using Amazon HPC Cluster?

[biodoc]

Apparently a member of the winning team in the recent PrimeGrid race rented computer resources on the Amazon High Performance Computing cluster. There's a debate on the PrimeGrid forum on whether this is "fair" or not. A couple of the leaders from the 2nd place team are crying foul play.

http://www.primegrid.com/forum_thread.php?id=3556

Here's a link to the users computers. Notice the Operating System column:

Linux
2.6.35.11-83.9.amzn1.x86_64

http://www.primegrid.com/hosts_user.php?userid=77656

Anyway, I looked into some of the details of "renting" computer power on Amazon

links:
http://aws.amazon.com/ec2/pricing/
http://aws.amazon.com/hpc-applications/

Cluster Compute Instances:

Quadruple Extra Large (linux): $1.60 per hour

The Cluster Compute instance family currently contains a single instance type, the Cluster Compute Quadruple Extra Large with the following specifications:

23 GB of memory
33.5 EC2 Compute Units (2 x Intel Xeon X5570, quad-core “Nehalem” architecture)
1690 GB of instance storage
64-bit platform
I/O Performance: Very High (10 Gigabit Ethernet)
API name: cc1.4xlarge

The spirit of Distributed Computing/BOINC suggests that you either own the computers you use or get permission from the owner.

Any thoughts on whether using rented computers is fair or not? I guess it's okay with me since you are paying for the computer time. I suspect more people will use this option in upcoming races. :hmm:

 

[somethingsketchy]

I'd say the act of "renting" Amazon HPC instances is stretching the spirit of the competition, however the guy(s), whom rented the instances, did PAY for the services, it's not like they got the instances for free.

As long as they did pay for the services, I'd say it would be okay. It would be unfair and BS, yes, but it could be within the boundaries of the competition rules.

If I counted correctly, I spotted 49 instances of the 2.6.35.11-83.9.amzn1.x86_64 units. Given the $1.60/hr per instance, I'd say this couldn't have been cheap. So in a way, the dude(s) that pulled this off, just about paid for a new desktop build, in the hourly fees (48 hours * $1.60/hr) * 49 instances = $3763.20 for 49 instances for 48 hours.

It's BS for sure but perhaps this is a valid tactical which, unfortunately, will under-score the efforts of those whom legitimately own their own rigs to compete in these challenges.

 

[biodoc]

I think you get 2 of those Xeons for $1.60/hr, although it's odd there is 49 so maybe I'm wrong.

2 x Intel Xeon X5570, quad-core “Nehalem” architecture (8 threads)

Based on his total points for single X5570, I believe he rented for about 12-15 hours.

12 hours * $1.60/hr) * 24 instances (48 nodes) = $480.80

He claims his bill will be about $200 so maybe he will be surprised when the he gets the bill!

 

[VirtualLarry]

will the DC app run unmodified on Amazon's cluster, or did he have to hack the app to get it to work on their system? Seems like a lot of work to me.

 

[RobertE]

For me it's really pushing the spirit of the challenges.

However, it's really no different than someone borg'ing all the computers at their office/school/etc.

 

[somethingsketchy]

will the DC app run unmodified on Amazon's cluster, or did he have to hack the app to get it to work on their system? Seems like a lot of work to me.

IIRC, the *nix OS for the Amazon HPC instances is a regular *nix environment, so any *nix client (e.g. Ubuntu) would work just as normal (albeit with a lot more processor cores and RAM).

I figured it was "plug and play" with the BOINC software to the HPC instance

 

[Ken g6]

The spirit of Distributed Computing/BOINC suggests that you either own the computers you use or get permission from the owner.

It's not just a good idea - it's the law!

Any thoughts on whether using rented computers is fair or not? I guess it's okay with me since you are paying for the computer time. I suspect more people will use this option in upcoming races. :hmm:

Renting computers is paying for permission from the owner. It's fine by me, if quite expensive.

 

[waffleironhead]

I'm fine with it. Whether buying or renting processing time, the user is still paying to play. It does throw a wrench into things, but there are plenty of people with more disposable income than me. My lowly quad isnt competing with most of the users anyway.


After reading the amazon compute FAQ, it looks like the user is going to get a sizable bill for the resources he used, unless he used to spot instance pricing. I could be wrong though, its just not clear how many hours he used.

 

[somethingsketchy]

It's probably safe to say he had at least 40 hours (8 hours of leeway between server errors and WU clean-up before end of competition), which at any rate would be expensive, then again I noticed s/he averaged 6,000 points per instance, so perhaps 40+ hours is more realistic.

 

[waffleironhead]

It's probably safe to say he had at least 40 hours (8 hours of leeway between server errors and WU clean-up before end of competition), which at any rate would be expensive, then again I noticed s/he averaged 6,000 points per instance, so perhaps 40+ hours is more realistic.

Then I'm going to have to agree with your earlier post, but the price per hour could have been lower if using the spot instance pricing.

Maybe he will post a screencap of the bill later.

 

[Rudy Toody]

I will be hanging a For Rent sign on the Rudy Toody Cloud for the next race. Stay tuned for details.

 

[somethingsketchy]

I will be hanging a For Rent sign on the Rudy Toody Cloud for the next race. Stay tuned for details.

Unfortunately I can't get away with that business model...no one would want to rent compute time from Sketchy Computations, LLC

 

[biodoc]

:biggrin::biggrin:

 

[PCTC2]

Unfortunately I can't get away with that business model...no one would want to rent compute time from Sketchy Computations, LLC

I don't know... if you're rates were good enough maybe I would. .

How about Polaris Computer Technology Corporation Cloud? . I think if I rented out my systems I would go over my power allotment in my colocation. XD

 

[Rudy Toody]

I have decided to change the name from the Rudy Toody Cloud to the Rudy Toody Meringue since it is powered by lemons.

 

[zzuupp]

I don't like it. However, there's nothing inherently wrong with that situation. It's that user's/team's $$$, and their choice.

 

[sangyup81]

Since it benefits PrimeGrid, I don't think they should stop it but they should categorize the rankings so that there is an overall ranking and a ranking for those who don't use such services.