Poem@Home GPU

[biodoc]

Alyx is putting up some serious numbers! :thumbsup:

 

[Alyx]

Rawr! Off to the races.

 

[blckgrffn]

Rawr! Off to the races.

Very !

 

[Sunny129]

would someone be so kind as to cut and paste his or her entire app_info.xml for a fellow cruncher? i'd copy an app_info.xml from another project and just replace the pertinent info, but i'm not 100% certain that the POEM@Home app_info.xml doesn't use parameters that other projects don't use...

TIA,
Eric


*EDIT* - apparently i'm blind and didn't notice the app_info.xml in the very first post :whiste:

 

[biodoc]

would someone be so kind as to cut and paste his or her entire app_info.xml for a fellow cruncher? i'd copy an app_info.xml from another project and just replace the pertinent info, but i'm not 100% certain that the POEM@Home app_info.xml doesn't use parameters that other projects don't use...

TIA,
Eric


*EDIT* - apparently i'm blind and didn't notice the app_info.xml in the very first post :whiste:

Welcome to POEM@Home, Sunny129! :thumbsup:

 

[Sunny129]

Welcome to POEM@Home, Sunny129! :thumbsup:

thanks! i do want to say that i had 2 motivations for signing up. first and foremost i was looking for a 2nd Milkyway@Home backup project, just in case my 1st backup project ever happens to be down at the same time as MW@H. and the other reason for signing up for POEM@Home is b/c i've traditionally only engaged in the physics and astronomy projects, and thought it was high time to add a humanity-benefiting project to the mix, even if its only a backup project for now...

a quick experiment in which i played with various <count> parameter values in the app_info.xml revealed that the GPU utilization of my shader-unlocked HD 6950 2GB card was 34%, 56%, 70%, 78%, and 84% when running 1, 2, 3, 4, and 5 simultaneous tasks respectively...though it should be noted that i was also running at the time a double-threaded Test4Theory@Home task (consuming just under 2 CPU cores), and 4 simultaneous Einstein@Home BRP4 ATI GPU tasks on the HD 5870 2GB (consuming 0.5 CPU cores each). this is on a 6-core CPU. it should also be noted that i did not have the time to do a meaningful experiment documenting several tasks and their run times in each configuration, so i can't really comment at this point in time on whether or not there is an increase in crunching efficiency (or more appropriately, a decrease in its inverse - run time per task) to be had by running multiple tasks simultaneously.

i do hope to follow up with such an when i have the time. i suppose it would mostly be for my own benefit, since even though there are folks out there in DC land w/ hardware configurations that are very similar or perhaps identical to my own, the chances that any one of them are running the same or a similar combination of CPU and GPU tasks would be slim to none. that being said, i'm sure that even those folks with dissimilar hardware configurations and/or project/application choices might find the data useful in some way or another. and so when i get around to it, i will document it thoroughly and post the results in spreadsheet form...

 

[Alyx]

I'm starting to get tired of not being able to use the CPU cores for other work. I might swap projects once I hit 10M so I can run WCG on the CPUs.

 

[Sunny129]

Alyx, a few questions:

1) are you not running any CPU tasks on the machine(s) that are running POEM@Home on the GPUs? or are you able to run at least a few CPU tasks, but not as much as you'd like?
2) how many POEM tasks are you running simultaneously on each of your 6950's? and how long do they take to complete?

TIA,
Eric

 

[Alyx]

Alyx, a few questions:

1) are you not running any CPU tasks on the machine(s) that are running POEM@Home on the GPUs? or are you able to run at least a few CPU tasks, but not as much as you'd like?
2) how many POEM tasks are you running simultaneously on each of your 6950's? and how long do they take to complete?

TIA,
Eric

On my 2 6950s (in different hosts), I'm running 6 POEM tasks and 2 Neurona tasks on i7 cpus. The POEM tasks are pretty inefficient so they need a whole processor core to feed them, without and you can really see the GPU usage lower a lot. With 6 POEM tasks, all 6 take about 50min so I'm doing 6 tasks an hour.

 

[blckgrffn]

FWIW, I am crunching on 1045t 4 Poem tasks concurrently, each taking an hour and forty minutes along with 3 CPU tasks. The last (fourth) Poem task was only worth ~10k ppd even with a dedicated CPU so I revoked it. I think the GPU (4770) is tapped out.

 

[Sunny129]

thanks for the info...i'm contemplating making POEM a full-time project instead of just using it as a backup to Milkyway@Home.

at any rate, that explains alot. i'm currently running 5 simultaneous POEM tasks on an HD 5870, and they're taking 80-90 min. to complete. but i'm not leaving a free CPU core for each POEM GPU task - rather i'm running 2 Einstein CPU tasks and a double-threaded Test4Theory@Home CPU task (on a 6-core 1090T). so i guess that explains significant difference in our run times...though i expect my run times to decrease in a few days when i deplete Einstein@Home CPU task queue. i'll continue to run Test4Theory@Home (which will consume 2 CPU cores), and leave the remaining 4 cores free to service POEM@Home ATI tasks...

 

[Sunny129]

My cc_config file has extra settings because I normally have 2 different gpu projects running at once but in the last week I have both gpus running poem.

<cc_config>
  <options>
     <allow_multiple_clients>1</allow_multiple_clients>
     <use_all_gpus>1</use_all_gpus>     
     <report_results_immediately>1</report_results_immediately>
  </options>
</cc_config>

The best version of the amd drivers that have worked for me is 12.1. It cut my cpu usage by almost half. Before using 12.1 each wu would use a full core, but with 12.1 it averages 60% of a core.

salvorhardin, could you lease elaborate on why you need the <allow_multiple_clients>1</allow_multiple_clients> parameter? i'm running Milkyway@Home on an HD 6950 and POEM@Home on an HD 5870 in the same box, and both projects are crunching away problem-free without that parameter in my POEM app_info.xml...nor does it require the <use_all_gpus>1</use_all_gpus> parameter to recognize a 2nd GPU. i'm not understanding why you need them.

On my 2 6950s (in different hosts), I'm running 6 POEM tasks and 2 Neurona tasks on i7 cpus. The POEM tasks are pretty inefficient so they need a whole processor core to feed them, without and you can really see the GPU usage lower a lot. With 6 POEM tasks, all 6 take about 50min so I'm doing 6 tasks an hour.

Alyx - is simply limiting CPU task usage to 2 cores your only method of making 6 full cores available to the 6 POEM tasks? or have you also played with the <avg_ncpus>0.25</avg_ncpus> & <max_ncpus>1</max_ncpus> parameters in your app_info.xml at all?...for instance, maybe by changing one or both of them to a value of 1?

 

[Alyx]

Alyx - is simply limiting CPU task usage to 2 cores your only method of making 6 full cores available to the 6 POEM tasks? or have you also played with the <avg_ncpus>0.25</avg_ncpus> & <max_ncpus>1</max_ncpus> parameters in your app_info.xml at all?...for instance, maybe by changing one or both of them to a value of 1?

I edit my xml file too. Err and it looks like I remembered wrong, i only had 5 tasks?? Later I'll check my other host maybe it had 6 and I edited this machine wrong... But anyways.

<app_info>

<app>
<name>poemcl</name>
<user_friendly_name>POEM++ OpenCL</user_friendly_name>
</app>

<file_info>
<name>poemcl_0.1_windows_intelx86__opencl_ati_100</name>
<executable/>
</file_info>

<app_version>
<app_name>poemcl</app_name>
<version_num>1</version_num>
<plan_class>opencl_ati_100</plan_class>
<avg_ncpus>1.0</avg_ncpus>
<max_ncpus>1.0</max_ncpus>
<flops>4.1e10</flops>
<coproc>
<type>ATI</type>
<count>0.2</count>
</coproc>
<cmdline></cmdline>
<file_ref>
<file_name>poemcl_0.1_windows_intelx86__opencl_ati_100</file_name>
<main_program/>
</file_ref>
</app_version>

</app_info>

 

[Sunny129]

ok, so if you'll recall from one of my previous posts, i was running 5 simultaneous POEM tasks on the HD 5870, but i wasn't leaving a CPU core free for each POEM task running. instead, i was running them along side 4 CPU tasks (2 Einstein@Home CPU tasks and a a double-threaded Test4Theory@Home CPU task) and 2 simultaneous Milkyway@Home tasks on the other GPU in the machine, an HD 6950. with those 4 CPU cores preoccupied, i was essentially leaving only 2 cores free for 5 POEM GPU tasks (i have a hex-core 1090T CPU). to be honest, CPU task run times didn't suffer much at all in that situation, and also POEM GPU task run times didn't seem unreasonably slow for my level of hardware. but i wanted to see how much i could improve efficiency, so i finished off my cache of Einstein@Home CPU tasks and configured the machine for testing. the test configuration included the Test4Theory@Home double-threaded CPU task and the 2 simultaneous Milkyway@Home tasks on the HD 6950 GPU as constants b/c they always have and always will be running 24/7 (so fair warning - the test is really based around my project and application selections, and does not isolate the POEM GPU tasks for pure baseline purposes). the variable in the test is of course the number of simultaneous POEM GPU tasks. so without further ado, here is the processed data:

there are a few things of interest to note:

1) with the Test4Theory@Home double-threaded CPU task consuming just under 2 full CPU cores, and the 2 Milkyway@Home GPU tasks consuming a negligible amount of CPU resources (approx. 0.75% of a CPU core each), there were essentially just over 4 full CPU cores available to POEM GPU tasks throughout the test. nevertheless, i did push beyond 4 simultaneous POEM tasks to see how much the CPU core deficiency would affect GPU task efficiency and run times.

2) setting the <avg_ncpus> and <max_ncpus> parameter values in the app_info.xml both to 1 hardly did anything for efficiency, CPU/GPU utilization, or run times, so i left it the way it was when i first copied it to make my own, with <avg_ncpus> tset o 0.25 and <max_ncpus> set to 1.

3) the law of diminishing returns is apparent from the very beginning. as you can see, the more POEM tasks i ran in parallel, the more efficiently they ran...well at least that much is true for up to 7 simultaneous POEM tasks. i'm sure at some point, say n simultaneous POEM tasks, efficiency would max out, and going to n+1 simultaneous tasks would actually cause efficiency to decrease. i didn't bother going that far with the test b/c by the time i got to 7 simultaneous POEM tasks, efficiency & run times were starting to plateau enough to warrant concluding the test.

4) despite not having a free CPU core per POEM task once i got beyond testing 4 simultaneous POEM tasks, the drop off in performance was hardly significant, and seemed to fall right in line w/ expectations as far as the law of diminishing returns is concerned.

5) i will say that i was able to obtain better results while running 7 simultaneous POEM GPU tasks than the chart shows, and i did that by suspending Test4Theory@Home and making those CPU resources available to POEM@Home...though i never actually ran a full length test in that configuration. i cut it short for 2 reasons...1) i was simply getting lazy, and 2) it doesn't matter to me if efficiency can be improved while running 7 POEM tasks - not running Test4Theory@Home is not an option.

 

[salvorhardin]

salvorhardin, could you lease elaborate on why you need the <allow_multiple_clients>1</allow_multiple_clients> parameter? i'm running Milkyway@Home on an HD 6950 and POEM@Home on an HD 5870 in the same box, and both projects are crunching away problem-free without that parameter in my POEM app_info.xml...nor does it require the <use_all_gpus>1</use_all_gpus> parameter to recognize a 2nd GPU. i'm not understanding why you need them.

...

I used the guide for multiple clients from vyper's blog and it used that parameter. Just assumed that I needed it. I'm currently just running poem for about 12hrs a day because of the summer heat. I run 4 at a time on my 5850 with nothing else using my cpu cores (mid 80s cpu usage). I usually run my cpu at stock clocks during the summer but it increased my run times by 10 minutes. I upped my cpu to 3.28GHz and now I'm averaging 1:30. Later this summer or early fall I'm planning to build an ivb computer and I'll see how much more I can get out of my 5850 (might splurge and get a new gpu though).

 

[biodoc]

The weather has finally started cooling down enough in New England so I started looking for a project to crunch with my NVIDIA GPUs. I was pleasantly surprised to see that POEM@home had released nvidia opencl apps for both linux and windows in late June.

I'm not using an app_info.xml file right now. With linux, the client grabs a full core and uses all of it but everything is going quite smoothly although it took a half-day yesterday for one rig to grab some WUs.

Another pleasant surprise from what I've read in the forums is the client works on older GPUs including 8800 GTS, and 9800GT, etc.

Finally, a BOINC GPU protein folding project other than GPU_grid, that seems to work smoothly.

With my two GTX460s and a GTX570, it looks like I can do 600K+ per day with just one client per GPU.


EDIT: I'm running either ubuntu or kubuntu 12.4 LTS, boinc 7.0.25 or 7.0.28 and the latest linux Nvidia drivers (304.37). ALL 64-bit.