Poem@Home GPU

salvorhardin

Senior member
Jan 30, 2003
390
38
91
Poem@Home released gpu applications at the end of december. It's currently available for linux (nvidia & amd) and windows (amd only). I've been running it for about a month now on my 5850.

Disclaimers:

  • They've been having growing problems with the amount of work being done by the gpus. There are times when work isn't available. They have been upgrading their servers to handle the load but so far it hasn't been able to keep up.

  • There is no applications for nvidia on windows, they are still working on it.
  • The windows app isn't very efficient so each wu basically needs a cpu core to feed it. The gpu might not also be fully utilized. When running my 5850 only I averaged 56% utilization.
  • To run gpu workunits you need to run at least boinc 7.0.


Some tips that I've found useful in crunching:
Boinc 7.0.14 works fine for me with minimum work buffer set to 1 day and max buffer set to 5 days.

The windows app isn't very efficient on using the gpu so I had to set an app info file to run four work units at a time. My app_info file:
Code:
<app_info>
<app>
<name>poemcl</name>
<user_friendly_name>POEM++ OpenCL</user_friendly_name>
</app>
<file_info>
<name>poemcl_0.1_windows_intelx86__opencl_ati_100</name>
<executable/>
</file_info>
<app_version>
<app_name>poemcl</app_name>
<version_num>1</version_num>
<plan_class>opencl_ati_100</plan_class>
<avg_ncpus>0.25</avg_ncpus>
<max_ncpus>1</max_ncpus>
<flops>2.1e12</flops>
<coproc>
<type>ATI</type>
<count>0.25</count>
</coproc>
<cmdline></cmdline>
<file_ref>
<file_name>poemcl_0.1_windows_intelx86__opencl_ati_100</file_name>
<main_program/>
</file_ref>
</app_version>
</app_info>

To help increase my chances of getting work I added <report_results_immediately>1</report_results_immediately> to my cc_config file so the client connects to the server as soon as a workunit is completed. My cc_config file has extra settings because I normally have 2 different gpu projects running at once but in the last week I have both gpus running poem.
Code:
<cc_config>
  <options>
     <allow_multiple_clients>1</allow_multiple_clients>
     <use_all_gpus>1</use_all_gpus>     
     <report_results_immediately>1</report_results_immediately>
  </options>
</cc_config>

The best version of the amd drivers that have worked for me is 12.1. It cut my cpu usage by almost half. Before using 12.1 each wu would use a full core, but with 12.1 it averages 60% of a core.

I'm going to be running poem until I hit 10 million at which point I'll move on to another gpu project. I've been looking at moowrapper for my 4830 and albert
for my 5850. 2012 will be a gpu year for me.
 
Last edited:

petrusbroder

Elite Member
Nov 28, 2004
13,348
1,156
126
Thanks for the info and code. I have been thinking for something new to do for my ATI-GPUs. :)
 

zzuupp

Lifer
Jul 6, 2008
14,866
2,319
126
I keep hoping they'll figure out the issues in the windows/nvidia combo
 

biodoc

Diamond Member
Dec 29, 2005
6,350
2,243
136
Thanks for the detailed info, salvorhardin. :)

I've been running Poem on my 2 4850s for awhile now on linux but as noted by salvorhardin, work is not always/rarely available. They are planning on a server upgrade to solve that problem so I believe the project will be down this thursday and friday for the upgrade.

I and others have not been able to get the nvidia app to work on linux and as far as I know, they haven't been able to solve that problem yet. For nvidia cards, you will need opencl version 1.1 and above (that rules out my GTX295:() and for ATI cards, you need opencl 1.0 and above. I suspect double precision capable gpus are also required but I'm not sure about that.

On linux, you also need to free up a CPU core to run the app efficiently.

It's nice to have another boinc biology project that is GPU capable AND linux friendly. :cool:
 

salvorhardin

Senior member
Jan 30, 2003
390
38
91
The server has been upgraded and it looks like there is more consistent work available. I wan't able to run poem the last couple of days, but I have been checking the amount of work available. It's been consistently over 5000 wus available. One of the changes they have done since the server upgrade is to go to a fixed credit system. Each gpu wu is now worth 2925. With my cpu cores feeding my gpus I'm now at 97% utilization on my 5850 with poem and 96% on my 4830 with moowrapper.
 

Sunny129

Diamond Member
Nov 14, 2000
4,823
6
81
The server has been upgraded and it looks like there is more consistent work available. I wan't able to run poem the last couple of days, but I have been checking the amount of work available. It's been consistently over 5000 wus available. One of the changes they have done since the server upgrade is to go to a fixed credit system. Each gpu wu is now worth 2925. With my cpu cores feeding my gpus I'm now at 97% utilization on my 5850 with poem and 96% on my 4830 with moowrapper.
out of curiosity, how long does it take your 5850 to crunch through the average POEM@Home WU?
 

salvorhardin

Senior member
Jan 30, 2003
390
38
91
It takes about 2hrs 5min, but I do 4 at time. I haven't tried 1 at a time, so I don't know if it will reduce the run time. I went straight to 4 at a time since I was seeing in different forums that the poem app doesn't fully utilize the gpu and you need to run more at a time in order to increase utilization.
 

blckgrffn

Diamond Member
May 1, 2003
9,687
4,348
136
www.teamjuchems.com
So, if I wanted to have that app_info file crunch just 1 GPU WU at a time, but reserve a whole CPU in BOINC at the same time, what lines would I change? I think I see what to change (.25 ATI GPUs to 1, set min/max CPU to 1) but maybe I am wrong :p

The other alternative, which is less desirable, is to drop a CPU out of BOINC entirely so that when POEM needs ~.095 CPUs, there is plenty of grunt for OpenCL over head. I have to do this when folding on the GPU as well...

Thanks in advance :)
 

salvorhardin

Senior member
Jan 30, 2003
390
38
91
If you wanted to do that I would just run poem without an app_info file and just limit you cpu projects so that 1 core is free. The settings to limit cpu for a poem wu are:

<avg_ncpus>0.25</avg_ncpus> - sets average cpu use
<max_ncpus>1</max_ncpus> - sets max cpu use
<count>0.25</count> - how many wus at a time (1=1, 0.5=2, 0.25=3)

When I was testing what settings would work on my system I noticed that poem would take the processing power it needed and ignored any settings related to cpu use. On my 5850 actual cpu use under windows task manager was 18% on a quad core system. Right now I got poem running on my 4830 with no app_info with 10% cpu and 67% utilization. For some reason I would not get any work with an app_info file.

...With my cpu cores feeding my gpus I'm now at 97% utilization on my 5850 with poem and 96% on my 4830 with moowrapper.

I recently found out that on a multi gpu system moo wrapper will use any gpu that is active in windows to crunch. It will ignore any settings set by the project or boinc. I had set my 5850 to be ignored by the client running moo, but I noticed increased run times in both poem and moo. After reading the moo forums I found out that since the program is not run directly by boinc it ignores any boinc settings.
 

blckgrffn

Diamond Member
May 1, 2003
9,687
4,348
136
www.teamjuchems.com
Thanks! I received one WU and dropping BOINC to 5 CPUs on my Thuban sees the video card getting hit a lot harder (full speed clocks, 40% usage) but that makes me think that another WU running on there concurrently would be the sweet spot :)

I am going to let this WU finish and then perhaps tweak some more.

Thanks again! Clarifying the app_info file details is really very helpful. I am still at the point that I find that type of tuning something akin to alchemy :)
 

salvorhardin

Senior member
Jan 30, 2003
390
38
91
Your Welcome. Once you add in an app_info you can adjust the settings without losing wus. The first time you start with an app_info you will lose any wus you had, but once you have started with it you can exit boinc and adjust the settings. You might have to go higher than 2 at a time. when you do multiple wus at a time it doesn't scale linearly. On my 5850 going from 2 to 4 at a time my utilization went from 39% to 57%. you just have to play with the settings to see what gives you the most ppd with the cpu utilization you want. When running 4 at a time my cpu was always in the 80 percentiles.
 

blckgrffn

Diamond Member
May 1, 2003
9,687
4,348
136
www.teamjuchems.com
Holy crap, I finally tweaked my config so I am running 3x GPU tasks concurrently (which is finally causing the GPU fan to spin up just enough every once in a while I can hear it, so I am going to call it good for now) and dedicated 2x CPUs...

I went from crunching one task in ~3300-3700 seconds to three tasks in 5300-5700 seconds. Before, it would have taken me ~10k seconds to do three WU's, now I am doing six in ~11k! Well worth the time and effort, IMHO. Which was minimal. I did end up having to reboot though, BOINC and POEM got angry...

Next week, the team lead once again WILL BE MINE!!!! :p

Also, hoping to spin up the other card next week. I am going to have to rebuild that cruncher completely, I think, to swap in a motherboard with 2x 16x PCIe slots. I may also move my 1090t in that build as well... decisions, decisions :)
 
Last edited:

Alyx

Golden Member
Apr 28, 2007
1,181
0
0
I put two 6950s on the project. Lets see if we can go up some ranks. :D
 

blckgrffn

Diamond Member
May 1, 2003
9,687
4,348
136
www.teamjuchems.com
I put two 6950s on the project. Lets see if we can go up some ranks. :D

Darn you!

*eyes another 5870 in FS/FT...*

Awesome, BTW :p If I can get 150k ppd on a 4770, I am eager to see what damage you can do!

I've found it best to pair a CPU per GPU WU, so I hope you have some threads available :) I would think ~5 GPU tasks per card (or more?) would be optimal.
 
Last edited:

Alyx

Golden Member
Apr 28, 2007
1,181
0
0
Darn you!

*eyes another 5870 in FS/FT...*

Awesome, BTW :p If I can get 150k ppd on a 4770, I am eager to see what damage you can do!

I've found it best to pair a CPU per GPU WU, so I hope you have some threads available :) I would think ~5 GPU tasks per card (or more?) would be optimal.

I've currently got it set to 5, but I'm only getting 2 tasks at a time right now. Hopefully it'll iron out once my host has some history with the Poem servers. I've got one card per machine, and i'm dedicating all 8 cores to the 5 tasks.

i have an HD 6950 2GB card for sale if you need to up your POEM production ;)

All you, blckgrffn. Boy I wish I could, but my wife would strangle me. :)
 

blckgrffn

Diamond Member
May 1, 2003
9,687
4,348
136
www.teamjuchems.com
I've currently got it set to 5, but I'm only getting 2 tasks at a time right now. Hopefully it'll iron out once my host has some history with the Poem servers. I've got one card per machine, and i'm dedicating all 8 cores to the 5 tasks.



All you, blckgrffn. Boy I wish I could, but my wife would strangle me. :)

Yeah, it took a bit, the WU's started coming in. I think it is giving me some ~20 at a time now. Not sure how long that took...

Haha, yep, my wife would strangle me too :p

I'd feel a touch silly buying a better card to crunch with than what I have in my main PC to game with... I guess I am not quite dedicated enough yet :D

I still think we are going to get some epic numbers once your GPU's spin up!

I have 3 CPU's for POEM and 3 for Malaria Control on my 1045t rig right now, seems like a good balance.

Thanks again to Salvorhardin for heads up and the assistance with configuring for best output :)
 

salvorhardin

Senior member
Jan 30, 2003
390
38
91
I'm glad to be seeing some more competition for my position. I had been running my 4830 with 2 at a time. After uninstalling boinc I've now been able to maintain a cache, previously I would have to keep hitting update just to get one task. I'm currently running einstein (2 at a time) on my 5850 and poem (3 at at time) on my 4830. My gpu utilization is currently 89% for the 5850 and 95% for my 4830, depending how much ppd I lose in einstein I might switch back to 2 at a time in poem. Hopefully running 3 at a time in poem will offset the loss in einstein and buy me some extra days at the top. My cpu doesn't have enough threads to feed my gpus. If I swith back to 2 at a time in poem my utilization on my 5850 increases to 94% but my 4830 decreases to 90%.
 

blckgrffn

Diamond Member
May 1, 2003
9,687
4,348
136
www.teamjuchems.com
Given that an unlocked 6950 or 7950 is worth nearly 500k ppd I expect to see some epic numbers, guys! :)

It's all about having enough CPU threads to feed the beasts. The x6's show their usefulness again :)
 

salvorhardin

Senior member
Jan 30, 2003
390
38
91
Looks like they finally created more work. There's currently around 30k unsent wus and 28k in progress. Earlier in the week there would always be around 5k combined. My queue is now up to 149 wus. On the 22nd we almost hit 1 million ppd as a team, about a third of aprils total production.

teammonthlysingle.php