PrimeGrid Challenges 2021

[Markfw]

Its now running at 4 ghz ! ETA is still 18 hours though....

 

[StefanR5R]

its now running at 3.8 ghz...

So, @StefanR5R , how is 18 hours on this project ??

Let's see:
Ryzen 5950X = 16 cores @ 3.8 GHz, dual-channel DDR4-3800:

2 tasks at a time in 18 hours -> 2.7 tasks/day / (16x3.8) =
0.044 tasks per day and core and GHz​

I take it from your posts that this is with the default 142 W power limit.

Xeon E5-2696-v4 = 22 cores @ 2.6 GHz, quad-channel DDR4-2400:

2 tasks at a time in 22 hours -> 2.2 tasks/day / (22x2.6) =
0.038 tasks per day and core and GHz​

That's at about 220 W or maybe more for the CPU.

Some of Intel's newer CPUs (but only the big ones) may be somewhat faster at it than the E5 v4, but they will certainly burn respectively more power.

Epyc 7452 = 32 cores @ ~2.9 GHz, octa-channel DDR4-3200:

8 tasks at a time in ~85 hours -> 2.3 tasks/day / (32x2.9) =
0.024 tasks per day and core and GHz​

That's with power limit upped to 180 W, but it would probably perform quite similarly at its default 155 W power limit. Unlike the Ryzen and the Xeon, this Epyc has too little L3 caches and is therefore a lot more constrained by RAM performance.

PS:
Keep in mind that llrSOB tasks are somewhat variable in size. This makes comparisons based on random tasks rather imprecise.

 

[Endgame124]

Well, before I get pissed, I will wait until these units finish, and see how the next does (like yours). But at least I know the memory is finally tweaked and working.

There is another level of memory tweaking beyond voltage you can do with sub timings. I was able to drop the latency on my memory kit from like 98ns stock to 64ns with fairly minimal fuss. I can grab my timings if you would like to play with tweaking at that level, but I don’t know much about MSI boards to say where to find the settings. I also don’t know if any of test tools like aida exist for Linux, or if you would mostly just be testing by running prime grid.

i also don’t know what the gains are in regards to prime grid. Obviously if it doesn’t hit memory very hard, tweaking sub timings won’t help much if at all.

 

[Markfw]

There is another level of memory tweaking beyond voltage you can do with sub timings. I was able to drop the latency on my memory kit from like 98ns stock to 64ns with fairly minimal fuss. I can grab my timings if you would like to play with tweaking at that level, but I don’t know much about MSI boards to say where to find the settings. I also don’t know if any of test tools like aida exist for Linux, or if you would mostly just be testing by running prime grid.

i also don’t know what the gains are in regards to prime grid. Obviously if it doesn’t hit memory very hard, tweaking sub timings won’t help much if at all.

well, if you could pass those along (the sub-timings) as I don't know any linux tools to do that.

 

[Endgame124]

well, if you could pass those along (the sub-timings) as I don't know any linux tools to do that.

 

[Ken g6]

Day 3 stats:

Rank___Credits____Username
8______2776694____crashtech
10_____2456463____xii5ku
13_____2152773____Icecold
29_____849301_____emoga
55_____390431_____Orange Kid
70_____312561_____Ken_g6
73_____310986_____Justinus
78_____307508_____biodoc
112____166768_____markfw
147____149814_____waffleironhead
210____5394_______SlangNRox

Rank__Credits____Team
1_____22317462___Ural Federal University
2_____18067393___Czech National Team
3_____13404558___SETI.Germany
4_____9878696____TeAm AnandTech
5_____8537096____Aggie The Pew
6_____6848956____Antarctic Crunchers
7_____4608224____BOINC@MIXI

I'm not the slowest contributor anymore! Though I don't know how long it will stay that way.

 

[Endgame124]

I’ve got Rosetta set to finish existing tasks, so tomorrow morning I’ll be joining prime grid. I’ll be late to the party, but I’m interested in seeing how well my 5950 current configuration does

 

[Icecold]

I wonder at what point all those single core VM's will complete their tasks and Antarctic Crunchers will skyrocket past some(most? all?) teams.

 

[Markfw]

My 3080 is now supposed to be in stock (and shipping out) tomorrow ! It changed by weeks every update until today, and it was only 2 days out, so maybe it will really come in this time !

 

[StefanR5R]

i also don’t know what the gains are in regards to prime grid. Obviously if it doesn’t hit memory very hard, tweaking sub timings won’t help much if at all.

PrimeGrid's LLR based subprojects generally hit the RAM rather hard. On processors without enough last level cache (currently for each SoB-LLR instance: ~25 MB, contiguously), this workload is memory-bound and leaves the execution units, notably the FMAs, rather underused. On processors with enough cache, e.g. Ryzen 5950X if one SoB-LLR instance is executed within one CCX, it appears there is still unusually heavy RAM access. (Remember the crashes which Mark reported initially.) But the SoB-LLR performance will definitely depend considerably less on RAM performance than for processors with smaller cache.

However, I don't know how much RAM latencies matter, versus RAM bandwidth.

 

[Markfw]

PrimeGrid's LLR based subprojects generally hit the RAM rather hard. On processors without enough last level cache (currently for each SoB-LLR instance: ~25 MB, contiguously), this workload is memory-bound and leaves the execution units, notably the FMAs, rather underused. On processors with enough cache, e.g. Ryzen 5950X if one SoB-LLR instance is executed within one CCX, it appears there is still unusually heavy RAM access. (Remember the crashes which Mark reported initially.) But the SoB-LLR performance will definitely depend considerably less on RAM performance than for processors with smaller cache.

However, I don't know how much RAM latencies matter, versus RAM bandwidth.

BTW, the initial crashes were memory running at 1.22v instead of 1.45.

 

[Markfw]

I just saw that running 3800 memory can take 10 watts out of the power budget. Can I up the power 10 watts ? or 20 ? Where is that in the bios ? I just went through the entire manual, and I don't see it.

 

[StefanR5R]

From Ian Cutress' interview with Forrest Norrod of AMD on Epyc Milan:

In Rome or in Naples, with eight channels of memory you could get full performance - you could get a pretty good well optimized and balanced system with only four channels of memory, obviously your theoretical bandwidth is cut in half, but it was well optimized. If you had six channels of memory, you get this somewhat unbalanced condition where latency and throughput [would depend on a number of factors], so that’s what we’ve really tried to address with the six channel to give that additional flexibility to right size the amount of memory for your workload without giving up any of the performance.

Incidentally, I tried llrSOB on Epyc 7452 (32-core Rome) with only 6 memory channels populated. Throughput fell into a bottomless pit. It's less than half compared to a full octa-channel configuration.

 

[Markfw]

So I have checked multiple times on it speed. Its 3.8 to 4 ghz...That good. The memory is at 3800 and the machine is stable with the memory timing tightened, but they are still running 20 hours ETA !!! I don't know how you guys are beating me on times, but I certainly have tried.

Edit: well, I decided to help the team a little more. I added a 7742 to the chase, and its running 8 tasks with 8 cpu's each and says 1 day 15 hours eta. Not bad for 2 ghz.

Edit 2:. The 7742 changed to 3 days 9 hours... WTF ???? I didn't turn SMT off, not sure on the EPYC how to do that. Could that be it ????

 

[Markfw]

Any help ? or abort the 7742 ?

Edit: rebooted to look how to disable SMT. I could not find it, but ETA went back to one day 14 hours, 2.1 % done in 1 hour 45 minutes.

edit 2 4.5% done in 5 1/2 hours.

 

[StefanR5R]

At least on Linux, there is no need to disable SMT. Just tell BOINC to use 50 % of the logical CPUs.

In the AMI BIOS of Supermicro boards, the SMT setting is supposed to be in "Advanced" -> "CPU Configuration".

Boincmgr's estimation of time remaining is not reliable. Perhaps better is to check how fast (well, slow) the completion percentage is progressing.

 

[Markfw]

At least on Linux, there is no need to disable SMT. Just tell BOINC to use 50 % of the logical CPUs.

In the AMI BIOS of Supermicro boards, the SMT setting is supposed to be in "Advanced" -> "CPU Configuration".

Boincmgr's estimation of time remaining is not reliable. Perhaps better is to check how fast (well, slow) the completion percentage is progressing.

Its at 8.3% done in 12 1/2 hours. Not good.

 

[StefanR5R]

In the worst case, memory is still unstable but such that SoB-LLR no longer crashes but instead engages its built-in error correcting mechanism. This would increase run time, due to extra calculations of those sections which were detected as invalid.

However, I am only saying that this is a possibility. I don't know if it actually happens on your computer, and whether or not this can be seen in the local files. One file to check would be "/var/lib/boinc/slots/*/stderr.txt", although I am not sure if such self-corrections are logged there.

 

[Ken g6]

Day 4 stats:

Rank___Credits____Username
8______4196942____xii5ku
9______4186546____crashtech
12_____3322074____Icecold
30_____1339156____emoga
56_____694382_____biodoc
59_____632898_____Orange Kid
68_____546906_____Ken_g6
78_____489958_____markfw
113____310986_____Justinus
153____224604_____waffleironhead
194____80912______SlangNRox

Rank__Credits____Team
1_____31452057___Ural Federal University
2_____25311460___Czech National Team
3_____21419553___SETI.Germany
4_____16025369___TeAm AnandTech
5_____12878078___Aggie The Pew
6_____11130896___Antarctic Crunchers
7_____7485742____AMD Users

Wow, our top two are really close!

 

[crashtech]

I have abandoned PrimeGrid for the FB Sprint Project, Sidock@Home. Sorry, guys. I'm not a huge fan of SoB because it overwhelms much of my hardware.

 

[StefanR5R]

All this time my computers couldn't keep up with @crashtech's overwhelmed hardware.
Now I'm really underwhelmed by my computers.

 

[crashtech]

I will return to Primegrid briefly at the conclusion of the Sprint...

 

[Endgame124]

Is there a post in this thread that I should follow for configuring Primegrid? Sounds like just going with whatever it configures on its own isn't a great idea?

 

[Icecold]

Is there a post in this thread that I should follow for configuring Primegrid? Sounds like just going with whatever it configures on its own isn't a great idea?

This one is probably as close as any as it pertains to this specific challenge - https://forums.anandtech.com/threads/primegrid-challenges-2021.2589211/post-40462572

But, the main things would be turn off SMT either through the bios, or task manager processor affinity(if running Windows), or by setting BOINC to use 50% of CPU's. Then configure in your Primegrid preferences to only run Seventeen or Bust tasks and to use 8 threads per task. I think if you're running Windows it will help to configure processor affinity per Primegrid process to keep each Primegrid task running on a single CCX. On Linux it seems to sort itself out on its own. This is all under the assumption it's on your 5950x, other processors would need different thread counts potentially based on cache, etc.

 

[Markfw]

I am at 13.6% done with my 8 tasks on my 7742 at 21 hours 40 minutes. Thinking of putting it back to Rosetta.