• Guest, The rules for the P & N subforum have been updated to prohibit "ad hominem" or personal attacks against other posters. See the full details in the post "Politics and News Rules & Guidelines."

Question Who is the Rasberry PI expert ? And a recommended system.

Page 2 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

Endgame124

Senior member
Feb 11, 2008
391
228
116
For multi pi case, I haven’t used it, but this one is recommended by someone on the hard forum

 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
Case is complicated. I’m using individual FLIRC cases, but I’ll share some ideas later.

Best place to buy multiple pis (cheaper than amazon)

Note, microcenter is cheapest for individual pi purchases, as they have a permanent $5 off MSRP In store option. However, you can only get one of each ram size per purchase with the discount (ex, buy a 2GB, 4gb, and 8gb in one purchase and get $15 off total).

OS: use raspbian lite. There are complications with this with Rosetta that are covered in my guide. I have shell scripts that will do all the config.

You’ll need a SD card reader / writer to flash the cards with the OS. Use the official software to do the flashing, it’s vastly simplified over any other process
OK, to stop half the question, I just ordered ONE to start with. Here: https://www.newegg.com/p/1HD-005N-000J9?Item=9SIAHUBBMR9923

I hope its a good setup. That site with multiple PI's looks like none of the "standard" ones.

Oh, and I like that case !!!
 

Endgame124

Senior member
Feb 11, 2008
391
228
116
OK. The littlest one is in bed, and the older one is sitting next to me watching paw patrol, so I have some time to type.

On the kit you bought. These starter kits are generally designed as a quick start to a desktop replacement type system. The SD card comes preloaded with NOOBs - the full version of Raspbian, complete with linux desktop, and all the raspberry pi apps. Since ram is precious with the Pi, I run headless and do everything over SSH and manage from command line. My day job is also in IT as a Performance engineer with 20 years of systems engineering experience, so I'm completely OK running entirely from command line.

Note that Pis don't really power down like a normal computer - the best you can do is something like "shutdown now" from the command line and it will put the pi in deep sleep. You would need to unplug and replug to power back up. That is why the power supply has a switch on it - to facilitate turning it on and off as desired (not really a use case for my pis, which are running 24x7.

For other items in the bundle:

Case:
Its a clear, plastic case. It likely has little airflow (I bought a similar starter set with my first pi). These are fine when using the pi as a desktop usually at idle. For 100% cpu 24x7, you'll find yourself thermally throttling. k

My recommendation here: Don't attach the top half of the case, leave it open. Also, stand the pi on its side - there has been lots of testing done to show you'll knock a few degrees off a pi

SD card:
The Samsung evo is a good card. The samsung select that I linked is just the amazon exclusive of the same card that costs a little bit less. 32GB is plenty.

Heatsinks:
Really the only heatsink that matters is the one that you apply to the CPU (silver capped chip in the center-ish of the board). minimal airflow with a single heatsink is enough to keep the Pi from thermally throttling.
 
  • Like
Reactions: Markfw

Endgame124

Senior member
Feb 11, 2008
391
228
116
I've now read through the whole thread.

If I understand correctly, this is your goal:
Create a low power DC compute farm from 8 to 12 Pi 4s to use on Rosetta

Based on your questions and comments, I believe the secondary goal is:
Optimize purchases to support multiple Pis in a cost efficient manner

This goal is totally achievable!

I have approached my pi cluster, from a high level like this:

1) Optimize power usage per pi by tweaking the OS and under volting each individual pi. However, I only use settings that work on all Pi 4s, as I don't want to manage individual pis. I've been able to push power load down to about 4.25 - 4.5 watts per pi, which is about the same efficiency as a power optimized threadripper 3990 (as far as I can tell - I don't have a thread ripper to compare)

2) Optimize points per day by using the most cut down version of Raspbian (Debian linux tailored to raspberry pi) to save ram for DC. Additionally, I try to never mix projects on the same pi - setting 2 Pis to run 2 processes of open pandemics and 2 procesess of Rosetta each yields about 20% lower points / credit than than dedicating one pi to each project.

3) Optimize cost first by using different size pis for specific projects. Open pandemics easily fits within 1GB of ram, so I'm deploying half of my pis as 2GB pis. When open pandemics finishes, I'll pick other DC projects to support with the 2GB pis - hopefully it will be something else medical related.

4) Optimize cost second by smart purchasing additional components. Ex, the 6 port Anker power brick is dramatically less expensive than buying 6x individual power bricks. I've played around with just using a single heatsink on the pis (skipping cases all together), and network booting the Pis to avoid needing to buy SD cards. I've had mixed success with this - I can share if you're interested.

5) Optimize my time to manage this, meaning that I've scripted the setup, and have a few tweaks in place to be able to quickly identify problems with an individual pi.


For references, you have the link to my pi thread here on the AT forums. I just skimmed my pi thread and realize it needs a lot of work. Adjustments for the pandemic have meant that I just have not had time to put much work into that thread. Maybe I can do it over a few nights this week. Anyway...

I've been probably slightly more active on the hardforum. If you want to see my progression in working with Pis, you can read these two threads on the hard forum:

There is also a long thread on the Rosetta forums that may be an interesting read:
 
  • Like
Reactions: biodoc and Markfw

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
For SD card, I prefer these


Power loss to a pi while it’s using the card can cause damage. I’ve lost 3 of the super cheap microcenter cards. Have had no problems with these Samsung cards. That said, I’m now running them on UPS, which has resolved all the power issues.
Do you need that card all the time ? or just to boot ?
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
And what do you think of this PSU ? 10 devices ?


OK, the rest of my cart. Please tell me what I am missing for an 8 card system (32 cores)

https://www.amazon.com/gp/product/B07TC2BK1X/ref=ox_sc_act_title_1?smid=AAU5UPIIBDRLP&psc=1

(times 7, only 4 gig ??)

Heat sinks ? (need one more)


The case (you recommended it)


The 32 gig cards (you recommended it)


Anything I am missing for the entire array ?
 
Last edited:

Endgame124

Senior member
Feb 11, 2008
391
228
116
Do you need that card all the time ? or just to boot ?
If you boot from it, you need to leave it in(no hot swap on the sd card)

You need storage for a few things

1) A place to store the Boinc files and work units
2) something to boot from
3) swap

Assuming you have network storage, 1 is easy. 2 and 3 are a lot more work, but can be done if you want to run entirely without a SD card
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
If you boot from it, you need to leave it in(no hot swap on the sd card)

You need storage for a few things

1) A place to store the Boinc files and work units
2) something to boot from
3) swap

Assuming you have network storage, 1 is easy. 2 and 3 are a lot more work, but can be done if you want to run entirely without a SD card
See above... BTW, so I need that adapter for the SD card that shows up in my newegg combo ? and will 4 gig work for WCG ? I think I will leave Rosetta for the EPYC boxes.

And lastly, what was the PPD/watt for EPYC vs the PI ????? I looked at the links, but its not clear.
 

Endgame124

Senior member
Feb 11, 2008
391
228
116
posting while wrangling kids :)

Short answers

Power supply - no idea, I’ve found quality varies a lot per unit. A overclocked pi can take 6w, an under volted one as little as 4.25w. Depending on use case 60w may not be enough. No idea how efficient that supply is either - might be 60% or 85%.

Heat sink should be fine - I’m using old vga memory heart sinks and they work fine

The pi uses micro SD. The cards come with a SD adapter so you can use a SD writer with them. I have one on my laptop - if you don’t have one, you’ll need one
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
BTW, the entire 32 core 8 slot PI setup mostly 4 gig, will be about $700 total. Thats why I am interested in it ppd VS EPYC. I have a 32 core EPYC (64 threads) sitting idle, as the motherboard is over $600 and memory is another $600
 

Endgame124

Senior member
Feb 11, 2008
391
228
116
Answering a few more questions quick

WCG projects are all different from what I understand. I’ve only used open pandemics, and it uses less than 250mb per process, so you can used 2GB pi 4s without any issue.

On the SD card, the Samsung select cards come packaged with an adapter. The question is, do you have a sad writer?
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
Answering a few more questions quick

WCG projects are all different from what I understand. I’ve only used open pandemics, and it uses less than 250mb per process, so you can used 2GB pi 4s without any issue.

On the SD card, the Samsung select cards come packaged with an adapter. The question is, do you have a sad writer?
I have many adapters that work with SD cardss to put them in USB slots, and I can transfer data that way. Is that what you mean ?
 

Endgame124

Senior member
Feb 11, 2008
391
228
116
Open pandemics points per day varies a bit - here is a copy paste for one of my pis:

23/2020 0:004:07:08:38 6,424 11
09/22/2020 0:003:09:33:53 5,132 9
09/21/2020 0:003:18:42:07 5,665 10
09/20/2020 0:004:05:20:05 6,180 11
09/19/2020 0:003:13:18:22 5,165 9
09/18/2020 0:002:15:34:50 4,012 6
09/17/2020 0:001:19:38:27 2,680 5
09/16/2020 0:002:23:37:39 4,388 8
09/15/2020 0:003:10:32:08 5,056 9
09/14/2020 0:003:11:59:22 5,191 9
09/13/2020 0:004:04:44:26 6,265 11
09/12/2020 0:004:19:47:34 7,353 12
09/11/2020 0:003:00:26:19 4,720 8
09/10/2020 0:004:01:33:50 6,249 11

Eyeball says it is around 5,500 / day (I didn’t actually do the math). Power draw for the pi is around 4.5w with wired networking set to 100mbit, LEDs off, hdmi off, Bluetooth off, etc.

I don’t have good numbers for pandemics points per watt for the higher end systems.

Rosetta average credit for a pi 4 4gb varies between 1000 and 1300.
 

Endgame124

Senior member
Feb 11, 2008
391
228
116
I have many adapters that work with SD cardss to put them in USB slots, and I can transfer data that way. Is that what you mean ?
I think so. I’m talking something like this:


I use the one built into my laptop, so this is an area I have little expertise with.
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
I can't link to that power supply anymore, the link now says it can't find the page.
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
Open pandemics points per day varies a bit - here is a copy paste for one of my pis:

23/2020 0:004:07:08:38 6,424 11
09/22/2020 0:003:09:33:53 5,132 9
09/21/2020 0:003:18:42:07 5,665 10
09/20/2020 0:004:05:20:05 6,180 11
09/19/2020 0:003:13:18:22 5,165 9
09/18/2020 0:002:15:34:50 4,012 6
09/17/2020 0:001:19:38:27 2,680 5
09/16/2020 0:002:23:37:39 4,388 8
09/15/2020 0:003:10:32:08 5,056 9
09/14/2020 0:003:11:59:22 5,191 9
09/13/2020 0:004:04:44:26 6,265 11
09/12/2020 0:004:19:47:34 7,353 12
09/11/2020 0:003:00:26:19 4,720 8
09/10/2020 0:004:01:33:50 6,249 11

Eyeball says it is around 5,500 / day (I didn’t actually do the math). Power draw for the pi is around 4.5w with wired networking set to 100mbit, LEDs off, hdmi off, Bluetooth off, etc.

I don’t have good numbers for pandemics points per watt for the higher end systems.

Rosetta average credit for a pi 4 4gb varies between 1000 and 1300.
I can't equate that to this, can you help ?


If that link does not work, can you look at my results ? Or compare to something you have ?
 

Endgame124

Senior member
Feb 11, 2008
391
228
116
I can't equate that to this, can you help ?


If that link does not work, can you look at my results ? Or compare to something you have ?
I can’t see your individual devices it seems, but I can see your user statistics. You have an average points per hour of run time of 195, while mine is 48. Given how many devices you have, and the fact I have a mix of pi 2s, 3s, and 4s. that may not be a very useful comparison.

Here is an older screen shot of the open pandemics performance of one of my 2GB pis:


Perhaps that will be useful to you? If not, I can break it down to what ever formula you would like.

Ex, I could total it up, divide by 4 to get points per thread, and then divide by 14 to get average points per day per thread. That could be divided by 24 to get average points per hour per thread. Each thread is about 1.06w on the pi, so that could be converted to average points per hour per watt hour.
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
I can’t see your individual devices it seems, but I can see your user statistics. You have an average points per hour of run time of 195, while mine is 48. Given how many devices you have, and the fact I have a mix of pi 2s, 3s, and 4s. that may not be a very useful comparison.

Here is an older screen shot of the open pandemics performance of one of my 2GB pis:


Perhaps that will be useful to you? If not, I can break it down to what ever formula you would like.

Ex, I could total it up, divide by 4 to get points per thread, and then divide by 14 to get average points per day per thread. That could be divided by 24 to get average points per hour per thread. Each thread is about 1.06w on the pi, so that could be converted to average points per hour per watt hour.
Whatever the best you can do. I am just trying to figure out if its worth the $700 vs another 16 or 32 core EPYC. I even have one if I can find the motherboard cheap.
 

Endgame124

Senior member
Feb 11, 2008
391
228
116
Whatever the best you can do. I am just trying to figure out if its worth the $700 vs another 16 or 32 core EPYC. I even have one if I can find the motherboard cheap.
From my WCG stats, where each Pi 4 is a 2GB variant, undervolted and power tweaked, using approximately 4.25 watts to 4.5 watts. Huge note here: each pi has a small variance in voltage and power draw from my experience, especially when going from 2GB to 4GB to 8GB models.

Additionally I don't 100% trust my UPS and Power meter at these low power draws, so there is some uncertainty. The_Heretic over on the [H] forum cut up a power supply and did some actual readings with a volt meter, and got 4.45w for a stock 4 GB Pi 4, which would put a power optimized one right around, or just under, 4w.

Power supplies also play a large part in the efficiency, and these usb power bricks are very hard to get efficiency numbers on. I e-mailed multiple vendors, and only Anker got back to me, stating that their power supplies are 85% efficient at any load. Real world testing only shows that 85% is when the power brick is near 50% capacity - with a very small load from a single pi, its more like 70%.

Averages for 1 pi:
4.25-4.5 watt drawn at the wall
Daily: 5263 points at 102 - 108 watt hours.
Hourly (real world): 219.29 points at 4.25 - 4.5 watt hours.
Per thread Per hour: 54.82 points at 1.0625 - 1.125 watt hours

expected for 8 pis.
34-36 watt drawn at the wall
Daily: 42,104 points at 816 - 864 watt hours.
Hourly (real world): 1,754.33 points at 34 - 36 watt hours.

There may be some variance between compute hours and real world hours as listed in the WCG stats, but I've run out of time to work out the actual compute time of the stats - it should be fairly close to real world hours, given its averaged over 13 days per pi.

My approximation of WCG OP Points per day per watt: 51.59

In past comparisons, the Pi 4s have come out slightly less efficient than a power optimized 3950x, and more efficient than a stock 3950x. Those big Ep
 
Last edited:

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
@Endgame124
OK, it came today. I installed everything like the manual says. I plug in the power, and I get a multicolored screen. It says I am supposed to click install, but there is nothing on the screen, not even a mouse cursor.
 

Endgame124

Senior member
Feb 11, 2008
391
228
116
@Endgame124
OK, it came today. I installed everything like the manual says. I plug in the power, and I get a multicolored screen. It says I am supposed to click install, but there is nothing on the screen, not even a mouse cursor.
Something like this thread?


I’ve heard of the rainbow screen and it being related to insufficient power. I don’t even own a mini hdmi cable, so I’ve never actually used a pi 4 with a screen before. I’ve always used raspbian lite and done everything via ssh and command line.
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
Well, they said something about the red light flashing meant bad power, but I have no flashing.. Solid red.
 

Markfw

CPU Moderator, VC&G Moderator, Elite Member
Super Moderator
May 16, 2002
20,812
8,997
136
update. I now I have OS installed, but the update failed for some reason (OS updates). I have rebooted. It was probably due to me uplugging the network cable. Its now on wifi. How do I start that again ?
 
Thread starter Similar threads Forum Replies Date
E Distributed Computing 11

ASK THE COMMUNITY