Question Rosetta@home now requires virtual box on ubuntu ?? linux/virtualbox experts needed to get more than 16 tasks !

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,483
14,434
136
I tried to start Rosetta back up after the WCG 17th Birthday event, but I get the error in red "virtualbox is not installed". So after my google research failed.... I typed:

sudo apt-get install virtualbox and rebooted. Now its working, but says "rosetta python projects (vbox64)"

So the question is, why is it now required, and is it faster or slower ? And what a pain to have to do this on every box that runs Rosetta now.
 

Icecold

Golden Member
Nov 15, 2004
1,090
1,008
146
I just tried to load up Rosetta on a machine that doesn't have Virtualbox installed and it won't get any work and shows the 'virtualbox is not installed' in the log entry that you were seeing. I'm going to install Virtualbox and will post back with the results. This thread has some info, it seems like they're out of the previous tasks and only 'python' tasks are available now from what i can tell by skimming it - https://boinc.bakerlab.org/rosetta/forum_thread.php?id=14771#102927
 

biodoc

Diamond Member
Dec 29, 2005
6,257
2,238
136
Check RAM usage. I think the python tasks take something like 8 GB per task. If so, it's a non-starter for me.
 

Icecold

Golden Member
Nov 15, 2004
1,090
1,008
146
I'm having the same issue you are Mark. I installed Virtualbox and it does download tasks now, but it's only running 16 tasks on a 256 thread EPYC with 256GB RAM.
 

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,483
14,434
136
I'm having the same issue you are Mark. I installed Virtualbox and it does download tasks now, but it's only running 16 tasks on a 256 thread EPYC with 256GB RAM.
If its 8 gig per tasks, you should have 32, so something is not right. We need the virtualbox and linux experts... HELP !!!!Edit: on one box with only 110 gig of ram, its using 22.5 gig actual and only running 14 tasks, I still could get at least 64 more tasks and not run out.

Edit: On my dual 7601 box, it has a bunch of computational errors, and is now only running 5 tasks and 10.5 gig of 256 gig memory usage !

Unless some magic happens, Rosetta may be dead for me. I will not let all these CPUs be idle.
 
Last edited:

Icecold

Golden Member
Nov 15, 2004
1,090
1,008
146
My memory usage is steadily increasing. It was at 41GB about a minute ago, and is up to 46.4GB.

I'm not sure how they could expect people to have 8GB RAM per thread though, I don't have any machines that with a RAM capacity higher than 2GB per thread.
 
Last edited:

Markfw

Moderator Emeritus, Elite Member
May 16, 2002
25,483
14,434
136
My memory usage is steadily increasing. It was at 41GB about a minute ago, and is up to 46.4GB.

I'm not sure how they could expect people to have 8GB RAM per thread though, I don't have any machines that with a RAM capacity higher than 2GB per thread.
They are not going to have a lot of contributors if this is the final result.
 

biodoc

Diamond Member
Dec 29, 2005
6,257
2,238
136
I'm running WCG and Rosetta on my 5950X and currently have 7 Rosetta python vbox tasks running. Each task uses 2 cpu threads for a total of 14 threads. The remainder of the available threads are being used by WCG. Highlighting one of the vbox tasks and choosing properties shows a "working set size" of 7.45 GB. I suspect this value is set by the server and is what limits the total number of running tasks but that's speculation on my part. So it looks like I'll need to run both projects to use all the CPU threads available.

Application: rosetta python projects 1.03 (vbox64) Name: boinc_cages_IL_2727241_55108 Resources: 2 CPUs Estimated computation size: 80,000 GFLOPs Virtual memory size: 2.89 GB Working set size: 7.45 GB Progress rate: 14.040% per hour Executable: vboxwrapper_26198_x86_64-pc-linux-gnu

EDIT: 7 rosetta python vbox tasks are using 28.81 GB of disk space! Check to make sure you have enough disk space allocated to boinc.
Screen shot of my 5950 running both WCG and Rosetta.
1637845193652.png
 
Last edited:

biodoc

Diamond Member
Dec 29, 2005
6,257
2,238
136
So it looks like I'll need to run both projects to use all the CPU threads available.
This isn't working. My cache is overloaded with rosetta vbox tasks and no new WCG are being downloaded. I guess the solution would be to run 2 boinc instances on the 5950X with one for Rosetta (14 threads) and the other WCG (18 threads). Or maybe messing with the resource share for each project might help.
 

crashtech

Lifer
Jan 4, 2013
10,521
2,111
146
It can be tough to get two projects to play nice within one client, in my experience. I usually don't even bother unless I'm just clearing out some stragglers.
 

Icecold

Golden Member
Nov 15, 2004
1,090
1,008
146
The answer for now(based on Michael Goetz's post to the Rosetta forum) is to overprovision RAM by using extra client instances since the tasks don't actually need 8GB per task. I was having some major performance issues even running 30 of the Rosetta tasks, though, BOINC manager kept disconnecting from the BOINC client like the client was hung up, etc. so I'm not sure if that will work or not.
 

StefanR5R

Elite Member
Dec 10, 2016
5,459
7,718
136
The answer for now(based on Michael Goetz's post to the Rosetta forum) is to overprovision RAM by using extra client instances since the tasks don't actually need 8GB per task. I was having some major performance issues even running 30 of the Rosetta tasks, though, BOINC manager kept disconnecting from the BOINC client like the client was hung up, etc. so I'm not sure if that will work or not.
Overprovisioning sounds tempting. I learned from @Endgame124 that this is even possible in a single client instance, by configuring >100% allowed RAM usage via an edited global_prefs_override.xml. *However*, based on what I saw at other projects which use vboxwrapper (I haven't tested this new Rosetta application yet), you definitely want to maintain a very responsive system at all times. Otherwise you might frequently see tasks which get stuck with "Postponed: VM job unmanageable, restarting later". Such tasks can then only be aborted, IME, with the respective loss of the task's computation up until that point. (Edit: There may be workarounds to get such tasks back up and running, but they don't seem reliable and feasible to automate.) Therefore I have doubts that overprovisioning of RAM is the way to go with a vboxwrapper based application like this.
 
Last edited:
  • Like
Reactions: Endgame124

voodoo5_6k

Senior member
Jan 14, 2021
395
443
116
I am leaving. WCG for now.
I'm with @Markfw. I was annoyed by the project official's lack of communication for quite some while anyhow (therefore, only my RPi4 8G was left running Rosetta, but it only occasionally got tasks for several weeks now). But reading through this thread, wow. So, yeah, all-in for WCG on my end too. Until something changes on their end, I don't see much benefit in having resources on that project.