News Rosetta's role in fighting coronavirus

[biodoc]

I'm going to make a report over at the R@h forum. But before that, the scatterplot...

Update,
what happens is that the faulty tasks never finish their first "decoy", and the watchdog hits 4 hours after the target CPU time. I was referred to Ralph@home, where Rosetta v4.15 (released on April 6) is waiting to be tested.

Thanks for reporting this. You'd think they'd cancel all the linux i686 tasks and stop sending them.

 

[StefanR5R]

Sadly, neither boinctasks nor boinccmd seem to be able to distinguish i686 from x86-64 tasks. I am getting only few of them now, but still some, and need to find them with boincmgr's task property window. Estimated time remaining is an indicator for these tasks too.

Meanwhile, I followed the moderator's suggestion and attached to Ralph, to test the newer application version. Of course there aren't tasks at Ralph for the time being. :-|

Edit,

You'd think they'd cancel all the linux i686 tasks and stop sending them.

Maybe they had cases of proper returns of i686-pc-linux-gnu results from other hosts. My hosts might still be "special" in some obscure way.

 

[Endgame124]

Sadly, neither boinctasks nor boinccmd seem to be able to distinguish i686 from x86-64 tasks. I am getting only few of them now, but still some, and need to find them with boincmgr's task property window.

Meanwhile, I followed the moderator's suggestion and attached to Ralph, to test the newer application version. (I set Rosetta to 0 resource share, and made sure to check Ralph in constant periods for tasks.) Of course there aren't tasks at Ralph for the time being. :-|

There aren’t any Rosetta tasks left either. I setup my laptop for Rosetta right before it ran out... really need to find another boinc project to join when Rosetta tasks are gone.

edit
I think WCG is the fallback project of choice with open pandemic coming online soon. all of my existing hosts have Rosetta jobs still to do, but as they run out, I’ll figure out how to add wcg as a backup project

 

[StefanR5R]

If you have more than one project active in the same boinc client, the client will take multiple factors into account when deciding how much work to fetch from where, and which of the queued tasks to start first. Among them: Which project hasn't been run in a while; what deadlines do the tasks have...

You can shift priorities somewhat by setting different "resource share" values in the web preferences of each of your active projects.

One resource share value is special: 0. This value will cause the client to fetch work from such a project only if there is no longer any work left ready to run in the queue but a CPU is becoming ready for work. Furthermore, this value will cause the client to suspend any running tasks of the 0-project as soon as new work can be fetched for any non-0-project. (Well, precisely speaking, it may not suspend tasks immediately. There is a delay for such transitions which is configured e.g. in boincmgr's computing prefs, "Switch between tasks every __ minutes".)

I enabled TN-Grid on my computers now, see this thread. Caveat: While TN-Grid has enough work in principle, its work generator produces at a distinctly limited pace.

 

[Assimilator1]

Backup for all the family pictures (downtime tolerable), backups For home systems, such as kodi boxes (downtime tolerable), DVR media storage for my 3 year old such as Clifford, Sesame Street, peppa pig, etc (mission critical, HA probably needed ). I don’t want to imagine the fallout if we couldn’t watch Clifford at the appointed time

OMG man, don't risk it!!!

Couldn't you back up the DVR stuff to one of your crunchers? Double redundancy

 

[biodoc]

Maybe they had cases of proper returns of i686-pc-linux-gnu results from other hosts. My hosts might still be "special" in some obscure way.

I went through my results and found the same thing with tasks with "Mini_Protein_binds_IL1R_COVID-19" in the name. I did find a few other tasks from different projects that completed successfully with the linux i686 app.

 

[StefanR5R]

On Rosetta application v4.12 and general work queuing:

4.12 was tested on Ralph but not thoroughly enough. We wanted to get it out anyway so that we can start working on the scaffolds. Time is important. We've been trying our best to get this next app version pushed out. But want it thoroughly tested now since we are still able to get important COVID-19 work done on R@h with 4.12.

Yes, the people behind the scenes are working hard at producing work and analyzing results but there will be plenty of work to come in the near future. This app update should help. We are also trying to recruit more researchers in the IPD to use R@h for their computing rather than relying just on our local options. This new update should allow many more researchers to work on a vast range of interesting targets not just related to COVID-19 but other important targets, but of course, COVID-19 has the highest priority. We do not want to just fill the queue with jobs that we feel already have enough samples as there are other distributed projects people can help with while we are low on work (hopefully in the near future this will not be an issue even with the amount of computing we have at hand).

(source)

 

[Endgame124]

For these Rosetta mobile WUs, anyone know if will they run on a raspberry pi? I've got several in the house that are generally idle (they have kodi installed to stream HD homerun video occasoinally) and I wouldn't mind setting them up for rosetta if they would work.

edit: looks like you can, but my old pi 2s will not work (pi 4 4gb preferred) and it does not work on raspbian (no doubling up with Kodi):

How to Fight Coronavirus With Your Raspberry Pi

Donate Pi’s processing power to medical research.

www.google.com

I've decided to give this a shot with one of my Pi 3s. It will require rearranging some systems so its probably going to take a day or two, but I'll almost certainly be able to get it running by the end of the weekend. Then I'll be able to (roughly) compare the rosetta performance per watt of a Core2Quad, vs an A10-5800k, vs a Raspberry Pi 3. The pi is interesting as you can turn off a lot of features on the board (Wifi, hdmi, usb, leds, etc) so they don't draw power, meaning about the only thing the board will be doing is processing Rosetta. If the Pi is anywhere near useful on a per watt basis, I'll pickup either a 2 GB or 4GB Pi 4 and test that as well.

 

[Assimilator1]

What CPU do they use?

 

[Endgame124]

What CPU do they use?

A 1.2GHz quad-core ARM Cortex-A53 CPU

 

[Endgame124]

Well, it was surprisingly easy just to add boinc to my existing Kodi Xbian boxes - this link lays out the process:

Turning Your Raspberry Pi Into a Science Research Station Via BOINC - DZone IoT

In this tutorial, we look at how to use the BOINC software installed on a Raspberry Pi to create your own personal science research station. Check it out!

dzone.com

Downside is there is currently no work available for portable devices, and I'm not 100% certain that the raspberry pi qualifies for that work anyway. I'll give it some time for rosetta to upload another batch of mobile jobs and see what happens.

Edit:
This page said ARM (raspberry pi) is supported for Rosetta, so now it's just a matter of WUs.

Choosing BOINC projects

boinc.berkeley.edu

 

[Endgame124]

I joined Asteroids@home with my pi, and have work processing, so I know the pi can process work. It's currently running a single asteroids task, using 1 CPU, and is using 1 Watt of power. We will see how it handles rosetta, but it certainly will be a low power draw.

Edit: With 4 asteroids tasks, its using 3 Watts of power.

Edit 2: Looks like rosetta isn't going to happen on the Pi 2s or Pi 3s:

Rosetta for Portable Devices needs 1907.35 MB RAM but only 770.35 MB is available for use.

 

[Howdy]

Someone needs to take these "i686-pc-linux-gnu" and place them someplace uncomfortable on their body. I think they put me in the penalty box making my machine crunch them!! I'm not chasing points but for poop's sakes give me at least a hundo for the 6hr effort!!

 

[StefanR5R]

If you get low credits for Rosetta v4.12 i686-pc-linux-gnu work, and the jobs take 4 hours longer than the target CPU time which is configured at your web prefs, then the low credit reflects a fault in this application version which makes it unable to generate more then one or a few "decoys", IOW is accomplishing only a fraction of the science compared to the x86-64 version. Alas this bug is not yet fixed in Rosetta v4.15 which is in alpha testing stage at Ralph@home.

Also, I still haven't figured out a way to keep i686 jobs away. Perhaps app_config.xml works for this.

I have very large core counts per computer, therefore it is currently worth for me to run Folding@home on them, and that's what I am doing for now instead of Rosetta.

 

[biodoc]

I was thinking about seting up an app_config.xml file to eliminate the 32-bit apps and discovered these lines in the sched_request_boinc.bakerlab.org_rosetta.xml file.

<platform_name>x86_64-pc-linux-gnu</platform_name>
<alt_platform>
<name>i686-pc-linux-gnu</name>
</alt_platform>

After more digging around I found this option for the cc_config.xml file

<no_alt_platform>0|1</no_alt_platform>
If enabled, the client will run applications only for its primary platform. For example, a Win64 machine will run only Win64 apps, and not Win32.

I incorporated the new line into the options section of the cc_config.xml and restarted boinc.

<cc_config>
<log_flags>
<task>1</task>
<file_xfer>1</file_xfer>
<sched_ops>1</sched_ops>
</log_flags>
<options>
<allow_multiple_clients>1</allow_multiple_clients>
<allow_remote_gui_rpc>1</allow_remote_gui_rpc>
<report_results_immediately>1</report_results_immediately>
<no_alt_platform>1</no_alt_platform>
<use_all_gpus>1</use_all_gpus>
</options>
</cc_config>

After restarting boinc, I requested tasks for Rosetta and it looks like I'm running only 64-bit jobs. The <alt_platform> line is now missing in the sched_request_boinc.bakerlab.org_rosetta.xml file.

EDIT: This is the source where I found the option for <no_alt_platform>.

 

[StefanR5R]

Are there any boinc projects out there which have x86-32 Linux binaries, but no x86-64 binaries? ... Perhaps amend cc_config.xml like so:

<!-- set to 0 when switching to a project which requires x86-32 support -->​

<no_alt_platform>1</no_alt_platform>​

PS, @biodoc, your advice is priceless, as always.

 

[biodoc]

Are there any boinc projects out there which have x86-32 Linux binaries, but no x86-64 binaries?

Yes, climateprediction,net uses 32-bit apps exclusively.

 

[StefanR5R]

A post in Ralph's message board on the background of recent Rosetta application updates:

The COVID-19 related updates to the 4.13 rosetta and rosetta_for_devices and 4.14 rosetta linux versions include a new target clash energy and fold-tree options to improve the interface design protocol. It will allow sampling with the ACE2 binding helix motif fixed along with the target and a fast target clash energy check. Successful binders using this protocol are already being tested and optimized in the wet lab but this will allow us to design many more candidates to increase our chances of designing successful binders with the most promising binder properties.

From Rosetta's message board, thread "Largest proteins ever run on R@h coming soon": Simulated proteins are getting larger, resulting in more cases of individual tasks either exceeding the target CPU time, or finishing earlier. This makes it difficult for boinc-client to manage the work buffer. Hence, don't set the buffer too large. In addition, the poster reminds to set the work buffer even smaller when you reconfigure the target CPU time in your web prefs.

In the quest for knowledge about COVID and other proteins, R@h is adapting to meet the challenge.

I wanted to try to help everyone be aware that the project is preparing some of the largest protein WUs even run on R@h. These will take much longer to run per model than smaller proteins. This is going to make "estimated runtime remaining" very difficult to show accurately. It will increase the likelihood of tasks running longer than your WU runtime preference, especially if you have a runtime preference that is less than the 8 hours default. To accommodate, the watchdog will be napping longer that he used to. Only ending WUs that have run more than 10 CPU hours longer than the runtime preference.

These long-running models are going to result in a high degree of variation in runtime between tasks. You might see one task granted nearly twice as much credit as another. That because it ran two models rather than one. Credit should generally be proportional to the amount of CPU time invested in the task.

This high variation in runtime is going to present a challenge for the BOINC Manager in deciding how much work it should be requesting. You can help the BOINC Manager avoid pulling down too much work if you adjust your preferences for how much work to store to be under a day.

Also, several of you have recently reported work units that have completed before their runtime preference. This is going to become more common with these long-running models as well. As an example if you have the default 8 hour runtime preference and the first model takes 5 hours to complete, then 5 hours is where it will stop and report back because a second model would exceed the preference. The Project Team prefers you leave the runtime preference unset, which presently results in 8 hour runtimes. But if these high variations in runtimes are presenting problems for you, I should point out that setting a longer runtime preference will generally result in more consistent completion times. Just beware that runtime preference changes are applied to your existing downloaded work units as well as new work requests. So I always suggest making changes only when you have settings for a small work cache, and to only change the runtime preference a couple of notches at a time, so the BOINC Manager has time to see WUs complete with the new runtimes. This helps it request the amount of work that matches your preferences.

The work units that were running very short models and ending when they reached 1,000 models (before their runtime preference), or other very round numbers, have been adjusted to allow a larger maximum number of models. This will help them fill out their runtime preference, using the additional time to compute more models. This will help runtimes of this type of work unit to be more consistent.

And a follow-up: 4 GB RAM per task is required for some models.

These larger than usual jobs come from our Robetta structure prediction server. They may or may not be related to COVID-19. The spike protein which was modeled in late January and early February, and continues to be modeled (variants and such), is over 1000 residues, and modeled as a symmetric trimer. The structure has since been determined by Cryo-EM (6vsb, 6vxx, and 6vyb), the latter two from Dr. Veesler's lab at the University of Washington. If the sequence is larger than 1000 residues (2000 is the maximum for a single chain), then the memory bound is set to 4gigs so if your device has enough memory per cpu set to run, then it can run such jobs.

(Found via the SG forum.)

 

[Assimilator1]

Wow! Thanks for the heads up, as it was, even after lowering the cache from 2+2 (fine for other projects) to 2+1 I was still only just making it within the deadlines, with 1 WU due to overshoot later by ~40mins. I aborted some WUs yesterday as they'd overshot, I belatedly read that I should've let them run for the 'watchdog' to end them & I would've still got some credit, doh! .
I've now lowered it to 1+1, hopefully that'll be low enough?

4GB RAM/task, wow! Didn't think the 32GB I fitted to my new rig would be insufficient for many years to come! But 10 threads of those WUs would use 40GB of RAM! Lol, well, glad it's going to get good use!
How does BOINC manage these tasks? Does it blindly d/l WUs & if RAM runs out, only run tasks theirs enough RAM for? Or can it detect WU type, & only d/l the big WUs when theirs enough RAM+time?

 

[Endgame124]

I’ll have to start pulling PCs off Rosetta if it doesn’t manage high memory tasks well. My laptop only has 4gb of ram, and my core 2quad is topped out at 8gb

 

[StefanR5R]

How does BOINC manage these tasks? Does it blindly d/l WUs & if RAM runs out, only run tasks theirs enough RAM for? Or can it detect WU type, & only d/l the big WUs when theirs enough RAM+time?

When a workunit is created at the project server, there are AFAIK per-host requirements for disk space and RAM embedded into the WU properties, besides other resource requirements. When a host requests new work at the project server's scheduler, these requirements are checked against what the host claims to offer to the project server, and based on that the scheduler will assign a task or won't.

However, these checks can only go so far: Firstly, the project admin may not have had all necessary information to predict the resource requirements of a new batch of workunits. Secondly, I understand the client and server can base these checks only on the resources of the host in its entirety, but can't really take into consideration (or even predict) how the host will handle a workload of n concurrent tasks.

The latter, instead, happens ad hoc locally on your host while the tasks are running. The client monitors how much disk and RAM each task is really using, and how much disk and RAM is really free for BOINC to use at any given time. When RAM usage by tasks gets too high, the client will suspend one task after another. When disk usage gets too high, the client may see itself forced to abort tasks eventually.

In extreme cases, the client may be unable to react on excessive RAM usage early enough, and the operating system's kernel may kill a task with high memory allocation.

I’ll have to start pulling PCs off Rosetta if it doesn’t manage high memory tasks well. My laptop only has 4gb of ram, and my core 2quad is topped out at 8gb

TN-Grid may be a good alternative if you are specifically looking for COVID-19 related CPU (non-GPU) work.

 

[Assimilator1]

Good to know, thanks .
Actually I remember now, a week or 2 ago I had a few tasks suspended due to lack of memory, was very surprised with 32GB of RAM! lol

 

[Endgame124]

When a workunit is created at the project server, there are AFAIK per-host requirements for disk space and RAM embedded into the WU properties, besides other resource requirements. When a host requests new work at the project server's scheduler, these requirements are checked against what the host claims to offer to the project server, and based on that the scheduler will assign a task or won't.

However, these checks can only go so far: Firstly, the project admin may not have had all necessary information to predict the resource requirements of a new batch of workunits. Secondly, I understand the client and server can base these checks only on the resources of the host in its entirety, but can't really take into consideration (or even predict) how the host will handle a workload of n concurrent tasks.

The latter, instead, happens ad hoc locally on your host while the tasks are running. The client monitors how much disk and RAM each task is really using, and how much disk and RAM is really free for BOINC to use at any given time. When RAM usage by tasks gets too high, the client will suspend one task after another. When disk usage gets too high, the client may see itself forced to abort tasks eventually.

In extreme cases, the client may be unable to react on excessive RAM usage early enough, and the operating system's kernel may kill a task with high memory allocation.


TN-Grid may be a good alternative if you are specifically looking for COVID-19 related CPU (non-GPU) work.

TN grid doesn’t show up on the boinc list: https://boinc.berkeley.edu/projects.php

Is it not a boinc project?

 

[TennesseeTony]

TN-Grid

You will need to enter the address manually (copy/paste) rather than choose from the list.

 

[Orange Kid]

TN grid doesn’t show up on the boinc list: https://boinc.berkeley.edu/projects.php

Is it not a boinc project?

It is on our list