News Rosetta's role in fighting coronavirus

[StefanR5R]

Perhaps FahCore was killed due to memory pressure. (If so, and if the kernel did this, then this left a message in dmesg.)

Of the nine FahCore_22 processes which I have running, three are occupying 1.8 GB memory right now (resident; virtual memory is up to 35 GB), the others less. Make sure that the fraction of memory configured for BOINC is low enough to leave space for the OS and for F@h.

Still, there may be nightly cronjobs for housekeeping on the system which can suddenly consume a lot of memory too. Make sure you have swap space for such peak situations.

 

[Endgame124]

Are there any guides on configuring and troubleshooting boinc / Rosetta on Linux? I setup my core 2quad (RHEL 7) with boinc, and the client works, but it never downloads data. Not seeing any logs in the boinc home directory or in /var/log saying what is going wrong.

 

[Assimilator1]

96 Gig of RAM! Holly sh*t! lol

I've fired up my 2nd rig again & got it crunching R@H for now, and I noticed it was 'pausing out of memory' on a couple of threads, but I think that's only happening when I'm at the machine, 'not in use' RAM is set to 90% (just increased 'at use' from 50 to 75%, not that it'll make much difference).
My R@H WUs are using ~250-350MB RAM atm, how much RAM are you guys seeing being used by it?

Stefan
Thanks for all the useful posts, and stats
Good to see they seem to be coping well so far, tbh I'm surprised they haven't been sucked dry yet!
Although they are getting low :-

Computing status
Work

Tasks ready to send

8069

 

[Assimilator1]

Here's a question for you guys, a friend of mine has no discrete graphics card atm, only an i5 (4670k he thinks) to crunch with, (I got him signed up to F@H TA , nil_point).
We know that F@H CPU scores are low, so would he be better off crunching Rosetta? Thinking mainly science wise, but also to a lesser degree ppd wise. Is this even answerable?

 

[Markfw]

Here's a question for you guys, a friend of mine has no discrete graphics card atm, only an i5 (4670k he thinks) to crunch with, (I got him signed up to F@H TA , nil_point).
We know that F@H CPU scores are low, so would he be better off crunching Rosetta? Thinking mainly science wise, but also to a lesser degree ppd wise. Is this even answerable?

How about 100 gig used for 107 tasks ?

 

[Assimilator1]

Crazy! lol

Any clue to my question?

 

[Markfw]

Crazy! lol

Any clue to my question?

Both will work on the CPU, but as you see Rosetta sucks a lot of ram. F@H scores really low on CPU. But every bit helps, its up to him.

Edit: And on my dual 7601 with 256 gig ram, its using 123 gig for 128 tasks.

 

[StefanR5R]

Are there any guides on configuring and troubleshooting boinc / Rosetta on Linux? I setup my core 2quad (RHEL 7) with boinc, and the client works, but it never downloads data. Not seeing any logs in the boinc home directory or in /var/log saying what is going wrong.

boincmgr advanced view -> Tools -> Event Log... brings up the log. The log content can be adjusted in cc_config.xml. (In some installations, this file already exists fully populated from the outset. In others, the file does not exist at all and needs to be created. After changes to the file, most changes are accepted after boincmgr advanced view -> Options -> Read config files, some need client restart. Documentation of the config options: boinc wiki. The log is supposedly also written to stdoutdae.txt in the client data directory.

how much RAM are you guys seeing being used by it?

I see minirosetta using typically 350 MB/task, rosetta typically 1.3 GB/task, but sometimes more. There is no way to configure a computer to receive minirosetta only --- except maybe by means of app_info.xml, which I haven't tested.

According to the rosetta forum, some rosetta batches require 2 GB.

We know that F@H CPU scores are low, so would he be better off crunching Rosetta? Thinking mainly science wise, but also to a lesser degree ppd wise. Is this even answerable?

There is no good way to compare points between projects.

Regarding points for the TeAm: While we are in a 28-day race in Folding@home, we are not currently in a specific Rosetta@home competition. We are in a year-long contest and an eternal contest though. ;-)

Good to see they seem to be coping well so far, tbh I'm surprised they haven't been sucked dry yet!
Although they are getting low :-

Computing status
Work

Tasks ready to send

8069

They always have a relatively steady "ready to send" count, and it is generally much lower than the "in progress" count. Therefore I believe they are constantly generating new jobs. But I am seeing more "No tasks sent" messages in the client logs now, which I think means that the rate of requests for new work is beginning to outpace the rate of work generation.

I hope more people find out that they can increase target task run times CPU times. And I hope fewer people fall for the urban myth that 4-h target runtime CPU time is the next best thing after sliced bread.

PS,
serverstatus shows 0 tasks ready to send at this moment.

 

[biodoc]

Not seeing any logs in the boinc home directory or in /var/log saying what is going wrong.

In addition to @StefanR5R 's comments, in Linux mint the boinc log is written to /var/log/syslog mixed in with other system messages. It's a bit messy but that's what we have to deal with.

 

[StefanR5R]

In Mint 18.3, it's /var/log/{boinc,boincerr}.log.

In OpenSUSE 15.0 and 15.1, it's mixed into /var/log/messages. What's more, the log is swamped with "No protocol specified" messages from boinc. This tells me that the logging affairs are not organized as intended.

(sudo journalctl -u boinc-client works, except that the "No protocol specified" spam hides everything useful.)

Edit: upstream boinc bug re: "No protocol specified" spam

I had a few looks just now in Mint 18.3 but didn't find a special configuration for systemd/journald which diverts boinc's log into its own file. But I don't know how systemd/journald work and are controlled. My main computer doesn't have them yet, and there boinc logs to /var/lib/boinc/stdoutdae.txt, like in the upstream documentation.

 

[Endgame124]

In Mint 18.3, it's /var/log/{boinc,boincerr}.log.

In OpenSUSE 15.0 and 15.1, it's mixed into /var/log/messages. What's more, the log is swamped with "No protocol specified" messages from boinc. This tells me that the logging affairs are not organized as intended.

(sudo journalctl -u boinc-client works, except that the "No protocol specified" spam hides everything useful.)

Edit: upstream boinc bug re: "No protocol specified" spam

I had a few looks just now in Mint 18.3 but didn't find a special configuration for systemd/journald which diverts boinc's log into its own file. But I don't know how systemd/journald work and are controlled. My main computer doesn't have them yet, and there boinc logs to /var/lib/boinc/stdoutdae.txt, like in the upstream documentation.

I've only got 3 lines of errors:
Failed to parse protect system value, ignoring: strict
Unknown lvalue 'ProtectControlGroups' in section 'Service'
Unknown lvalue 'ReadWritePaths' in section 'Service'

Do you have a full sample config file? cc_config.xml does not exist in the RHEL install (guessing it doesn't exist in the fedora one either).

 

[MereMortal]

I've only got 3 lines of errors:
Failed to parse protect system value, ignoring: strict
Unknown lvalue 'ProtectControlGroups' in section 'Service'
Unknown lvalue 'ReadWritePaths' in section 'Service'

Do you have a full sample config file? cc_config.xml does not exist in the RHEL install (guessing it doesn't exist in the fedora one either).

My apologies if you have done/know this, but it wasn't clear from your posts.

I would first verify that everything is working with the client:
systemctl status boinc-client

Also, if you haven't explicitly given yourself permission to run the boinc executables (i.e., you run them as root), they need to be run from /var/lib/boinc. So either cd to that directory before running or use, e.g., boincmgr -e /var/lib/boinc.

 

[borandi]

I put a dual 7642 system (192 threads) onto Rosetta and it only downloaded one workunit. :/

Added Universe@home and I got 196 units from U@H. Still only one from Rosetta.

 

[Assimilator1]

Re server status, weird, I'm still seeing 8069 tasks to send, even after a forced refresh.......

What decides whether you run mini Rosetta or not?
I'm guessing my 2nd rig is running mini Rosetta due to RAM usage, and apparently the Rosetta app is 4.07.
(I need to look at the target task times thing again, although atm my 2nd rig managed to grab some SETI WUs, so I've suspended R@H until SETI hibernates).

BTW, Rosetta@home has a feature not seen at many other BOINC projects: The target run time of the work units can be configured at the user page in Rosetta@home preferences. Default is 8 hours, but it can be set as high as 24 hours. The client then issues server requests less often, obviously. (I have set up mine to 16 hours currently.) A single work unit may analyze several models, one after another, and increasing the target run time means more models are tried by the same work unit until next server contact.

Is their an optimal time beyond which useful science diminishes??

 

[amd6502]

Re server status, weird, I'm still seeing 8069 tasks to send, even after a forced refresh.......

Yeah I'm not getting fresh tasks either. Same on the rosetta forum. That 8069 number might be lagging, or that the tasks ready to be sent are not matched to your OS.

Right now I was just getting started testing my windose laptop against my linix desktop. The windose laptop did get fresh tasks served just a few hrs ago.

I have a quick question:

Does anyone know whether the WU's are more or less uniform?

My laptop (2500u) is taking quite a bit longer than my desktop (8350) to complete. The laptop is saying 80 TeraFlops per WU. For the desktop I can't tell because the tasks have completed and been sent in (cleared locally). It's possible (maybe even likely) that on the laptop the tasks are set to run on minimum frequency.

On my linux desktop I adjust this by toggling the flag /sys/devices/system/cpu/cpufreq/ondemand/ignore_nice_load to 1 when I want the computer to be silent.

 

[Assimilator1]

Must admit I've paused Rosetta on my 2nd rig atm, managed to grab some SETI WUs so I'm doing that until it shuts down tomorrow *sniff*.
What CPUs are those 2? An AMD FX 8350? And?

 

[Endgame124]

My apologies if you have done/know this, but it wasn't clear from your posts.

I would first verify that everything is working with the client:
systemctl status boinc-client

Also, if you haven't explicitly given yourself permission to run the boinc executables (i.e., you run them as root), they need to be run from /var/lib/boinc. So either cd to that directory before running or use, e.g., boincmgr -e /var/lib/boinc.

Well, that's embarassing. I didn't know that the boinc service was separate from the boinc client. I started the service, restarted the client, unchecked the box to suspend while in use (I didn't want to wait to see if it would work), and now its running 1 process. Hopefully it pulls down a few more processes, as it looks like it only uses 1 CPU even though I'm set to use 100% of CPUs on the host.

 

[Markfw]

Well, that's embarassing. I didn't know that the boinc service was separate from the boinc client. I started the service, restarted the client, unchecked the box to suspend while in use (I didn't want to wait to see if it would work), and now its running 1 process. Hopefully it pulls down a few more processes, as it looks like it only uses 1 CPU even though I'm set to use 100% of CPUs on the host.

Rosetta will only give you one until it trusts you. Since my boxes have been on it forever, I can;t remember the requirements. Just wait a couple of days and all will be fine.

 

[Endgame124]

Rosetta will only give you one until it trusts you. Since my boxes have been on it forever, I can;t remember the requirements. Just wait a couple of days and all will be fine.

Awesome. I now have all of my existing hardware, excluding an old Pentium 200 (with MMX! And SLI Voodoo 2s!), engaged in either Rosetta or F@H. Next stop is upgrades and tweaks.

 

[Markfw]

Awesome. I now have all of my existing hardware, excluding an old Pentium 200 (with MMX! And SLI Voodoo 2s!), engaged in either Rosetta or F@H. Next stop is upgrades and tweaks.

DO report back to confirm what I have said... My memory is not what it used to be, but pretty sure on this.

 

[amd6502]

Must admit I've paused Rosetta on my 2nd rig atm, managed to grab some SETI WUs so I'm doing that until it shuts down tomorrow *sniff*.
What CPUs are those 2? An AMD FX 8350? And?

Yes my desktop is an FX 8350 and the laptop is a Ryzen 2500u in a 14" hp. I think windows is likely clocking my laptop at minimum frequency rate which I'm guessing would be around 1GHz or so, since boinc tasks are running at the windows equivalent of nice'd tasks.

 

[Howdy]

Rosetta's role in fighting coronavirus

boinc.bakerlab.org

"

Message 92681 - Posted: 31 Mar 2020, 0:46:18 UTC - in response to Message 92670.


Not only has the coronavirus spiked volunteer interest with our R@h project, but it is also spiking a lot of interest towards R@h within the lab so the communication should definitely get better. A lot has happened in a short amount of time.

1. As mentioned before we spiked ~4x or more in computing in the last week. So far our infrastructure has been handling this okay but we will definitely look into expansion and optimization as we expect many more volunteers to join in the near future. Also, our bandwidth has maxed out on a few occasions but has been okay since.

2. A number of outside groups have approached us to help.

a. The ARM developer community came together and helped port Rosetta for aarch64. The application is being tested on Ralph@home currently and will be released soon along with an announcement.
b. We are in communication with Microsoft to help us integrate and support Rosetta BOINC builds into the Rosetta Commons build and testing infrastructure and possibly with other developments.
c. We are in communication with Charity Engine to possibly use their infrastructure to run GPU machine learning based jobs and to help integrate such capability to R@h.



3. We have plenty of COVID-19 jobs but there may be a pause from time to time to give us a chance to prepare them based on previous results and new targets/methods etc.
4. We hope to update the Rosetta apps that are currently being tested on Ralph soon and plan to do another update in the near future, so 2 application updates are lined up for the near future.
5. With these updates, we hope to recruit more researchers within the lab to use this resource, but in the immediate future, priority will of course be towards COVID-19, specifically for the design of scaffolds and binders to COVID-19 targets as possible therapeutics.

Thanks Everyone!"

Rosetta is like F@H right now, so there may be some interruptions in the future. Apparently they are working on it.

 

[Endgame124]

DO report back to confirm what I have said... My memory is not what it used to be, but pretty sure on this.

I believe the same thing happened with my A10-5800K. I had almost no average points for 72 hours, and now its climbing each time I look at the report. I'll know in a few days if the linux client does the same thing, and I'll try to remember to post my results.

 

[Assimilator1]

Awesome. I now have all of my existing hardware, excluding an old Pentium 200 (with MMX! And SLI Voodoo 2s!), engaged in either Rosetta or F@H. Next stop is upgrades and tweaks.

Cool! I had a Pentium 200MMX (o/ced to 225 MHz), although with a V1 , I don't have those anymore but I still have my Celeron 366 & V2 (not on the Soyo m/brd they were originally in though), not run them in over 10yrs! lol

Yes my desktop is an FX 8350 and the laptop is a Ryzen 2500u in a 14" hp. I think windows is likely clocking my laptop at minimum frequency rate which I'm guessing would be around 1GHz or so, since boinc tasks are running at the windows equivalent of nice'd tasks.

What's nice'd tasks?? lol
Yea something's up with your laptop, I would've thought it'd be much quicker than your desktop, and the 2500u is supposed to have a base frequency of 2GHz.
I'm assuming you're running it off the mains, so maybe the windows power plan for when it's plugged in is virtually the same as when on battery?? Or maybe a bios power plan setting?
This for your 2500u seems oddly low & specific

Measured floating point speed

1000 million ops/se

 

[Endgame124]

DO report back to confirm what I have said... My memory is not what it used to be, but pretty sure on this.

Confirmed - Checked this morning and the core2quad is now using 4 CPUs for 4 processes and using 1.5GB of ram (6GB free). Points should start going up faster as it starts returning WUs now.

 

[Markfw]

Confirmed - Checked this morning and the core2quad is now using 4 CPUs for 4 processes and using 1.5GB of ram (6GB free). Points should start going up faster as it starts returning WUs now.

10 work units required before the bonus starts to kick in