8th Annual BOINC Pentathlon

Discussion in 'Distributed Computing' started by TennesseeTony, Apr 7, 2017.

  1. crashtech

    crashtech Diamond Member

    Joined:
    Jan 4, 2013
    Messages:
    7,585
    Likes Received:
    663
    My efforts to pass the buck have failed miserably, it seems... :p

    Looks like I might have to step up my game even further.
     
  2. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    Less than 13 days til go time, so....bumpin' it.
     
  3. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    Holy Crap, the Marathon project will get announced as soon as midnight UTC! Race/dumping begins on May the 5th (midnight UTC).

    I'm not sure I've ever seen a solid 5 days worth of work get queued for BOINC before, except for the LONG primegrid tasks and WCG perhaps.

    As we finish up the Formula BOINC Sprint challenge during the next 27 hours, be thinking about changing over to the Marathon project for this race! :D
     
  4. StefanR5R

    StefanR5R Senior member

    Joined:
    Dec 10, 2016
    Messages:
    922
    Likes Received:
    446
  5. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    Here is a list of the possible projects, and an in-depth explanation of the craziness that is about to happen. :D I'm stupid-excited! :D It would be GREAT (but not likely) that the overlapping projects could be GPU instead of CPU.

    https://www.seti-germany.de/boinc_pentathlon/25_en_Disciplines.html

    A word of caution. We will be competing against the world. No leagues, like Formula BOINC. We typically place in the mid teens. However, we do have several new power users this year, so who knows, maybe we can get closer to 10th, as opposed to 14th-16th? (bouncing up and down super duper stupid-excited!!!) :tonguewink:
     
    Ken g6 likes this.
  6. iwajabitw

    iwajabitw Senior member

    Joined:
    Aug 19, 2014
    Messages:
    691
    Likes Received:
    110
    Well crap, I have to come up with some kind of cooling plan, if I can to fire at least my GPU's back up.

    Edit: So that's 7pm CST the project gets announced correct? That gives me 2.5 to figure out how to move these things around. The wife is tired of all the heat in the house and I moved everything out of my daughters room into the dining room we never use. The dual 280x's were in our bedroom, she liked that when it was cold outside. Scratching my head because this sounds exciting and I have never participated.
     
    #31 iwajabitw, Apr 29, 2017
    Last edited: Apr 29, 2017
  7. iwajabitw

    iwajabitw Senior member

    Joined:
    Aug 19, 2014
    Messages:
    691
    Likes Received:
    110
    @TennesseeTony Moo, Amicable are the only GPU projects I am not already a member, signing up now. Question and PM the answer if you want. Know any good app_config settings for SETI, I have never crunched them on AMD just Nvidia config files. I only ask instead of looking because of the time constraint.
     
  8. iwajabitw

    iwajabitw Senior member

    Joined:
    Aug 19, 2014
    Messages:
    691
    Likes Received:
    110
    Ok got done. GPU systems ready for this race. Signed up on the new projects, to me, joined our team. Installed projects on each system and almost forgot to install lunatics optimized Seti files for the R9 280x's. Case sides off with a box fan positioned. My goal is to keep them cool enough for the heat not to fill the room and keep the AC running. I use an Ecobee thermostat with room sensors, so I moved them to the great room table with 10ft ceilings away from a sensor. Hope it works! Let's race!
     
  9. StefanR5R

    StefanR5R Senior member

    Joined:
    Dec 10, 2016
    Messages:
    922
    Likes Received:
    446
    Ken g6 likes this.
  10. StefanR5R

    StefanR5R Senior member

    Joined:
    Dec 10, 2016
    Messages:
    922
    Likes Received:
    446
    Argh... Cosmology@Home requires VirtualBox. I am not planning to install it on any of my Linux machines. This leaves my i7-6950X Windows box, but that one has also three GPUs in it. (Edit: And a laptop if it doesn't need to travel, and an i7-4960X but only at night when nobody is around.)

    Edit 2: There is also a legacy application which does not require VirtualBox. I suppose a huge number of users across all participating teams will run this legacy application, because they either do not want or cannot run VirtualBox, or because they are not satisfied with C@H+VB's possibly severe performance hogging.
     
    #35 StefanR5R, Apr 29, 2017
    Last edited: Apr 30, 2017
  11. iwajabitw

    iwajabitw Senior member

    Joined:
    Aug 19, 2014
    Messages:
    691
    Likes Received:
    110
    So Cosmology is CPU only it appears.
     
  12. iwajabitw

    iwajabitw Senior member

    Joined:
    Aug 19, 2014
    Messages:
    691
    Likes Received:
    110
    #37 iwajabitw, Apr 29, 2017
    Last edited: Apr 29, 2017
  13. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    Cosmology. Hmmmmm Well, it is a project in which I have needed to boost my score. And it's not a math project. So why am I not pleased?.....something I'm not remembering yet about that project.

    Could be something petty, like low points, I dunno. I hope they can keep up with the demand for tasks, they certainly are going to be put to the test! :D

    Edit: I just found a note I scribbled to myself on my list of 2016 DC Goals, Cosmology was marked out (I abandoned it ) then the word "Screwy" was beside it. Lol. I recall now they transitioned from 'normal' tasks, to the VM only tasks, maybe that was the screwy part?
     
  14. iwajabitw

    iwajabitw Senior member

    Joined:
    Aug 19, 2014
    Messages:
    691
    Likes Received:
    110
    After the last 5hrs I am throwing in the towel on Cosmology. After trying everything in several links in there forums ot just will not work. Bios settings good, updated virtual box, reinstalled the client several times. What I found that keeps happening is, in the link I posted above there is section that says to remove a certain VM disable line from a Boinc client_xxx.xml file. Well each time Boinc starts, updates or what ever, that line gets rewritten in the xml file. I can't get it to stop doing it. Don't know what else to try.
     
  15. StefanR5R

    StefanR5R Senior member

    Joined:
    Dec 10, 2016
    Messages:
    922
    Likes Received:
    446
    I'll begin testing C@H on Monday...

    https://boinc.berkeley.edu/download_all.php currently offers three versions of BOINC+VB, 64 bit (C@H needs 64 bit):
    • BOINC 7.7.2 + VB 5.1.18 - "development version (may be unstable - use only for testing)"
    • BOINC 7.6.33 + VB 5.0.18 - "recommended version"
    • BOINC 7.4.42 + VB 4.3.12 - "older version"
    Besides these combos, it is of course also possible to install BOINC without VB, and install VB separately (as an alternative on Windows, but the only way on other OSs). Oracle's site currently offers VB 5.1.22 for download.

    According to forum messages, C@H had been made compatible with VB up to 5.1 last summer. Any general recommendations about which combination of versions to prefer?

    Edit:
    I think I will use my Linux machines without VirtualBox in City Run and Swimming (and Sprint or Cross Country if they are CPU instead of GPU) -- and outside of those in Marathon with Cosmology's legacy application which does not require VirtualBox.

    Edit 2:
    The legacy application is furthermore able to run on hosts with 32 bit OS.

    Edit 3:
    Some C@H forum users suggest to install the latest VirtualBox 5.1.x version from Oracle's site. This worked better for them than the VB version which is bundled in berkeley.edu's default BOINC+VB download.
     
    #40 StefanR5R, Apr 30, 2017
    Last edited: Apr 30, 2017
  16. StefanR5R

    StefanR5R Senior member

    Joined:
    Dec 10, 2016
    Messages:
    922
    Likes Received:
    446
    Microsoft provide a tool which detects whether the processor implements the virtualization features needed by Virtual PC (and needed by C@H's VB application), and whether or not these features have been enabled in the BIOS:
    https://www.microsoft.com/en-us/download/details.aspx?id=592

    Edit, May 1:
    This tool does nothing on Win 7 pro SP1. Perhaps the tool is too old.
     
    #41 StefanR5R, Apr 30, 2017
    Last edited: May 1, 2017
  17. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    The tool only works up to Windows 7, but it is certainly a good idea to check the BIOS settings to ensure the hardware is activated.
     
  18. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    Please see the FAQ for Cosmology.

    Being fairly new to using VM's myself, I found the information helpful. Here's how to control how many threads you want the VM to use:

    How can I limit the number of CPUs used?
    camb_boinc2docker is multi-threaded and will use up all available cores which BOINC allows it to. For example, if in the BOINC computing preferences you have set “Use at most 50% CPU time” and you have a 4-core processor, the job will use two of them.

    If for whatever reason you wish to limit the number of cores used without changing the global BOINC CPU usage, you can do so by creating a file called "app_config.xml" in the Cosmology@Home project folder and adding the following text, with the “2” under avg_ncpus replaced by however many CPUs you want to use (thanks to Crystal Pellet):

    <app_config>
    <app>
    <name>camb_boinc2docker</name>
    <max_concurrent>1</max_concurrent>
    </app>
    <app_version>
    <app_name>camb_boinc2docker</app_name>
    <plan_class>vbox64_mt</plan_class>
    <avg_ncpus>2</avg_ncpus>
    </app_version>
    </app_config>

    Notes:

    • If after this BOINC gives an error reading your app_config.xml file, make sure you saved the file with a character encoding apropriate for your system (options to do so may vary by text editor).
    • You will need to restart your BOINC client for this take effect
    • This will only affect jobs started after you created the file (jobs started before will show “X CPUs” but still run using all of them, you can just abort these)
    • Reseting or removing/readding the project will delete this file so you will have to remake it
     
    StefanR5R likes this.
  19. StefanR5R

    StefanR5R Senior member

    Joined:
    Dec 10, 2016
    Messages:
    922
    Likes Received:
    446
    According to some postings at the Cosmology@Home forums, camb_boinc2docker scales well unto 8 cores. Beyond that, it does not utilize all cores fully. YMMV.
     
    TennesseeTony likes this.
  20. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    What effect on system RAM does each VM instance have?

    So for a 16 thread system, you could do 2 8-thread tasks, for example, and change the value of the max_concurrent entry to 2, and the avg_ncpus to 8?

    like so?

    <app_config>
    <app>
    <name>camb_boinc2docker</name>
    <max_concurrent>2</max_concurrent>
    </app>
    <app_version>
    <app_name>camb_boinc2docker</app_name>
    <plan_class>vbox64_mt</plan_class>
    <avg_ncpus>8</avg_ncpus>
    </app_version>
    </app_config>
     
  21. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    It seems vLHC had a limit of 8 as well, but you could conveniently control vLHC via web preferences I believe. I wonder if this is a limitation of the VM client being employed?
     
  22. Ken g6

    Ken g6 Programming Moderator, Elite Member
    Moderator

    Joined:
    Dec 11, 1999
    Messages:
    13,075
    Likes Received:
    494
    Now I'm worried. The WCG server says no tasks are available for OpenZika! So should I dump the ~1 day worth of tasks I've got on FormulaBOINC, or save them (and not buffer any others) for the Pentathlon?
     
  23. StefanR5R

    StefanR5R Senior member

    Joined:
    Dec 10, 2016
    Messages:
    922
    Likes Received:
    446
    Regarding memory requirements with 2 or more guests on the same hypervisor, somebody with actual VirtualBox experience will hopefully chime in.

    What I found by a quick search is that
    • VirtualBox supports "memory ballooning". This just means that the hosts's RAM can be over-subscribed, as long as the guests actually use less RAM than they allocated, which is the typical case.
    • VirtualBox supports "page fusion", i.e. memory de-duplication: Pages of different guests with identical contents are detected by the hypervisor, and mapped to a common physical page.
    https://docs.oracle.com/cd/E26217_01/E35769/html/virtualbox.html
    Apparently, this is subject to local configuration of the hypervisor, and I suppose there are performance implications.

    As for whether or not (a) using virtual cores of a hyperthreaded CPU, (b) running more than one camb_boinc2docker instance at a time improves throughput most certainly needs to be tested. I only skimmed the Cosmology@Home forums so far and haven't noticed a straightforward answer to such questions.
     
    #48 StefanR5R, Apr 30, 2017
    Last edited: Apr 30, 2017
  24. StefanR5R

    StefanR5R Senior member

    Joined:
    Dec 10, 2016
    Messages:
    922
    Likes Received:
    446
    The WCG admins know that the Pentathlon is ahead. I hope they will make tasks available again soon.

    5 days bunkering time may be borderline for WCG anyway. It may need an <ncpus> adjustment in cc_config.xml, and a keen eye on WU deadlines.
     
  25. TennesseeTony

    TennesseeTony Elite Member

    Joined:
    Aug 2, 2003
    Messages:
    2,220
    Likes Received:
    538
    We are a LOOONG way from the City Run (WCG-Zika) so dump them.