Technical News, and SETI@home daily stats for 27. - 29.12.2007.

[Rattledagger]

Tweenday One (Dec 26 2007)

The weekend was a difficult as we kept splitting noisy/fast work, so our back-end production was running full speed most of the time, clogging several pipes, filling some queues, emptying others, etc. We were able to keep reaching our current outbound ceiling of 60 Mbits/sec, so despite the problems we were sending out work as fast as we could otherwise. That's good, but bigger pipes would be better. Also one of the assimilators was failing on a particular result. We're not sure why, but I deleted that one result and that particular dam broke. Some untested forum code was put on line which also wreaked minor havoc. Not my fault.

Anyway.. this is a short mini week for us in between Xmas/New Year's. Since we weren't around yesterday, we had our normal weekly outage today. Also took care of cleaning some extra "bloat" in our database. About 20% of the rows in the host table were hosts that last connected over a year ago and ultimately never got any credit. We blitzed all those.

Upon restarting everything this afternoon after the outage I noticed the feeder executables had disappeared sometime around 3-4 days ago (luckily images of the executables remained in memory since we had no downtime over the weekend). We have snapshots on that filesystem so recovery was instantaneous, but the initial disappearance is mysterious and a bit troubling.

- Matt

Tweenday Two (Dec 27 2007)

("Tweenday" referring to the scant few work days between Xmas and New Year's holidays).

As we progress in our back-end scientific analysis we need to build many indexes on the science database (which vastly speed up queries). In fact, we need and hope to create 2 indexes a week for the next month or two. Seems easy, but each time you fire off such a build the science database locks up for up to 6 hours, during which there will be no assimilation and no splitting of new workunits. Well, we were planning to build another index today but with the frequent "high demand" due to our fast-return workunits the ready-to-send queue is pretty much at zero. So if we started such an index build y'all would get no work until it was done. We decided to postpone this until next week when hopefully we'll have a more user-friendly window of opportunity.

In the meantime, I've been trying to squeeze more juice out of our current servers. I'm kinda stumped as to why we are hitting this 60 MB/sec ceiling of workunit production/sending. I'm not finding any obvious I/O or network bottlenecks. However, while searching I decided to "fix" the server status page. I changed "results in progress" to "results out in the field" which is more accurate. This number never did include the results waiting for the redundant partners to return. So I added a "results returned/awaiting validation" row which also isn't exactly an accurate description either but is the shortest phrase I could think up at the time. Basically these are all the results that have been returned and have yet to enter the validation/assimilation/delete pipeline, after which it is "waiting for db purging." To use a term coined elsewhere, most of these results, if not all, are waiting for their "wingman" (should be "wingperson"). At this point if you add the results ready to send, out in the field, returned/awaiting validation, and awaiting db purging, you have an exact total of the current number of all results in the BOINC database. Thinking about this more, to get a slightly more accurate number of results waiting to reach redundancy before entering the back-end pipeline you take the "results returned/awaiting validation" and subtract 2 times the workunits awaiting validation and subtract 2 times the workunits awaiting assimilation. Whatever.. you get the basic idea. If I think of an easier/quicker way to describe all this I will.

Answering some posts from yesterday's thread:



>>> Missing files like that prompt me to make an immediate fsck on the filesystem.


Very true - except this is a filesystem on network attached storage. The filesystem is propietary and out of our control, therefore no fsck'ing, nor should there be a need for manual fsck'ing.


>>> Why are the bits 'in' larger than the bits 'out'?


In regards to the cricket graphs, the in/out depends on your orientation. The bytes going into the router are coming from the lab, en route to the outside world. So this is "outbound" traffic going "into" the router. Vice versa for the inbound. Basically: green = workunit downloads, blue line = result uploads - though there is some low-level apache traffic noise mixed in there (web sites and schedulers).

- Matt

#____Total Work Done____Todays WD_______AWD________overtake________Team-name
01______387.481.708______546.003______543.883______impossible______SETI.USA
02______318.028.066______664.806______626.727______impossible______SETI.Germany
03______134.818.058______160.590______165.085______impossible______L'Alliance Francophone
04______115.220.617_______64.415_______65.308______impossible______BroadbandReports.com Team Starfire
05______112.515.236______147.383______137.782______impossible______BOINC Synergy
06_______96.971.240_______39.096_______44.546______impossible______Czech National Team
07_______90.378.170_______91.154_______92.533______impossible______SETI@Netherlands
08_______79.976.001______118.650______118.782______impossible______The Knights Who Say Ni!
09_______42.646.042_______31.008_______43.213______impossible______Overclockers.com
10_______40.909.812______-54.645______-46.204________885 days______OcUK - Overclockers UK
11_______39.985.243_______32.333_______32.048______impossible______Team Art Bell
12_______33.508.452_______12.533_______10.753______impossible______Team 2ch
13_______28.802.060_______-2.318________3.590______impossible______The Planetary Society
14_______28.287.723_______36.978_______32.987______impossible______Team MacNN
15_______23.802.900______-83.537______-85.800________277 days______BOINC.Italy
16_______12.837.318______-50.930______-49.576________259 days______Ars Technica
17__________230.429_______45.677_______52.338______impossible______Team China
18_______73.096.318______154.854______155.612______notanoption_____TeAm AnandTech
19__________-74.320________5.529_______-2.243______impossible______SETI@Taiwan
20_________-990.430______-49.510______-54.668______impossible______Universe Examiners
21_______-2.084.902______-41.784______-33.208______impossible______Phoenix Rising
22_______-3.946.191_______63.868_______56.060_________70 days______Team Starfire World BOINC
23______-14.302.008______-31.393______-34.237______impossible______Canada
24______-14.805.026______-63.459______-62.640______impossible______Dutch Power Cows
25______-16.586.853______-60.538______-63.359______impossible______Amateur Radio Operators
26______-16.881.954______-74.751______-74.672______impossible______PC Perspective Killer Frogs
27______-16.971.108______-99.851______-95.238______impossible______Hewlett-Packard
28______-20.971.984______-30.101______-28.281______impossible______UK BOINC Team
29______-23.066.108______-59.075______-59.840______impossible______Team NIPPON
30______-23.265.974______-86.155______-82.273______impossible______Team MacAddict
31______-23.596.275______-28.219______-28.843______impossible______US NAVY
32______-23.827.124______-41.587______-43.077______impossible______BOINC SETI@home RUSSIA
33______-23.879.654_______29.761_______16.653______1.434 days______AUSTRIA - NATIONAL - TEAM
34______-24.395.676______-53.808______-52.348______impossible______BOINC@AUSTRALIA
35______-25.518.030______-56.920______-58.881______impossible______BOINC@Denmark
36______-25.992.218______-76.667______-70.101______impossible______Hungary
37______-26.216.355_____-106.856_____-104.523______impossible______2CPU.com
38______-26.850.754_____-119.687_____-119.445______impossible______Planet 3DNow!
39______-27.158.057______-32.629______-34.146______impossible______U.S.Air Force
40______-32.042.301_____-110.052_____-110.054______impossible______Portugal@Home
41______-33.476.282______-90.486______-89.479______impossible______SETI.hr
42______-34.125.861______-85.791______-83.207______impossible______SETI@klamm.de
43______-37.531.988_____-109.018_____-110.137______impossible______Team EDGE
44______-37.640.053_____-108.210_____-106.608______impossible______HispaSeti & BOINC
45______-37.936.496______-96.474______-92.429______impossible______BOINC.SK
46______-38.323.760_____-135.139_____-135.028______impossible______Picard
47______-38.863.109_____-100.008______-98.143______impossible______SETI Sverige [Sweden]
48______-39.195.485_____-138.374_____-138.997______impossible______LittleWhiteDog
49______-40.676.648______-63.864______-61.646______impossible______BOINC@Poland
50______-42.324.148_____-114.021_____-112.520______impossible______BOINC UK

Appart for Anandtech's stats, it shows how much more/less than Anandtech.
Also shows based on Average Work Done how many days for Anandtech to overtake the team, or be overtaken by a team behind...



There's been big problems downloading wu's the last couple days due to hitting some sort of bandwidth-limit, and this also means difficulty downloading stats...
So, these are average for 3 days...

As for most "common" AR, would guess on 0.3something, but not sure...


Anyway, I'll have to fix a screwed-up OS-install :frown: so most likely these are the last stats in 2007.

So, happy new year in advance

 

[petrusbroder]

Thanks for the stats and the info, Rattledagger!

Happy New Year to you and thanks for all the effort in 2007! :gift:

 

[bryanW1995]

we need to get some of our friends back in january after the f&h xmas race is over. taiwan is not giving up!

 

[petrusbroder]

Well, the TeAm will get a few of my comps back ...

OTOH: it is nice to see that the production has stabilized around 150K - 160K, which is considerably more than the 90K - 110K we did before the King's Gauntlet Race. At least we are climbing (albeit slowly) and not stepping down any more.

 

[bryanW1995]

yeah, the gauntlet helped us tremendously.

man, has anyone else ever noticed that a keyboard is VERY LOUD when you're hung over?

 

[Assimilator1]

LMAO ,type with gloves on

Thx for the info RD
Btw is their any quick & easy way to see the AR of qued WUs? ,I can't find any program to do that