BOINC Pentathlon 2022

Page 3 - Seeking answers? Join the AnandTech community: where nearly half-a-million members share solutions and discuss the latest tech.

thigobr

Senior member
Sep 4, 2016
231
165
116
My client is getting some network errors when trying to upload task results to Universe@Home:
Project communication failed: attempting access to reference site [---] Internet access OK - project servers may be temporarily down.
 

StefanR5R

Elite Member
Dec 10, 2016
5,459
7,717
136
Everybody is getting that. The cause is two-fold:
  • The BHspin v2 application has comparably large result files. (Not as large as the currently inactive ULX application though.) But more importantly, each result is associated with 6 (six) result files. Only after all six files of a result were uploaded, the result can be reported. — The upshot: Universe@home has got a lot of inbound traffic, both regarding the volume and the request count, when many people want to report results.
  • On three out of 14 days of the "Obstacle Run", the organizers give an extra incentive to work the Obstacle Run rather than any other of the disciplines, by multiplying all credits of the day by a factor of 2, 2.5, or 3. And today is such a day, and the factor is the maximum of 3.
    The result of these "bonus days" is that many folks don't just run Universe@home normally. Instead they retain results (a.k.a. "bunker" results) in order to report them later whan one of these bonus days happens. (And they hope they get their results validated on that very day too.)
    So, today, people are trying to report results which come from several days of prior computer time.
And so, Universe@home's network is heavily congested.

Now, if more of the folks who had been bunkering were more experienced with bunkering + knew better which project they are dealing with, they would have uploaded those 6 files per each result already much earlier, and would have only suppressed the one request which would report the result until today. (One such request can report hundreds of results at once, so that part accounts for only a tiny fraction of the current traffic.)

Of course the Pentathlon organizers must have been more less aware of this particular issue of Universe@home during their planning. But either they underestimated the extent of that issue (again), or they had been hoping that only the first "bonus day" (or "obstacle") would be as much affected. I for one don't have high hopes for the remaining two obstacle days to go by much better than this one though.
 
  • Like
Reactions: biodoc

mmonnin03

Senior member
Nov 7, 2006
214
213
116
There is 1 file several kb in size, like 5-15kb. The rest are bytes. Result files are tiny. If the app wrote 1 file or zipped it, there'd be so many fewer connections and drive writes.
 

StefanR5R

Elite Member
Dec 10, 2016
5,459
7,717
136
There is 1 file several kb in size, like 5-15kb.
The size of the one "large" file varies a lot. Perhaps similar to how the task duration varies. (U@h puts out batches of workunits, and within one batch, the task duration gradually creeps up from first WUs to last WUs.) Just looking at one random computer of mine, it has got "large" files sized 13...40 kB, and two outliers which are 114 and 160 kB.

Still you are of course right, these are not big file sizes, compared with some other projects (protein folding projects, e.g.). It really is the unlucky split into six individual files = the need for ≈six times as many transactions, compared to the technically feasible minimum, which is causing the congestion. Six times during normal operation, that is. Currently, because so many transactions are failing and retried, the traffic jam is even more severe.
 
Last edited:

thigobr

Senior member
Sep 4, 2016
231
165
116
I see. The node was able to upload and fetch more work now. Before it was out of work.
There's a back off policy implemented but it seems that's not helping that much...
 

voodoo5_6k

Senior member
Jan 14, 2021
395
443
116
The size of the one "large" file varies a lot. Perhaps similar to how the task duration varies. (U@h puts out batches of workunits, and within one batch, the task duration gradually creeps up from first WUs to last WUs.) Just looking at one random computer of mine, it has got "large" files sized 13...40 kB, and two outliers which are 114 and 160 kB.

Still you are of course right, these are not big file sizes, compared with some other projects (protein folding projects, e.g.). It really is the unlucky split into six individual files = the need for ≈six times as many transactions, compared to the technically feasible minimum, which is causing the congestion. Six times during normal operation, that is. Currently, because so many transactions are failing and retried, the traffic jam is even more severe.
As a few hundred tasks were still stuck in my upload queue I had a quick look myself at the file sizes (out of curiosity), before finally getting rid of them ;) I basically see the following pattern per WU (from my current sample of the population):
2 files in the 0.01 - 0.02 KB range,
1 in the 0.1 - 0.2 KB range,
1 at around 1.1 KB,
1 at around 2 KB, and
1 weighing in at something between 100 - 250 KB.

So, yeah, seems like a lot of unnecessary network overhead. Independent of the server's network capacity, this is highly inefficient (and like already said by @mmonnin03 easily remediated). The drive writes however must happen at some point (as within the compressed archive, there are still 6 individual files which need to be uncompressed eventually) although not necessarily on the upload server (and therefore not affecting us). Still, it would be better to generate a larger, single file in the first place, I'd say.
 

Skillz

Senior member
Feb 14, 2014
911
929
136
Resistance is futile. :cool:

Wondering how far you can throw the javelin.

Keep up the good work and get some moose smilies ffs. :tearsofjoy:


forest-gump-wave.gif
 
  • Love
  • Haha
Reactions: Emploi and pipin

Emploi

Junior Member
May 7, 2022
6
9
36
Hi Forest, your mama always said: ‘Life was like a box of chocolates. You never know what you’re gonna get".... :eek:

But, impressive output too Skillz and co crunchers on TaaT. :cool:
 

Skillz

Senior member
Feb 14, 2014
911
929
136
Hi Forest, your mama always said: ‘Life was like a box of chocolates. You never know what you’re gonna get".... :eek:

But, impressive output too Skillz and co crunchers on TaaT. :cool:

We got a 5.5M point bunker that ruined our plans. That's what we got! :tearsofjoy:
 
  • Wow
Reactions: cellarnoise

voodoo5_6k

Senior member
Jan 14, 2021
395
443
116
Hm, and flush with cross-country's ending.

@Skillz Shall then some of us move over from PG to work on Einstein bunkers? And if so, who and when?