BBC CCE

[Assimilator1]

Thanks RD for the info
Makes some sense now

 

[Rattledagger]

Originally posted by: Insidious
It sounds to me like some of these projects are taking "algorithmic shortcuts" that require huge samples to overcome (make the errors due to shortcuts statistically insignificant in the end). That seems to be what I was missing.

IMO, an example of the falacy: "we don't need to hire programmers, we can do it ourselves. We have PHDs ya know"

Hmm....

(a+b)+c != a+(b+c)


Well, this coupled with AFAIK ocassionally Intel rounds-up while Amd rounds-down on the same calculation, and relying on various standard math-libraries that isn't giving the same result across platforms, it's no wonder the end-result can be different at the end...

... but have no idea if also "real" hired programmers would make one or more of these mistakes or not... :beer:


As for taking shortcuts, of course there's shortcuts taken. Example from SETI@Home, they've decided there's little point to cumpute gaussians below certain angle-range and so on, and also using approximations instead of always calculating exact is also done, since for the end-result it doesn't really matter if a signal is reported with power 123,4567 or the "correct" 123,4565 or something.

Not to forget, even Seti_Enhanced does a much more detailed analysing and is therefore 1,5 dB more sensitive, re-doubling would pick-up an extra 0,25 dB sensitivity but this was decided against since little scientific benefit for doubling...


Bringing the thread back to Climateprediction, the low-resolution BBC/CPDN Coupled Model uses maybe 6 months for a full run, while the 3x more sensitive "Seasonal Attribution"-model even this model doesn't have dynamic ocean would use 10+ years to run... So, with adding the neccessary dynamic ocean for a full 160-year-model, it would likely take 15 years or something to finish...

So, 6 months with low-resolution or 10+ years with mid-range resolution...

... Well, it's not really a surprise they're taking some shortcuts there possible, to get something useful now and not in 10 years time... :beer:


BTW, one interesting side-note, not sure on the horizontal resolution in the CPDN-models since different ways to specify, but MET-office has this computer-table over horizontal resolution in km and #levels:

Year____km____level
1972____300____10
1982____150____15
1991_____90____19
1997_____60____38

BBC/CPDN uses level-19, indicating it's 15-years-old resolution we're running at, while "Seasonal Attribution" uses 30 levels...

 

[petrusbroder]

Thanks, RD. The info about CPDN is very interesting ... 10 years+ crunching and that when doing DC - I shudder at the implication of repetitive compexity in the math especially considering in that these models can hit constraints.
OTOH: what today would take 10 years, how fast would it be in 5 years - considering the development of processors, Moore's lwa etc? But that is getting us somwhat off topic ....

 

[Insidious]

I guess I'm not totally onboard with the assumptions that lead to (hypothetically) 10 year work units.

This, IMO is the crux of DC. Designing a method which breaks a huge job into 100,000 small ones. (as opposed to 1,000 medium ones)

Science is only hendered by too many nay-sayers to new ideas.... too many assumptions of impossibility.

When I look at the work of the Folding @ Home project, or the United Devices efforts.... Is 10 years really so preposterous? How long have they been crunching? Just think of the possibilities if this length of study had been assumed in the first place and shortcuts had not been taken to avoid a project that "took too long."

I just don't buy the notion that "it would be too hard without shortcuts"

-Sid

 

[Rattledagger]

Originally posted by: Insidious
I guess I'm not totally onboard with the assumptions that lead to (hypothetically) 10 year work units.

This, IMO is the crux of DC. Designing a method which breaks a huge job into 100,000 small ones. (as opposed to 1,000 medium ones)

Science is only hendered by too many nay-sayers to new ideas.... too many assumptions of impossibility.

When I look at the work of the Folding @ Home project, or the United Devices efforts.... Is 10 years really so preposterous? How long have they been crunching? Just think of the possibilities if this length of study had been assumed in the first place and shortcuts had not been taken to avoid a project that "took too long."

I just don't buy the notion that "it would be too hard without shortcuts"

Well, for one thing, if you're going to slice a CPDN-model into smaller pieces, you'll need to make sure to eliminate any roundoff-differences between platforms, this will likely take some months to accomplish and possibly also give a slower-running model.

When this is done, you'll need to buy more servers to handle the huge increase in upload/download-data... this with money you don't have...

As for the actual model, just like with Folding@home 1 finished 100-generation-model or whatever isn't really usable, you needs to finish many models, let's say 10k finished 160-year-models, each split into 160 pieces, giving a total of 1.6M pieces to hand-out...


But hang on, we've overlooked a very important fact, you can't just hand-out 1.6M pieces, seeing the weather in 1923 depends on the weather in 1922... Meaning, you're down to 10k models again, each using roughly 450 MB memory and taking 1 month to crunch, meaning you're still need 160 months to get 1 model to finish...

Ok, some computers are faster than 1 month/model, and computers will also get faster and faster as things progress. But, also remembers there's a 200-year ocean-spinup in the basis before the 160-year model can even start to run, and if you're also needing to increase resolution on this model it would maybe take 5 years or something just waiting on the spinup being ready before can start handing-out the main 160-year-models...


Could CPDN release a high-resolution-model, of course they can. But, if not mis-remembers there was a hint a long time ago what a high-resolution-model uses 800 MB memory or something, so how many users would you expect would run this high-resolution-model?

Well an indication is this, the BBC-model uses roughly 75 MB and a 160-year-model takes roughly 6 months, and has 92k users with credit and 59k active... "Seasonal Attribution" uses 430 MB and a 1-year-model takes roughly 1 month, and has 867 users with credit and 592 active...


As for Folding@home, they've got 40+ upload/download-servers, all is AFAIK dual-cpu, some is Xeons and some is Opterons, with total capasity over 40 TB. As a comparison, CPDN has AFAIK 3 upload-servers, but not quite sure here since found this post from 2 months ago:

the only internet traffic should be the daily (80KB) trickles (every model-year; 25920 timesteps), and the uploads (5MB) after a model-decade (259200 timesteps).

So the experiment will be no more than 100 MB of uploads spread out over (at the fastest) a 3 month period. That's not including your initial BOINC/Experiment Manager (11MB) and experiment data (12MB) download.

Hmmm, I just realized we have 3TB of uploads coming back to a .5TB server! ;-)

(Carl Christensen)